VCS Monitoring and Troubleshooting Using Brocade Network Advisor Brocade Network Advisor is a unified network management platform to manage the entire Brocade network, including both SAN and IP products. This technical brief describes comprehensive, effective monitoring and troubleshooting features for network administrators to augment their investments in VCS fabric deployments.
CONTENTS Introduction...3 Key Performance Indicator (KPI) Monitoring...3 Network Topology Monitoring...6 L2 Topology... 7 Ethernet Fabric Topology... 8 VLAN Topology... 8 Host Topology... 9 Proactive Event Actions...9 Configuration Change Tracking... 13 Flow Monitoring... 14 AMPP Integrity Checks... 16 Fabric-Level Diagnostics... 17 Conclusion... 18 VCS Monitoring and Troubleshooting Using Brocade Network Advisor 2 of 18
INTRODUCTION Brocade Network Advisor is a unified network management platform to manage the entire Brocade network, including both Storage Area Network (SAN) and Internet Protocol (IP) products. In particular, it provides comprehensive, effective monitoring and troubleshooting features for network administrators to augment their investments in Brocade VCS fabric deployments: Manages a Brocade VCS fabric as a single virtual switch and provides drill-down into the individual Brocade VDX switches for maintenance and operations Highlights Key Performance Indicators (KPIs) that affect network performance by using extensive analytics and detailed drill-downs Helps to proactively manage network issues using performance threshold events and by deploying user-defined event actions Enables visualization of VCS fabric traffic path and network latency monitoring for fault isolation via hop-by-hop inspection Supports Virtual Machine (VM)-level monitoring to identify top-talker applications by leveraging sflow technology and vcenter integration KEY PERFORMANCE INDICATOR (KPI) MONITORING Figure 1. Brocade Network Advisor dashboard with KPIs. The Brocade Network Advisor dashboard provides a high-level overview of the whole network and current status of managed products. It displays KPIs of products in the network using inventory, events, and performance widgets. Users can monitor network health by using several precanned and user-defined analytical widgets for both product and port level measures: Products with top CPU, memory utilization, and temperature readings Ports with top/bottom traffic utilization, packet discards, and packet errors Top flows exchanging traffic in the network All the widgets provide detailed drill-down data and historical trends for effective troubleshooting. Users can also customize the dashboards to create and arrange widgets based on their needs. Network Scope and Time Scope selections at a global level in the dashboard enable network administrators to correlate various data points from different widgets for the given managed products, port set, and time period. By default, the Network Scope drop-down list displays all the discovered VCS fabrics and pre-defined/userdefined product groups. The Time Scope selection ranges from 30 minutes to 30 days. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 3 of 18
Figure 2 shows the selection of user-defined network Aggregation Group to monitor all the fabrics in the aggregation layer of the network. Similarly, you can also select the user-defined Core Network network scope in another custom dashboard to keep an eye on the performance of uplinks from aggregation to core network. Figure 2. Brocade Network Advisor dashboard with network and time scope selections. All the dashboard widgets display the top and bottom monitored values in the widget title, as shown in Figure 3, so that the values are noticeable even when the widgets are minimized. Figure 3. Dashboard widget showing top port utilization. Time-Series widgets display the trend of monitored values for the selected time period. They also support network event indications as shown in Figure 4. With this feature, administrators can correlate spikes and lows of monitored values with network events that occurred during the time interval. Event icon tooltips provide additional details about the events. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 4 of 18
Figure 4. The Time-Series widget shows event indications. When creating port-level dashboard widgets, users can also choose specific port types (such as Inter-Switch Link [ISL], initiator, or target), as shown in Figure 5, to focus on specific segments of the network. Figure 5. Dashboard widget creation using port type selection. When administrators need to monitor Network-Attached Storage-(NAS) related flows, they can create the Top Flows widget and select the storage protocols filters (for instance, Internet Small Computer Systems Interface [iscsi], Network File System [NFS], and so forth), as shown in Figure 6. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 5 of 18
Brocade Network Advisor uses sflow data to analyze the top flows and also leverages vcenter integration to highlight the VM information in the widgets. The Top VM Flows widget displays VCS edge port information, along with the amount of traffic exchanged. The Flow Monitoring section provides more details on how to configure sflow and view the widgets and reports. Figure 6. Storage flow monitoring widget creation. NETWORK TOPOLOGY MONITORING You can discover all the products in the network by specifying the IP addresses in the Discovery dialogue box. Brocade Network Advisor also discovers the physical links between products by using Link Layer Discovery Protocol (LLDP)/Foundry Discovery Protocol (FDP) information. Figure 7 shows how to discover the products in the network by specifying out-of-band management IP address ranges. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 6 of 18
Figure 7. Discovery dialogue box. L2 Topology The L2 Topology view displays physical connectivity between all products in the network, as well as the health status of products. You can view physical port and trunk information link-level tooltips. By using the product icon representations, users can easily distinguish the product type (VCS cluster, chassis, fixed-configuration switch, and so forth). Users can also right-click on the products to launch various troubleshooting tools, such as real-time performance graphs, configuration change tracking, events, and L2 traceroute. Figure 8. L2 Topology view. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 7 of 18
Figure 9 shows two VCS fabrics connected using Multichassis Trunking (MCT) to a core Brocade MLX network. The L2 topology link tooltip provides the MCT uplink details. Figure 9. L2 topology link highlighting MCT details. Ethernet Fabric Topology The Ethernet fabric topology feature within Brocade Network Advisor displays physical connectivity between Brocade VDX member switches within a VCS fabric. Link-level tooltips provide trunk and port details, as shown in Figure 10. You can launch the fabric topology view from the L2 Topology by right-clicking the Go To option in the VCS fabric. Figure 10. VCS fabric topology. VLAN Topology The VLAN Topology view shows physical connectivity details between Brocade VDX switches and other switches in a specific Virtual Local Area Network (VLAN). Figure 11 shows the VLAN uplinked from the Brocade VCS fabric to core Brocade MLX switches. Figure 11. VLAN topology. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 8 of 18
Host Topology The Host Topology view shows the connectivity of a managed host to Brocade SAN and VCS fabrics and traditional Ethernet switches. Link-level tooltips provide more detailed information on the ports used for connections. If the managed host is discovered via the VMware vcenter server, then virtual infrastructure inventory information from vcenter is also shown in the Brocade Network Advisor product tree. This includes virtual data centers, hypervisor hosts, virtual Network Interface Cards (NICs), and the VMs. Figure 12. Host topology. PROACTIVE EVENT ACTIONS Brocade Network Advisor helps network administrators to proactively manage network performance issues such as high link utilization, device CPU and temperature spikes by creating threshold violation events and executing user-defined event actions. Figure 13 shows how to define threshold values for link utilization in the Simple Network Management Protocol (SNMP) Collector wizard: Figure 13. Performance threshold definition. Brocade Network Advisor can create threshold violation events based on arm and re-arm values provided by the user. Users can further define event actions that need to be triggered for threshold violation events or any other event occurrences. Figures 14 through 17 show how to select rising and falling threshold events and port operational status change events. You also need to select source ports for the events in the Event Action wizard. In case of platform-level events (CPU, memory, and temperature spikes), you need to select specific products or all products as sources for the events. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 9 of 18
Figure 14. Rising and falling threshold events selection. Figure 15. Link operational status change events selection. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 10 of 18
Figure 16. Port selection for event actions. Figure 17. Product selection for event actions. Figure 18 shows to how to set up e-mail alerting as an event action: VCS Monitoring and Troubleshooting Using Brocade Network Advisor 11 of 18
Figure 18. E-mail alerting for events. Figure 19 shows how to execute Command-Line Interface (CLI) commands to disable a port based on threshold violation on errors or discards. Similarly, users can audit network configurations and detect configuration anomalies using the Run Policy Monitor action. Figure 19. Triggering configuration deployments for events. Network administrators can also set up various Master Log Filters to monitor events that are occurring in various segments of network. Figure 20 shows that the VCS Events filter is selected to display only VCS fabricrelated events in the network. Figure 20. Master log filters. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 12 of 18
CONFIGURATION CHANGE TRACKING Brocade Network Advisor provides the ability to track the configuration changes against a user-designated baseline configuration for all managed products. With the change tracking feature, you can easily compare the differences between two configuration versions. Configuration events that caused the changes are also correlated and displayed in the comparison view. Figure 21. Configuration change tracking. Figure 22 shows configuration comparison against a baseline and the correlated product configuration events. The Events view shows that the user has changed the sflow collector configuration in the product by using the NETCONF protocol. Figure 22. Configuration comparison with event correlation. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 13 of 18
FLOW MONITORING Brocade Network Advisor supports end-to-end flow monitoring by collecting sflow data from all Brocade products and performing analytics on the collected data. Brocade Network Advisor monitors top talkers, VLANs, VMs, and the network ports used by them. Both VCS fabric-level and Brocade VDX member-level reports are supported. Figure 23 shows the launch points for sflow reports and configuration options. Figure 23. Top flows report. Since sflow is a sampling technology, you need to configure sampling values for the ports. Brocade Network Advisor supports Adaptive Sampling configuration by setting optimal values based on the port speed. Custom sampling values can also be configured. You should add the Brocade Network Advisor server as an sflow collector by following the steps in the sflow configuration wizard. Figure 24. sflow sampling configuration. Top talking flows can be displayed in the dashboard by creating performance monitoring widgets. Figure 25 shows the top IP flows with source, destination endpoints, Brocade VDX ports, and the traffic exchanged between them. Similarly, you can also create top MAC and VM flow widgets. Drill-down provides the top flows trend in report format, as shown in Figure 26. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 14 of 18
Figure 25. Top IP Flows dashboard widget. Figure 26. Top IP Flows report. With Brocade Network Advisor s sflow custom reports feature, the user can create customized flow reports. For example, by specifying the VLANs used by tenants (in data center environments) as filters, you can generate a custom report that displays various top talking traffic flows in the tenant network, in order to identify any congestion, as shown in Figure 27. Network administrators can also use various other filters, such as source/ destination IP address and MAC address. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 15 of 18
Figure 27. Tenant-level Top Flows report. AMPP INTEGRITY CHECKS Brocade Network Advisor provides simplified management of Application Migration Port Profile (AMPP) configurations. During VM migrations from one host to another host, network administrators need to make sure that port profiles are properly applied, based on MAC addresses. If port profiles are configured manually using the CLI, there is a possibility of MAC address or network setting mismatches in VCS fabric clusters. You can use port profile comparison views to troubleshoot such scenarios within a VCS fabric cluster and across VCS fabric clusters. Three types of port profile comparisons are provided: MAC Match: Source and target profiles have the same associated MAC. Name Match: Source and target profiles have the same name. Network Match: Source and target profiles have the same networking characteristics (MAC, VLAN, Quality of Service [QoS], and Access Control List [ACL] values). Figure 28. Port profile comparison showing network setting mismatches. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 16 of 18
Figure 29 shows the port profiles with the same MAC addresses but with similar and mismatched port profile names. Figure 29. Port profile comparison showing name mismatches. FABRIC-LEVEL DIAGNOSTICS Brocade Network Advisor provides VCS fabric-level diagnostics for advanced troubleshooting by leveraging L2 (Layer 2) multipath traceroutes from the switches to display the traffic path. VCS fabric instrumentation also enables Brocade Network Advisor to provide network latency details in the fabric context. Also, if network latency falls above a certain threshold, Brocade Network Advisor can facilitate hop-by-hop inspection to pinpoint the bottleneck. You can also launch real-time and historical performance measure graphs of the ports from this dialogue box, for further troubleshooting. As a result, you can isolate faults more quickly, lowering operation costs. Figure 30. Ethernet fabric traceroute. VCS Monitoring and Troubleshooting Using Brocade Network Advisor 17 of 18
To find the end devices (hosts, VMs, and storage devices) that are connected to a VCS fabric, network administrators can use the Address Finder tool, as shown in Figure 31. Launch points are also provided to view the end device and the port properties in detail. Figure 31. Locating end devices with Address Finder. CONCLUSION In summary, Brocade Network Advisor enables network administrators to more rapidly adopt and operate VCS fabric technology, while lowering the total cost of ownership by providing effective monitoring and troubleshooting tools. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 12/13 GA-TB-492-00 ADX, AnyIO, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript, VCS, VDX, and Vyatta are registered trademarks, and The Effortless Network and The On-Demand Data Center are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may be trademarks of others. Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government