Critical Capabilities for Network Performance Monitoring and Diagnostics

Transcription

1 G Critical Capabilities for Network Performance Monitoring and Diagnostics Published: 1 February 2016 Analyst(s): Vivek Bhalla, Sanjit Ganguli NPMD tools are top of mind among infrastructure and operation leaders looking to manage IT services and network infrastructures. This evaluation of NPMD tool capabilities across five common usage scenarios will help IT leaders to foster better relationships with technical and business stakeholders. Key Findings Many network monitoring teams have yet to successfully make the leap from basic, reactive network availability management to proactive performance management. Network performance monitoring and diagnostics technology has seen incremental innovation during this review period, with core functionality across the vendors remaining largely stagnant. On-premises software and appliances are the dominant delivery models; however, SaaS offerings are slowly becoming available, although they haven't been widely adopted. Many NPMD solutions are assembled from multiple products, which can support the modular adoption of NPMD capabilities; however, this can add significant complexity to procurement and ongoing maintenance. Recommendations Invest in NPMD tools to contextualize value and report on improved service quality and application response time for lines of business. Use NPMD technology to manage network demand and capacity planning, particularly before introducing bandwidth-intensive services, such as voice over IP and unified communications. Gauge the ability of NPMD tools' embedded analytics and contextual user interface drill-down workflows to support IT operations' efforts to become more proactive. Strike a cost-effective balance between probe packet-based inspection methods for missioncritical component granular insight and flow collection techniques for holistic, infrastructurewide coverage.

2 What You Need to Know Unlike their network fault or event management counterparts, network performance monitoring and diagnostics (NPMD) tools enable IT and network operations teams to understand the ongoing behavior of the network and its constituent elements in response to traffic demands and changes in network utilization. Measuring and reporting on network performance is crucial to ensuring that performance stays at an acceptable level. It can also highlight opportunities to enhance business value for internal end users and external customers through improved application delivery. Finally, NPMD tools enable improved network capacity management, thereby optimizing capital investment for networking equipment and services. 1 The client input from which this critical capability analysis was derived came from a survey of end users and from the Gartner inquiry process. Analysis Critical Capabilities Use-Case Graphics Page 2 of 35 Gartner, Inc. G

3 Figure 1. Vendors' Product Scores for the Network Operator Use Case Gartner, Inc. G Page 3 of 35

4 Figure 2. Vendors' Product Scores for the Network Architect Use Case Page 4 of 35 Gartner, Inc. G

5 Figure 3. Vendors' Product Scores for the IT Operations Generalist Use Case Gartner, Inc. G Page 5 of 35

6 Figure 4. Vendors' Product Scores for the IT Operations Manager Use Case Page 6 of 35 Gartner, Inc. G

7 Figure 5. Vendors' Product Scores for the Line-of-Business User Use Case Gartner, Inc. G Page 7 of 35

8 Vendors Automic Streamcore 6.3 and Sysload 6.0 Founded in 1985, Automic Software is a privately held company headquartered in Bellevue, Washington, with European headquarters in Vienna, Austria. Automic's NPMD products, Streamcore v.6.3 and Sysload v.6.0, were evaluated for this research. This product is an on-premises software and appliance offering. Originally focused on workload automation, application release automation and automated service orchestration, Automic added an NPMD offering when it acquired Orsyp in May The acquisition has evolved into a dedicated business unit focused on NPMD. The products are primarily deployed as hardware appliances, although the v.6.3 release introduces a Streamcore virtual appliance. Updates in Sysload to HTML5 provide visually compelling user interface (UI) captures. Typical operator workflows include troubleshooting and optimization use cases, with an innovative approach to summarizing global service performance among many sites. The company offers above-average service delivery monitoring capabilities, but products lack the analytics and integration of most other solutions. Although there are plans to consolidate the solution sets, multiple graphical UIs (GUIs) exist among the Streamcore/Sysload product sets and the rest of the Automic product line. Gartner's survey of end users rated Automic's NPMD solution as joint strongest in terms of maintenance, reliability and ease of upgrading. This survey showed Automic's NPMD solution to be the most expensive in terms of total cost. The same survey indicated Automic's service subscription costs accounted for the greatest percentage of total spending on tooling costs. Of the end users surveyed, 50% indicated that they had upgraded to the current release in The product is well-suited to enterprises that need to monitor large numbers of sites, such as those in the retail, banking and government sectors. CA Technologies Unified Infrastructure Management (UIM) 8.3 Founded in 1976, CA Technologies is a public company headquartered in New York City, NY. CA's NPMD product, Unified Infrastructure Management (UIM) v.8.3, was evaluated for this research. This product is an on-premises software offering. Although CA's history in this market spans several acquisitions, the current offering is predominantly based on the legacy NetQoS and Nimsoft technologies. CA's focus CA UIM v.8.3 is a departure from previous strategy that centered on CA Performance Manager as the primary platform. Requirements can be supplemented by and integrated with CA Network Flow Analysis (NFA) for traffic monitoring, CA Unified Communications Monitor for the monitoring of voice and video technologies, and CA Application Delivery Analysis (ADA), which is focused on packet-level inspection for applications. These three products came from the NetQoS acquisition. Only NFA has been integrated into the UIM platform. CA's OEM relationship with Viavi Solutions for the Gigastor Page 8 of 35 Gartner, Inc. G

9 product has been discontinued, limiting its ability to provide high-end storage options for packet data. The CA solution for NPMD focuses on reporting and workflow to identify and isolate issues. The lack of specific hardware appliances makes this software-based solution flexible and cost-effective; however, deployment complexity can be an issue. Recent enhancements include integration between UIM and NFA, support for big data monitoring and bulk configuration of probes. Gartner's survey of end users indicated that the company's breadth of product solutions is a strength. The same survey indicated CA's implementation, training and professional services costs accounted for the greatest percentage of total spending on tooling, and the vendor was rated the weakest in terms maintenance costs. CA was also rated among the lowest vendors in terms of overall satisfaction. Of the end users surveyed, 33% indicated that they had upgraded to the current release in This product is well-suited to large enterprises and service providers that need multitenancy and a focus on flow and Simple Network Management Protocol (SNMP) monitoring. Cisco Prime Infrastructure 3.0, NAM 6.2 and Collaboration 10 Founded in 1984, Cisco is a public company headquartered in San Jose, California. Cisco's NPMD products, Prime Infrastructure (PI) v.3.0, Network Analysis Module (NAM) v.6.2, Prime Collaboration (PC) v.11.0 and Cisco Connected Analytics (CCA) v.1.0 were evaluated for this research. These products are on-premises software offerings. PI's multiple capabilities span the network configuration, monitoring and the management of Cisco equipment. PC is focused on complete life cycle management of Cisco unified communications (UC) solutions. The NAM is a hardware appliance or module that can be included in many Cisco network devices for handling deep packet inspection (DPI). The solutions offer a common UI, except for the NAM component, and deep life cycle support of all Cisco products on the day they ship. The focus on Cisco technologies limits attention to third-party devices and technology, which are common in today's networks. Recent enhancements typically mirror new Cisco hardware capabilities and additional APIs for integration, including increased support for Wireless LAN monitoring and overlay networks. As part of this suite, Cisco has recently introduced its 1.0 version of CCA (formerly known as Prime Analytics) to provide real-time analytics for network and business intelligence (BI). Although promising, this solution isn't part of many of its NPMD workflows. Gartner's survey of end users rated Cisco's NPMD solution strongly in terms of ease of upgrading, total cost of implementation and total cost of maintenance. The same survey indicated Cisco's NPMD solution to be the third weakest in terms of historical reporting and exporting. Of the end users surveyed, 80% indicated that they had upgraded to the current release in This product is well-suited to global enterprises with heavy investment in Cisco hardware that want to manage the complete life cycle. Capabilities such as hardware-embedded DPI provide NPMD advantages for organizations standardized on Cisco. Gartner, Inc. G Page 9 of 35

10 Corvil Network Data Analytics Platform Tera (9.0) Founded in 2000, Corvil is a privately owned company headquartered in Dublin, Ireland. Corvil's NPMD product, Network Data Analytics Platform Tera, was evaluated for this research. This product is an on-premises appliance. Network Data Analytics Platform Tera is a competitively priced, packet-based NPMD tool that focuses on analyzing high volumes of wire data in stream. The company has focused on improving the modularity of the product with the release of Tera. The new platform includes the ability to add new decoders, analytics, dashboards and data-publishing streams outside the main development cycle in an agile manner. Tera has also extended support to application events that can be decoded and correlated with network alerts data and published in Corvil's analytics stream. In Gartner's survey of end users, Corvil was rated strongest or joint strongest in terms of direction and vision, service and support, maintenance and reliability, ease of implementation, ease of upgrading, cost of maintenance and integration with third-party tools. Of the remaining capabilities, navigation and workflow was the area that end users cited as requiring the most attention. Corvil was also among one of the lowest-ranked vendors in terms of total network elements being instrumented. Corvil was rated among the strongest vendors in terms of overall satisfaction. Of the end users surveyed, 80% indicated that they had upgraded to the current release in The product is well-suited to enterprises in need of short-time-scale and high-granularity network monitoring, such as in the electronic-trading market and other digital businesses. Flowmon Networks Flowmon 7.03 Founded in 2007, Flowmon Networks is a privately owned company headquartered in Brno, Czech Republic. Flowmon Networks' NPMD offering, Flowmon v.7.03, has been evaluated for this research. This product is an on-premises hardware, virtual appliance and SaaS offering. Flowmon is a competitively priced, flow-oriented NPMD tool designed to meet the most demanding of performance and scalability requirements. The vendor is one of the first to support 100G environments with its 100 GbE Flowmon probe. The tool can process and analyze more than 250,000 flows per second. End-user experience visibility on a per-application basis is limited for hosted virtual desktop (HVD) environments, as is packet support, which focuses on flow conversion, as opposed to end-user or timing metrics. Gartner's survey of end users indicated Flowmon s NPMD tool was used to instrument the second highest number of network elements and monitor the third-highest number of flows per second. The same end-user survey scored integration with third-party tools joint lowest. However, more than 60% of those same respondents integrated the tool with a third-party security information and event management (SIEM) tool. This appears to be an area of specialization. Of the end users surveyed, 100% indicated that they had upgraded to the current release in This product is well-suited to midmarket, large enterprise and service providers that need a scalable. flow-oriented NPMD solution with diverse deployment options and an ability to meet security use cases. Page 10 of 35 Gartner, Inc. G

11 Fluke Networks TruView 10.0 and TruView Live 1.0 Sold by Danaher Corp. to NetScout Systems in July 2015, the Fluke Networks entity is headquartered in Everett, Washington. Fluke's NPMD products, Visual TruView v.10.0, TruView Live and OptiView XG were evaluated for this research. These products are on-premises appliance, virtual appliance and SaaS-based offerings. Fluke Networks focuses on network test and monitoring solutions. TruView focuses on workflow that is easy to learn and remember. OptiView XG provides endpoint and additional data to TruView to support centralized troubleshooting analysis. Recent enhancements include a big data store for scalability, enhancements to Advanced Path Analysis, and active testing capabilities for Web and voice over IP (VoIP). Applications such as Microsoft Lync and Citrix environments are also supported. TruView v.10.0 is a single, modular, appliance-based offering that can be repurposed to target multiple data source types, depending on specific requirements. TruView appliances are sized based on network storage and processing. TruView Live, deployed as a SaaS offering, is among the first NPMD products to offer this deployment model, and, although it has yet to be a proven model, it has the potential to be disruptive. Gartner's survey of end users indicated that Fluke's customers favored the ease of use of TruView for troubleshooting, along with the combination of Fluke's flow-based, SNMP and deep-packet visibility. Clients have reported that Fluke's alerting capabilities and its ability to report on business transactions are areas for improvement. Of the end users surveyed, 50% indicated that they had upgraded to the current release in This product is well-suited to large enterprises and service providers that need extensive NPMD capabilities and value ease of workflow via contextual drill-down navigation. Fluke Networks completed the vendor survey and provided customer references prior to the completion of the acquisition by NetScout. NetScout did not respond to requests for supplemental information and/or engage in Gartner's standard procedures to address the contents of this document. Hence, Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with end users who have evaluated or deployed each NPMD product. NetScout has sued Gartner over the content of the 2014 NPMD Magic Quadrant, and that lawsuit is pending. In addition, as noted above, NetScout has refused Gartner's invitation to engage in Gartner's standard procedures to address the content of this document. Genie Networks GenieATM and VM Founded in 2000, Genie Networks is a privately owned company headquartered in Taipei, Taiwan. Genie's NPMD product, GenieATM and VM, have been evaluated for this research. These products are an on-premises hardware and software offering. Gartner, Inc. G Page 11 of 35

12 Genie Networks focuses on network fault and performance monitoring solutions. Genie relies primarily on SNMP, flow-based and proprietary data gathering techniques to present a real-time status of the network. Correlation with fault events and alarm generation against service models are standard features. Genie introduced a new virtual machine (VM) variant of its GenieATM product in August 2015 that offers the features of the appliance-based option as a software image that can be downloaded and installed in a virtualized infrastructure. Multiprotocol Label Switching (MPLS) support was another aspect that was enhanced with a dedicated appliance focused on providing visibility of this technology. The UI continues to be less sophisticated than those of the leading NPMD tools. Gartner has observed that Genie customers continue to report the estimated total cost of the solution to be competitive, compared with most of the other vendors included in this research. Maintenance and reliability also continues to gain positive feedback from end users. These products are well-suited to service providers that require a scalable combined network fault management (NFM) and NPMD solution. Genie Networks did not respond to requests for supplemental information and/or engage in Gartner's standard procedures to address the contents of this document. Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with end users. HPE Network Node Manager i and Real User Monitor 9.25 Founded in 2015, Hewlett Packard Enteprise (HPE) is a public company headquartered in Palo Alto, California. HPE's NPMD products, Network Node Manager i (NNMi) v and Real User Monitoring (RUM) v.9.25 have been evaluated for this research. These products are on-premises software offerings. Designed for network operations specialists and architects and reflecting its predominant use, the HPE solution flexibly supports service delivery monitoring and diagnosis, informed primarily by SNMP and flow data. Packet inspection and additional application context gathering to support complex diagnosis is provided through a data interchange, modeling and process-level integration with HPE's RUM product. Support for application mapping requires that users include the NNMi Smart Plug-In Performance for Traffic for flow-based traffic mapping. However, although HPE has simplified the pricing and packaging of its NPMD solution, it is composed of multiple component products. This results in two distinctly different UIs. Recent enhancements include improved support for mobile application monitoring and network virtualization. Integration with Network Automation presents an interesting differentiator to traditional NPMD solutions. In Gartner's survey of end users, HPE's implementation, training and professional services costs accounted for the lowest percentage of total spending on tooling. The same end users scored HPE poorly in terms of historical reporting and exporting, the ability to commit and deliver on requested enhancements, and the cost of implementation. However, two-thirds of these same end users stated that they anticipated increasing their investment of their HPE NPMD solution, and Gartner has witnessed evidence of improved renewal rates. HPE was rated poorly in terms of overall Page 12 of 35 Gartner, Inc. G

13 satisfaction. Of the end users surveyed, 67% indicated that they had upgraded to the current release in This product is well-suited to large enterprises, especially those with significant HPE IT operations management (ITOM) tooling investments, with deep and varied NPMD requirements. InfoVista VistaInsight for Networks 5.3 and 5View Suite Founded in 1995, InfoVista is a privately owned company headquartered in in Les Ulis, France. Infovista's NPMD offering, VistaInsight for Networks v.5.3, 5View Service Data Manager v.3.4, 5View Netflow v.7.1, 5View Applications v.7.0, and 5View Mediation v.5.2 have been evaluated for this research. These products are on-premises software offerings. The products can be scaled up and deployed, depending on the level of visibility required, including SNMP, flow and packet data. Each of the products in the offering collects and rolls data up incrementally, allowing for large scale. Each part of the offering is multitenant, supporting multiple users in a single master instance of the solution. This architecture, although scalable, comes with complexity requiring a services engagement to deploy. End users would typically be using VistaInsight, which provides a reporting interface to pull usage and performance reports. Due to the number of products, UIs, and implementation nuances, Gartner has observed that InfoVista caters to and has become embedded in large carrier customers. End users report Infovista has a high cost of licensing and implementation. Gartner's survey of end users indicated that InfoVista's NPMD tool was used to instrument the highest number of network elements and monitor the second highest number of flows per second. Gartner has observed that end users report increased satisfaction with the tool's navigation and workflow, as well as contextual drill-down and troubleshooting, both previously cited as product weaknesses. Of the end users surveyed, 40% indicated that they had upgraded to the current release in This product is well-suited to large carriers that require diverse deployment options, multitenancy and extensive reporting from their NPMD investments. NetScout ngeniusone 5.4 Founded in 1984, NetScout Systems is a public company headquartered in Westford, Massachusetts. NetScout's NPMD product, ngeniusone v.5.4, was the product evaluated for this research. The product is an on-premises appliance and software offering. NetScout's singular focus on supporting any and all of the diagnostic needs of service provider and large enterprise network specialists has resulted in a diverse set of hardware appliances and software that is managed through an increasingly (although not completely) cohesive ngeniusone UI. The design and capability of the solution enable it to support deep diagnostic workflows, with some application context where network specialists are enabled more by flexibility than guidance. Adaptive Session Intelligence technology enables the solution to support 40GbE and 100GbE environments, while retaining visibility and keeping storage requirements manageable. The solution requires hardware appliances, so it is sized based on network data storage and processing Gartner, Inc. G Page 13 of 35

14 requirements. Gartner has observed that NetScout customers report the latest version of ngeniusone as having improved service representation. NetScout customers also continued to report limited upgrade paths for hardware elements of the solution, resulting in higher-thanexpected total cost of ownership (TCO). This product is well-suited to service providers and large enterprises with network operations teams and architects that need deep NPMD capabilities at extreme scale to support their own monitoring and diagnostic efforts. NetScout did not respond to requests for supplemental information and/or engage in Gartner's standard procedures to address the contents of this document. Hence, Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with end users who have evaluated or deployed each NPMD product. NetScout has sued Gartner over the content of the 2014 NPMD Magic Quadrant, and that lawsuit is pending. In addition, as noted above, NetScout has refused Gartner's invitation to engage in Gartner's standard procedures to address the content of this document. Niksun NetVCR Alpine 4.5 Founded in 1997, Niksun is a privately owned company headquartered in Princeton, New Jersey. Niksun's NPMD offering, NetVCR running Alpine v.4.5, was evaluated for this research. This product is an on-premises appliance and virtual appliance offering. The product is focused on high-fidelity data capture at speeds exceeding 100 Gb/sec. The top-end platform for 100G support is Niksun's Supreme Eagle, which promises faster processing and more storage than earlier 100G offerings. This ability to process at scale makes the product attractive in environments in which high-speed transactions are prevalent, including organizations that have an Internet of Things (IoT) initiative. In Gartner's evaluation, the product competently handled endpoint, component and link monitoring. However, IT operations analytics (ITOA) capabilities were below par, compared with the leading tools included in this research. The product demonstrates the ability to support the most-scalable environments, including 3G and 4G cellular networks. NetVCR's product strengths appeal to an exclusive group: those that prioritize robust network infrastructures with high transaction rates over other requirements. Additional modules can be used to analyze, report and troubleshoot market data feeds and trade transactions, as well as oversee and administer VoIP systems. The product is well-suited to global service providers, large financial institutions, and the defense and intelligence communities with consolidated IT and security requirements. Niksun did not respond to requests for supplemental information and/or to engage in Gartner's standard procedures to address the contents of this document. Hence, Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's marketing collateral, public information and discussions with end users. Page 14 of 35 Gartner, Inc. G

15 Paessler AG PRTG Network Monitor /2996 Founded in 1997, Paessler is a privately owned company headquartered in Nuremberg, Germany. Paessler's NPMD offering PRTG Network Monitor v.15.3 has been evaluated for this research. The product is an on-premises software offering. PRTG Network Monitor is a competitively priced, consolidated NFM, NPMD and infrastructure monitoring tool that relies primarily on SNMP, flow-based and proprietary data-gathering techniques to present a real-time status of the network. Correlation with fault events and alarm generation against service models are standard features. Latest product enhancements include cloud-based sensors, clustered high-availability, automatic device discovery that recommends the right sensor for the device type detected, the support for mobile apps via a push-notification feature that enables alerts to be sent to mobile devices, support for MongoDB, and increased sensor reporting and analysis. The UI is less sophisticated than the leading NPMD tools; however, a recent overhaul has reduced the disparity. Integration with Google Analytics and an optional ticketing system provide workflow aids beyond the core network monitoring focus. Gartner's survey of end users indicated Paessler's NPMD tool was the most competitively priced in terms of overall cost and annual maintenance. Paessler was also rated among the strongest vendors in terms of overall satisfaction. The same end-user survey scored integration with thirdparty tools joint second lowest, whereas the vendor was ranked among the lowest three for contextual navigation and workflow. Of the end users surveyed, 100% indicated that they had upgraded to the current release in This product is well-suited to small or midsize businesses (SMBs), midmarket businesses, large enterprises and managed service providers that need a competitively priced NPMD solution with a small deployment footprint and tightly coupled architecture. Riverbed SteelCentral Suite Founded in 2002, Riverbed is a privately owned company headquartered in San Francisco, California. Riverbed's NPMD offerings, SteelCentral NPM, Netsensor 3.0, UCExpert 3.0 and Portal 1.0 have been evaluated for this research. The products are on-premises appliance and software offerings. As one of the handful of NPMD vendors with a full-featured application performance monitoring (APM) offering (with some products being included in both), the solution provides a broader, deeper set of application visibility and capabilities. SteelCentral NPM consists of multiple components that include SteelCentral AppResponse 9.5.3, SteelCentral NetProfiler , SteelCentral NetShark , SteelCentral Packet Analyzer and SteelCentral Transaction Analyzer The solution does require hardware appliances, so it is sized based on network data storage and processing requirements. Riverbed continues to make progress on this front, notably with SteelCentral Portal, which is now the primary interface for consolidated APM and NPMD visualization and as the launch pad for detailed diagnostic workflows. Gartner's survey of end users indicated Riverbed rated highest in terms of ease of upgrading and joint highest in terms of ease of implementation. The results with regard to ease of implementation Gartner, Inc. G Page 15 of 35

16 are somewhat at odds with end-user feedback during the past 12 months, particularly where multiple Riverbed products were involved. Riverbed was rated as second highest in terms of maintenance and reliability, and historical reporting and exporting. Riverbed was also rated among the strongest vendors in terms of overall satisfaction. The same end-user survey apportioned the cost of implementation, training and professional services as a percentage of overall NPMD cost as second highest. Of the end users surveyed, 40% indicated that they had upgraded to the current release in This product is well-suited to large enterprises and service providers with NPMD requirements that include integration with APM tooling and are intended to support the needs of network and application teams. SevOne PAS 5.4 and PLA 2.0 Founded in 2005, SevOne is a privately owned company headquartered in Wilmington, Delaware. SevOne's NPMD products, SevOne Performance Appliance Solution (PAS) v.5.4 and Performance Log Appliance (PLA) v.2.0 were evaluated for this research. These products are on-premises physical and virtual appliance offerings. SevOne PAS provides a scalable NPMD solution based on a distributed peer-to-peer architecture. PLA adds the ability to ingest unstructured data, which is a differentiator in the NPMD market. The tool demonstrates significant attention to visualizing and reporting on how network performance affects service and application delivery. Although PAS lacks native non-flow-based DPI collection and analysis, packet-based analytics are served through an OEM and largely nonexistent from standard troubleshooting workflows. Recent enhancements include foreign language localization and improvements in reporting. SevOne has an unique, guaranteed, 10-day certification for SNMP adapter for new devices for free. In Gartner's survey of end users, SevOne was rated strongest or joint strongest in terms of direction and vision, services and support capabilities, historical reporting and exporting, navigation and workflow ability to commit and deliver on requested enhancements, cost of implementation and cost of maintenance. Of the remaining capabilities, ease of implementation was the area that end users cited as requiring the most attention. Of the end users surveyed, 40% indicated that they had upgraded to the current release in This product is well-suited to large enterprises and service providers that need a flow and log analytics-oriented NPMD solution that can span a distributed and evolving environment. SolarWinds Performance Monitor 11.5 and Traffic Analyzer 4.1 Founded in 1999, SolarWinds is a public company headquartered in Austin, TX. SolarWind's NPMD products, Network Performance Monitor 11.5 and SolarWinds NetFlow Traffic Analyzer 4.1 were evaluated for this research. These products are on-premises software offerings. SolarWinds NPM and SolarWinds NetFlow Traffic Analyzer cover monitoring of network elements for fault, availability and performance. Packet analysis is included as part of the NPMD product, but requires the use of separately installed Packet Capture Sensors. These workflows offer limited Page 16 of 35 Gartner, Inc. G

17 depth for troubleshooting, as compared with other NPMD solutions. Recent feature improvements include wireless heat maps, updated alerting engine and several automation/analysis functions (for example, capacity forecasting, automatic dependencies and automatic geolocation). In Gartner's survey of end users, SolarWinds ease of use and accessibility were cited as product strengths. Most of those surveyed reported no issues during the testing phase of the evaluation process. The same end users rated SolarWinds among the strongest vendors in terms of overall satisfaction. Those clients also reported satisfaction of SolarWinds product direction and vision. Of the end users surveyed, 40% indicated that they had upgraded to the current release in This product is well-suited to SMBs that require an SNMP monitoring solution for infrastructure monitoring, with the ability to expand with NetFlow and packet-based analysis. Viavi Solutions Observer 17 Founded in 1994, the Performance Management business unit of Viavi Solutions is headquartered in Minneapolis, Minnesota. Viavi Solutions is a public company. Viavi's NPMD offering, the Observer Performance Management Platform v.17 has been evaluated for this research. The platform comprises hardware and software products that are delivered as on-premises offerings. The Observer Performance Management Platform is a competitively priced, packet-based NPMD tool that focuses on network and application performance visibility. The Observer platform includes the Observer Apex, Observer Analyzer, Observer GigaStor, Observer Probes, Observer Infrastructure and Observer Management Server. The solution is oriented toward breadth and depth of packet capture and inspection. Viavi continues to design and manufacturing most of its own components and software. Latest updates include a software edition of Gigastor that delivers the same packet capture capabilities as the core product, with 250Gb and 1Tb capacity options. A solid-state drive (SSD)-based portable model of Gigastor was also released in 2015 that supports 10Gb packet capture and storage at full line rate. The last major update in September 2014 also included overhaul of the UI in the form of Observer Apex, an HTML 5 Web-based interface. Gartner's survey of end users indicated Viavi's NPMD tool was rated strongly in terms of ease of implementation. The same end users scored Viavi as the lowest-ranked vendor in terms of ease of upgrading and was joint lowest in terms of navigation and workflow. (This may be due to those end users that have older versions of the product.) Viavi was also rated among the lowest vendors in terms of overall satisfaction. Of the end users surveyed, 33% indicated that they had upgraded to the current release in This product is well-suited to large enterprises with deep and varied NPMD requirements to support network architects and network operations specialists. Context The goal for many IT organizations purchasing an NPMD tool is to provide efficient and effective visibility and understanding of network infrastructure, and to reduce the total cost of maintaining it. NPMD tools focus on identifying and reporting network performance issues that affect IT services in Gartner, Inc. G Page 17 of 35

18 the production environment, and assisting IT organizations to resolve those issues in a timely manner. Clients vary among those that have formed their NPMD solutions from a single product from a single vendor, multiple products from a single vendor and, more commonly, multiple products from multiple vendors combined to complete the NPMD feature set. Gartner outlines major use cases, details and functional areas, and rates each tool's ability to support a typical use scenario. These use cases are derived via hundreds of interactions with Gartner clients using the technology. There are many NPMD tool vendors in the market; some were unable to meet the inclusion criteria established, and are not covered in this research. This analysis complements the "Magic Quadrant for Network Performance Monitoring and Diagnostics" and focuses on product capability. The Magic Quadrant highlights a broad set of criteria that includes corporate visibility, vision, marketing and geographic focus. Gartner strongly recommends that organizations use this research in conjunction with the Magic Quadrant, inquiries with analysts, and other Gartner research to define requirements and select vendors that match their needs. In the course of this research, several key observations emerged that should be carefully considered during NPMD strategy formulation and solution selection, including: NPMD technology has seen incremental innovation over this review period, with core functionality across the vendors remaining largely stagnant. Normalization of UI and workflows across several vendor's toolsets has shown some progress, but much work must still be done. Ease of use remains an area that needs improvement and varies significantly across vendors and within solutions. Limited support exists within NPMD solutions for the monitoring of applications hosted in the private and public cloud, as well as SaaS, and for applications accessed via mobile devices. Although ITOA has not yet fully materialized, vendors have increasingly begun to leverage big data back ends or build operational analytics overlays to facilitate data analytics across all captured data. Support for monitoring software-defined network (SDN) environments remains absent in most NPMD solutions; however, most NPMD vendors now offer virtual instances of their products for instantiation in SDN environments. Flow protocol support and actual flow data utilization vary significantly across vendors. NPMD solutions have a primary data source (SNMP, flow or packet) and have a difficult time leveraging the benefits of each data source for the maximum value to the user. Pricing and product/capability packaging vary significantly across vendors. On-premises software and appliances are the dominant delivery models; however, SaaS offerings are slowly becoming available, although not adopted widely. Page 18 of 35 Gartner, Inc. G

19 Integrations with other ITOM solutions (even other availability and performance solutions) are an afterthought for most, which encourages the use of these tools in the network silo. Although some vendors offer integrated APM and NPMD solutions, buying centers remain divided, with limited joint deals. Many NPMD solutions are assembled from multiple products, which can enable modular adoption of NPMD capabilities, but also can add significant complexity to procurement and ongoing maintenance. It remains imperative that organizations purchase tools that closely match their current maturity levels. Many network monitoring teams have yet to successfully make the leap from basic, reactive network availability management to proactive performance management. Although tool investment can play a part in this maturation, simply investing in NPMD tooling without similar investments in training, integration and processes will yield limited results at best. Gartner recommends that network teams assess their current state of maturity on a regular basis, individually and at the organizational level, to provide this perspective. To help, teams can use Gartner's ITScore for Infrastructure and Operations (ITSIO; see "How to Improve I&O Maturity by Using the ITScore"). Concurrent with this approach, NPMD tools should be evaluated to meet the growing demands from non-network infrastructure and operations (I&O) groups. Consider IT operations holistically in that it incorporates people and processes alongside technology. By reaching out to constituent lines of business (LOBs), it may be possible to see where data derived from the NPMD tool can prove valuable to other domains of the IT operations team. For example, those tasked with tracking configuration changes and the history of such changes would gain valuable insight from weekly network performance reports. This would help identify unauthorized updates to devices, systems or applications or changes made legitimately that had an adverse or inconsequential effect in terms of expected efficiency gains. Network capacity and utilization reports could be used by provisioning and procurement teams to help anticipate future demand for equipment and the cost of acquiring it. Product/Service Class Definition NPMD tools typically include the following primary capabilities: Passive-monitoring techniques analyze network traffic seen by network probes or appliances attached to mirror ports on network switches, or summarized from flow-based instrumentation embedded in network hardware, to provide a high-level summary of network conversations. The ability to inspect the contents of the network packet is essential to the forensics of network-based issues, especially in the context of voice-, video- and network-based applications. This means the product must have the ability to decode and present data on the application, not just expose the raw packet data. The ability to create, manage and report on synthetic test transactions leverages features normally embedded in the network hardware device's OS, such as Service Assurance Agents (SSA), IP SLAs and MediaNet. The ability to generate high-level dashboards and metric roll-ups involves overall network performance and health. Gartner, Inc. G Page 19 of 35

20 Predefined, out-of-the-box, static thresholds indicate warning and critical conditions for relevant metrics that are monitored, and escalation of threshold violations to event correlation and analysis (ECA) and notification management systems. These thresholds are userconfigurable. The approach organizations have taken for monitoring the network has been focused on network probes and via SNMP polling of network devices. SNMP polling provides valuable volume, error and utilization data, but lacks granularity by not breaking down the use of the bandwidth (see Note 1). Probe packet-based inspection was introduced to fill this gap; however, due to the cost of the devices, as well as management and implementation complexity, probes cannot provide visibility in all physical and logical network locations (see Note 2). This requirement for better data than SNMP polling, at a level that is more cost-effective than deploying a large number of packet inspection probes, resulted in the emergence of summarized flow-based techniques (see Note 3). Understanding how tool functionality has evolved for each data source, along with the caveats at each level, is crucial when determining which NPMD tool/tools are appropriate for your organization. This research examines five critical capabilities that differentiate NPMD products in various use cases: Endpoint/component/link monitoring Service delivery monitoring Diagnostics ITOA Integration These critical capabilities represent the most important and differentiated features and functions for NPMD tools. The capabilities in each of these areas were weighted for their respective use cases. Critical Capabilities Definition Endpoint/Component/Link Monitoring The ability to monitor, diagnose and generate alerts for endpoints (e.g., servers, VMs, storage systems or anything with an IP address), network components (including routers, switches and other network devices) and network links, which provide connectivity between network-attached infrastructure. Service Delivery Monitoring Service delivery dashboards help IT organizations understand their alignment with IT objectives, as shown through key performance indicators (KPIs). Reporting is assessed on the product's ability to provide trend reports that measure operational productivity across core processes. Dashboard and reporting capabilities must be tied to business value metrics, common critical success factors and KPIs. The tool must offer multidimensional charts that show how related Page 20 of 35 Gartner, Inc. G

21 metrics affect each other. These would be based on the tools' ability to monitor, diagnose and generate alerts for dynamic end-to-end network service delivery, as it relates to end-user experience, business service delivery and infrastructure component interaction. Diagnostics The ability to facilitate or automate typical steps taken by an operator to troubleshoot, triage and diagnose issues. Aspects that enhance a tool's diagnostics include contextual drill-down, intuitive workflow and navigation aids (such as hover-over features) and simple search mechanisms. IT Operations Analytics The ability to leverage one or more ITOA techniques to distill, infer and/or create action-oriented advice from the dataset collected, which can include metric, time-series-based and unstructured data sources. ITOA capabilities include the ability to support model discovery, ascertain the root causes of performance problems, assess the impacts of multiple known root causes, predict events and recommend remediation. Model discovery is the basis of the other four use cases. Many NPMD tools claiming ITOA capabilities often only filter, aggregate and visualize data. Such tools do not perform inferences; at best, they only facilitate inferences taking place within a human mind (diagnostics). It is important to understand the distinction if automated model discovery, root cause analysis, cause impact determination, prediction or action recommendation is the desired requirement. Integration and Interoperability The tools' ability to be integrated with other third-party tools and the ability of those third-party tools to integrate with NPMD tools are increasingly important factors, particularly for organizations that license best-of-breed ITOM products, as opposed to products from a single vendor. Products are evaluated on the presence of out-of-the box integration capabilities (via APIs or connectors) that link to network fault management, network automation, ECA tools, application performance management, infrastructure performance management and unified monitoring tools. Use Cases Network Operator Responsibilities include troubleshooting network-related hardware/software issues, monitoring/ configuring network devices and making recommendations to network architects. Network operators are typically members of a team that's responsible for monitoring and maintaining the network infrastructure. Extended responsibilities may include analysis of network components for correct configuration (including patching), upgrades and making recommendations for network enhancements. In enterprises and government organizations, they may also serve as a Gartner, Inc. G Page 21 of 35

22 liaison with communication service providers to ensure the proper service delivery, operation and compliance of the provider's equipment, and ascertain delineation of responsibility where fault isolation is not immediately apparent. Network Architect Responsibilities include conceptual network design/planning, communicating architecture rationale to technical/business stakeholders, governance and escalation handling. Unlike generalists, engineers and operators, architects are largely proactive and work predominantly on planning, documentation and optimization, new projects and infrastructure strategies. Network architects often report outside the infrastructure and operations teams, typically working in an enterprise architecture or project management office. Network architects are relied on to provide functional, technical and/or process leadership. The formulation of best practices to ensure effective workflow and the monitoring of industry/technological trends, adoption and direction are common extended responsibilities. IT Operations Generalist Responsibilities include end-user support, fault validation, service desk ticket generation, troubleshooting broad IT issues and working with multiple domain specialists. IT operations generalists are typically members of a team that is responsible for the monitoring and support of the IT infrastructure. This can include endpoints, servers, applications and/or the network (including voice and video services). Given their broad remit, IT operations generalists normally interface with fault and availability monitoring tools and, in less-mature organizations, deal with issues reactively as they are being reported by end users. Collaboration with server, database, application support, application development, network, storage and security teams is frequently the case. As a result, fault identification and isolation is a frequent task as is problem hand-off. IT Operations Manager Responsibilities include team management, service compliance, escalation handling, cross-domain knowledge and experience, report generation and continual service improvement. IT operations managers typically oversee a team of IT operations generalists. As a result, they are often a point of escalation and require adequate high-level visibility as to the immediate health of business applications and services, as well as the underlying infrastructure that supports them. Historical and performance reporting assists IT operations managers in moving to more-proactive states whereby emerging issues can be anticipated and dealt with prior to end users being affected. Tracking and reporting of SLA compliance, and the identification of cross-domain constraints are also requirements that an IT operations manager would seek from any tool. Line-of-Business User Responsibilities include planning and reporting divisional and regional activities and performance in the service of business objectives (e.g., growth, cost and risk). Page 22 of 35 Gartner, Inc. G