G00276258 Critical Capabilities for Network Performance Monitoring and Diagnostics Published: 1 February 2016 Analyst(s): Vivek Bhalla, Sanjit Ganguli NPMD tools are top of mind among infrastructure and operation leaders looking to manage IT services and network infrastructures. This evaluation of NPMD tool capabilities across five common usage scenarios will help IT leaders to foster better relationships with technical and business stakeholders. Key Findings Many network monitoring teams have yet to successfully make the leap from basic, reactive network availability management to proactive performance management. Network performance monitoring and diagnostics technology has seen incremental innovation during this review period, with core functionality across the vendors remaining largely stagnant. On-premises software and appliances are the dominant delivery models; however, SaaS offerings are slowly becoming available, although they haven't been widely adopted. Many NPMD solutions are assembled from multiple products, which can support the modular adoption of NPMD capabilities; however, this can add significant complexity to procurement and ongoing maintenance. Recommendations Invest in NPMD tools to contextualize value and report on improved service quality and application response time for lines of business. Use NPMD technology to manage network demand and capacity planning, particularly before introducing bandwidth-intensive services, such as voice over IP and unified communications. Gauge the ability of NPMD tools' embedded analytics and contextual user interface drill-down workflows to support IT operations' efforts to become more proactive. Strike a cost-effective balance between probe packet-based inspection methods for missioncritical component granular insight and flow collection techniques for holistic, infrastructurewide coverage.
What You Need to Know Unlike their network fault or event management counterparts, network performance monitoring and diagnostics (NPMD) tools enable IT and network operations teams to understand the ongoing behavior of the network and its constituent elements in response to traffic demands and changes in network utilization. Measuring and reporting on network performance is crucial to ensuring that performance stays at an acceptable level. It can also highlight opportunities to enhance business value for internal end users and external customers through improved application delivery. Finally, NPMD tools enable improved network capacity management, thereby optimizing capital investment for networking equipment and services. 1 The client input from which this critical capability analysis was derived came from a survey of end users and from the Gartner inquiry process. Analysis Critical Capabilities Use-Case Graphics Page 2 of 35 Gartner, Inc. G00276258
Figure 1. Vendors' Product Scores for the Network Operator Use Case Gartner, Inc. G00276258 Page 3 of 35
Figure 2. Vendors' Product Scores for the Network Architect Use Case Page 4 of 35 Gartner, Inc. G00276258
Figure 3. Vendors' Product Scores for the IT Operations Generalist Use Case Gartner, Inc. G00276258 Page 5 of 35
Figure 4. Vendors' Product Scores for the IT Operations Manager Use Case Page 6 of 35 Gartner, Inc. G00276258
Figure 5. Vendors' Product Scores for the Line-of-Business User Use Case Gartner, Inc. G00276258 Page 7 of 35
Vendors Automic Streamcore 6.3 and Sysload 6.0 Founded in 1985, Automic Software is a privately held company headquartered in Bellevue, Washington, with European headquarters in Vienna, Austria. Automic's NPMD products, Streamcore v.6.3 and Sysload v.6.0, were evaluated for this research. This product is an on-premises software and appliance offering. Originally focused on workload automation, application release automation and automated service orchestration, Automic added an NPMD offering when it acquired Orsyp in May 2014. The acquisition has evolved into a dedicated business unit focused on NPMD. The products are primarily deployed as hardware appliances, although the v.6.3 release introduces a Streamcore virtual appliance. Updates in Sysload to HTML5 provide visually compelling user interface (UI) captures. Typical operator workflows include troubleshooting and optimization use cases, with an innovative approach to summarizing global service performance among many sites. The company offers above-average service delivery monitoring capabilities, but products lack the analytics and integration of most other solutions. Although there are plans to consolidate the solution sets, multiple graphical UIs (GUIs) exist among the Streamcore/Sysload product sets and the rest of the Automic product line. Gartner's survey of end users rated Automic's NPMD solution as joint strongest in terms of maintenance, reliability and ease of upgrading. This survey showed Automic's NPMD solution to be the most expensive in terms of total cost. The same survey indicated Automic's service subscription costs accounted for the greatest percentage of total spending on tooling costs. Of the end users surveyed, 50% indicated that they had upgraded to the current release in 2015. The product is well-suited to enterprises that need to monitor large numbers of sites, such as those in the retail, banking and government sectors. CA Technologies Unified Infrastructure Management (UIM) 8.3 Founded in 1976, CA Technologies is a public company headquartered in New York City, NY. CA's NPMD product, Unified Infrastructure Management (UIM) v.8.3, was evaluated for this research. This product is an on-premises software offering. Although CA's history in this market spans several acquisitions, the current offering is predominantly based on the legacy NetQoS and Nimsoft technologies. CA's focus CA UIM v.8.3 is a departure from previous strategy that centered on CA Performance Manager as the primary platform. Requirements can be supplemented by and integrated with CA Network Flow Analysis (NFA) for traffic monitoring, CA Unified Communications Monitor for the monitoring of voice and video technologies, and CA Application Delivery Analysis (ADA), which is focused on packet-level inspection for applications. These three products came from the NetQoS acquisition. Only NFA has been integrated into the UIM platform. CA's OEM relationship with Viavi Solutions for the Gigastor Page 8 of 35 Gartner, Inc. G00276258
product has been discontinued, limiting its ability to provide high-end storage options for packet data. The CA solution for NPMD focuses on reporting and workflow to identify and isolate issues. The lack of specific hardware appliances makes this software-based solution flexible and cost-effective; however, deployment complexity can be an issue. Recent enhancements include integration between UIM and NFA, support for big data monitoring and bulk configuration of probes. Gartner's survey of end users indicated that the company's breadth of product solutions is a strength. The same survey indicated CA's implementation, training and professional services costs accounted for the greatest percentage of total spending on tooling, and the vendor was rated the weakest in terms maintenance costs. CA was also rated among the lowest vendors in terms of overall satisfaction. Of the end users surveyed, 33% indicated that they had upgraded to the current release in 2015. This product is well-suited to large enterprises and service providers that need multitenancy and a focus on flow and Simple Network Management Protocol (SNMP) monitoring. Cisco Prime Infrastructure 3.0, NAM 6.2 and Collaboration 10 Founded in 1984, Cisco is a public company headquartered in San Jose, California. Cisco's NPMD products, Prime Infrastructure (PI) v.3.0, Network Analysis Module (NAM) v.6.2, Prime Collaboration (PC) v.11.0 and Cisco Connected Analytics (CCA) v.1.0 were evaluated for this research. These products are on-premises software offerings. PI's multiple capabilities span the network configuration, monitoring and the management of Cisco equipment. PC is focused on complete life cycle management of Cisco unified communications (UC) solutions. The NAM is a hardware appliance or module that can be included in many Cisco network devices for handling deep packet inspection (DPI). The solutions offer a common UI, except for the NAM component, and deep life cycle support of all Cisco products on the day they ship. The focus on Cisco technologies limits attention to third-party devices and technology, which are common in today's networks. Recent enhancements typically mirror new Cisco hardware capabilities and additional APIs for integration, including increased support for Wireless LAN monitoring and overlay networks. As part of this suite, Cisco has recently introduced its 1.0 version of CCA (formerly known as Prime Analytics) to provide real-time analytics for network and business intelligence (BI). Although promising, this solution isn't part of many of its NPMD workflows. Gartner's survey of end users rated Cisco's NPMD solution strongly in terms of ease of upgrading, total cost of implementation and total cost of maintenance. The same survey indicated Cisco's NPMD solution to be the third weakest in terms of historical reporting and exporting. Of the end users surveyed, 80% indicated that they had upgraded to the current release in 2015. This product is well-suited to global enterprises with heavy investment in Cisco hardware that want to manage the complete life cycle. Capabilities such as hardware-embedded DPI provide NPMD advantages for organizations standardized on Cisco. Gartner, Inc. G00276258 Page 9 of 35
Corvil Network Data Analytics Platform Tera (9.0) Founded in 2000, Corvil is a privately owned company headquartered in Dublin, Ireland. Corvil's NPMD product, Network Data Analytics Platform Tera, was evaluated for this research. This product is an on-premises appliance. Network Data Analytics Platform Tera is a competitively priced, packet-based NPMD tool that focuses on analyzing high volumes of wire data in stream. The company has focused on improving the modularity of the product with the release of Tera. The new platform includes the ability to add new decoders, analytics, dashboards and data-publishing streams outside the main development cycle in an agile manner. Tera has also extended support to application events that can be decoded and correlated with network alerts data and published in Corvil's analytics stream. In Gartner's survey of end users, Corvil was rated strongest or joint strongest in terms of direction and vision, service and support, maintenance and reliability, ease of implementation, ease of upgrading, cost of maintenance and integration with third-party tools. Of the remaining capabilities, navigation and workflow was the area that end users cited as requiring the most attention. Corvil was also among one of the lowest-ranked vendors in terms of total network elements being instrumented. Corvil was rated among the strongest vendors in terms of overall satisfaction. Of the end users surveyed, 80% indicated that they had upgraded to the current release in 2015. The product is well-suited to enterprises in need of short-time-scale and high-granularity network monitoring, such as in the electronic-trading market and other digital businesses. Flowmon Networks Flowmon 7.03 Founded in 2007, Flowmon Networks is a privately owned company headquartered in Brno, Czech Republic. Flowmon Networks' NPMD offering, Flowmon v.7.03, has been evaluated for this research. This product is an on-premises hardware, virtual appliance and SaaS offering. Flowmon is a competitively priced, flow-oriented NPMD tool designed to meet the most demanding of performance and scalability requirements. The vendor is one of the first to support 100G environments with its 100 GbE Flowmon probe. The tool can process and analyze more than 250,000 flows per second. End-user experience visibility on a per-application basis is limited for hosted virtual desktop (HVD) environments, as is packet support, which focuses on flow conversion, as opposed to end-user or timing metrics. Gartner's survey of end users indicated Flowmon s NPMD tool was used to instrument the second highest number of network elements and monitor the third-highest number of flows per second. The same end-user survey scored integration with third-party tools joint lowest. However, more than 60% of those same respondents integrated the tool with a third-party security information and event management (SIEM) tool. This appears to be an area of specialization. Of the end users surveyed, 100% indicated that they had upgraded to the current release in 2015. This product is well-suited to midmarket, large enterprise and service providers that need a scalable. flow-oriented NPMD solution with diverse deployment options and an ability to meet security use cases. Page 10 of 35 Gartner, Inc. G00276258
Fluke Networks TruView 10.0 and TruView Live 1.0 Sold by Danaher Corp. to NetScout Systems in July 2015, the Fluke Networks entity is headquartered in Everett, Washington. Fluke's NPMD products, Visual TruView v.10.0, TruView Live and OptiView XG were evaluated for this research. These products are on-premises appliance, virtual appliance and SaaS-based offerings. Fluke Networks focuses on network test and monitoring solutions. TruView focuses on workflow that is easy to learn and remember. OptiView XG provides endpoint and additional data to TruView to support centralized troubleshooting analysis. Recent enhancements include a big data store for scalability, enhancements to Advanced Path Analysis, and active testing capabilities for Web and voice over IP (VoIP). Applications such as Microsoft Lync and Citrix environments are also supported. TruView v.10.0 is a single, modular, appliance-based offering that can be repurposed to target multiple data source types, depending on specific requirements. TruView appliances are sized based on network storage and processing. TruView Live, deployed as a SaaS offering, is among the first NPMD products to offer this deployment model, and, although it has yet to be a proven model, it has the potential to be disruptive. Gartner's survey of end users indicated that Fluke's customers favored the ease of use of TruView for troubleshooting, along with the combination of Fluke's flow-based, SNMP and deep-packet visibility. Clients have reported that Fluke's alerting capabilities and its ability to report on business transactions are areas for improvement. Of the end users surveyed, 50% indicated that they had upgraded to the current release in 2015. This product is well-suited to large enterprises and service providers that need extensive NPMD capabilities and value ease of workflow via contextual drill-down navigation. Fluke Networks completed the vendor survey and provided customer references prior to the completion of the acquisition by NetScout. NetScout did not respond to requests for supplemental information and/or engage in Gartner's standard procedures to address the contents of this document. Hence, Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with end users who have evaluated or deployed each NPMD product. NetScout has sued Gartner over the content of the 2014 NPMD Magic Quadrant, and that lawsuit is pending. In addition, as noted above, NetScout has refused Gartner's invitation to engage in Gartner's standard procedures to address the content of this document. Genie Networks GenieATM 6.3.1 and VM Founded in 2000, Genie Networks is a privately owned company headquartered in Taipei, Taiwan. Genie's NPMD product, GenieATM 6.3.1 and VM, have been evaluated for this research. These products are an on-premises hardware and software offering. Gartner, Inc. G00276258 Page 11 of 35
Genie Networks focuses on network fault and performance monitoring solutions. Genie relies primarily on SNMP, flow-based and proprietary data gathering techniques to present a real-time status of the network. Correlation with fault events and alarm generation against service models are standard features. Genie introduced a new virtual machine (VM) variant of its GenieATM product in August 2015 that offers the features of the appliance-based option as a software image that can be downloaded and installed in a virtualized infrastructure. Multiprotocol Label Switching (MPLS) support was another aspect that was enhanced with a dedicated appliance focused on providing visibility of this technology. The UI continues to be less sophisticated than those of the leading NPMD tools. Gartner has observed that Genie customers continue to report the estimated total cost of the solution to be competitive, compared with most of the other vendors included in this research. Maintenance and reliability also continues to gain positive feedback from end users. These products are well-suited to service providers that require a scalable combined network fault management (NFM) and NPMD solution. Genie Networks did not respond to requests for supplemental information and/or engage in Gartner's standard procedures to address the contents of this document. Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with end users. HPE Network Node Manager i 10.01 and Real User Monitor 9.25 Founded in 2015, Hewlett Packard Enteprise (HPE) is a public company headquartered in Palo Alto, California. HPE's NPMD products, Network Node Manager i (NNMi) v.10.01 and Real User Monitoring (RUM) v.9.25 have been evaluated for this research. These products are on-premises software offerings. Designed for network operations specialists and architects and reflecting its predominant use, the HPE solution flexibly supports service delivery monitoring and diagnosis, informed primarily by SNMP and flow data. Packet inspection and additional application context gathering to support complex diagnosis is provided through a data interchange, modeling and process-level integration with HPE's RUM product. Support for application mapping requires that users include the NNMi Smart Plug-In Performance for Traffic for flow-based traffic mapping. However, although HPE has simplified the pricing and packaging of its NPMD solution, it is composed of multiple component products. This results in two distinctly different UIs. Recent enhancements include improved support for mobile application monitoring and network virtualization. Integration with Network Automation presents an interesting differentiator to traditional NPMD solutions. In Gartner's survey of end users, HPE's implementation, training and professional services costs accounted for the lowest percentage of total spending on tooling. The same end users scored HPE poorly in terms of historical reporting and exporting, the ability to commit and deliver on requested enhancements, and the cost of implementation. However, two-thirds of these same end users stated that they anticipated increasing their investment of their HPE NPMD solution, and Gartner has witnessed evidence of improved renewal rates. HPE was rated poorly in terms of overall Page 12 of 35 Gartner, Inc. G00276258
satisfaction. Of the end users surveyed, 67% indicated that they had upgraded to the current release in 2015. This product is well-suited to large enterprises, especially those with significant HPE IT operations management (ITOM) tooling investments, with deep and varied NPMD requirements. InfoVista VistaInsight for Networks 5.3 and 5View Suite Founded in 1995, InfoVista is a privately owned company headquartered in in Les Ulis, France. Infovista's NPMD offering, VistaInsight for Networks v.5.3, 5View Service Data Manager v.3.4, 5View Netflow v.7.1, 5View Applications v.7.0, and 5View Mediation v.5.2 have been evaluated for this research. These products are on-premises software offerings. The products can be scaled up and deployed, depending on the level of visibility required, including SNMP, flow and packet data. Each of the products in the offering collects and rolls data up incrementally, allowing for large scale. Each part of the offering is multitenant, supporting multiple users in a single master instance of the solution. This architecture, although scalable, comes with complexity requiring a services engagement to deploy. End users would typically be using VistaInsight, which provides a reporting interface to pull usage and performance reports. Due to the number of products, UIs, and implementation nuances, Gartner has observed that InfoVista caters to and has become embedded in large carrier customers. End users report Infovista has a high cost of licensing and implementation. Gartner's survey of end users indicated that InfoVista's NPMD tool was used to instrument the highest number of network elements and monitor the second highest number of flows per second. Gartner has observed that end users report increased satisfaction with the tool's navigation and workflow, as well as contextual drill-down and troubleshooting, both previously cited as product weaknesses. Of the end users surveyed, 40% indicated that they had upgraded to the current release in 2015. This product is well-suited to large carriers that require diverse deployment options, multitenancy and extensive reporting from their NPMD investments. NetScout ngeniusone 5.4 Founded in 1984, NetScout Systems is a public company headquartered in Westford, Massachusetts. NetScout's NPMD product, ngeniusone v.5.4, was the product evaluated for this research. The product is an on-premises appliance and software offering. NetScout's singular focus on supporting any and all of the diagnostic needs of service provider and large enterprise network specialists has resulted in a diverse set of hardware appliances and software that is managed through an increasingly (although not completely) cohesive ngeniusone UI. The design and capability of the solution enable it to support deep diagnostic workflows, with some application context where network specialists are enabled more by flexibility than guidance. Adaptive Session Intelligence technology enables the solution to support 40GbE and 100GbE environments, while retaining visibility and keeping storage requirements manageable. The solution requires hardware appliances, so it is sized based on network data storage and processing Gartner, Inc. G00276258 Page 13 of 35
requirements. Gartner has observed that NetScout customers report the latest version of ngeniusone as having improved service representation. NetScout customers also continued to report limited upgrade paths for hardware elements of the solution, resulting in higher-thanexpected total cost of ownership (TCO). This product is well-suited to service providers and large enterprises with network operations teams and architects that need deep NPMD capabilities at extreme scale to support their own monitoring and diagnostic efforts. NetScout did not respond to requests for supplemental information and/or engage in Gartner's standard procedures to address the contents of this document. Hence, Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's own marketing collateral, public information and discussions with end users who have evaluated or deployed each NPMD product. NetScout has sued Gartner over the content of the 2014 NPMD Magic Quadrant, and that lawsuit is pending. In addition, as noted above, NetScout has refused Gartner's invitation to engage in Gartner's standard procedures to address the content of this document. Niksun NetVCR Alpine 4.5 Founded in 1997, Niksun is a privately owned company headquartered in Princeton, New Jersey. Niksun's NPMD offering, NetVCR running Alpine v.4.5, was evaluated for this research. This product is an on-premises appliance and virtual appliance offering. The product is focused on high-fidelity data capture at speeds exceeding 100 Gb/sec. The top-end platform for 100G support is Niksun's Supreme Eagle, which promises faster processing and more storage than earlier 100G offerings. This ability to process at scale makes the product attractive in environments in which high-speed transactions are prevalent, including organizations that have an Internet of Things (IoT) initiative. In Gartner's evaluation, the product competently handled endpoint, component and link monitoring. However, IT operations analytics (ITOA) capabilities were below par, compared with the leading tools included in this research. The product demonstrates the ability to support the most-scalable environments, including 3G and 4G cellular networks. NetVCR's product strengths appeal to an exclusive group: those that prioritize robust network infrastructures with high transaction rates over other requirements. Additional modules can be used to analyze, report and troubleshoot market data feeds and trade transactions, as well as oversee and administer VoIP systems. The product is well-suited to global service providers, large financial institutions, and the defense and intelligence communities with consolidated IT and security requirements. Niksun did not respond to requests for supplemental information and/or to engage in Gartner's standard procedures to address the contents of this document. Hence, Gartner's analysis for this vendor is based on other credible sources, including previous vendor briefings and interactions, the vendor's marketing collateral, public information and discussions with end users. Page 14 of 35 Gartner, Inc. G00276258
Paessler AG PRTG Network Monitor 15.3.17.29995/2996 Founded in 1997, Paessler is a privately owned company headquartered in Nuremberg, Germany. Paessler's NPMD offering PRTG Network Monitor v.15.3 has been evaluated for this research. The product is an on-premises software offering. PRTG Network Monitor is a competitively priced, consolidated NFM, NPMD and infrastructure monitoring tool that relies primarily on SNMP, flow-based and proprietary data-gathering techniques to present a real-time status of the network. Correlation with fault events and alarm generation against service models are standard features. Latest product enhancements include cloud-based sensors, clustered high-availability, automatic device discovery that recommends the right sensor for the device type detected, the support for mobile apps via a push-notification feature that enables alerts to be sent to mobile devices, support for MongoDB, and increased sensor reporting and analysis. The UI is less sophisticated than the leading NPMD tools; however, a recent overhaul has reduced the disparity. Integration with Google Analytics and an optional ticketing system provide workflow aids beyond the core network monitoring focus. Gartner's survey of end users indicated Paessler's NPMD tool was the most competitively priced in terms of overall cost and annual maintenance. Paessler was also rated among the strongest vendors in terms of overall satisfaction. The same end-user survey scored integration with thirdparty tools joint second lowest, whereas the vendor was ranked among the lowest three for contextual navigation and workflow. Of the end users surveyed, 100% indicated that they had upgraded to the current release in 2015. This product is well-suited to small or midsize businesses (SMBs), midmarket businesses, large enterprises and managed service providers that need a competitively priced NPMD solution with a small deployment footprint and tightly coupled architecture. Riverbed SteelCentral Suite Founded in 2002, Riverbed is a privately owned company headquartered in San Francisco, California. Riverbed's NPMD offerings, SteelCentral NPM, Netsensor 3.0, UCExpert 3.0 and Portal 1.0 have been evaluated for this research. The products are on-premises appliance and software offerings. As one of the handful of NPMD vendors with a full-featured application performance monitoring (APM) offering (with some products being included in both), the solution provides a broader, deeper set of application visibility and capabilities. SteelCentral NPM consists of multiple components that include SteelCentral AppResponse 9.5.3, SteelCentral NetProfiler 10.8.5, SteelCentral NetShark 10.8.5, SteelCentral Packet Analyzer 10.8.5 and SteelCentral Transaction Analyzer 17.0. The solution does require hardware appliances, so it is sized based on network data storage and processing requirements. Riverbed continues to make progress on this front, notably with SteelCentral Portal, which is now the primary interface for consolidated APM and NPMD visualization and as the launch pad for detailed diagnostic workflows. Gartner's survey of end users indicated Riverbed rated highest in terms of ease of upgrading and joint highest in terms of ease of implementation. The results with regard to ease of implementation Gartner, Inc. G00276258 Page 15 of 35
are somewhat at odds with end-user feedback during the past 12 months, particularly where multiple Riverbed products were involved. Riverbed was rated as second highest in terms of maintenance and reliability, and historical reporting and exporting. Riverbed was also rated among the strongest vendors in terms of overall satisfaction. The same end-user survey apportioned the cost of implementation, training and professional services as a percentage of overall NPMD cost as second highest. Of the end users surveyed, 40% indicated that they had upgraded to the current release in 2015. This product is well-suited to large enterprises and service providers with NPMD requirements that include integration with APM tooling and are intended to support the needs of network and application teams. SevOne PAS 5.4 and PLA 2.0 Founded in 2005, SevOne is a privately owned company headquartered in Wilmington, Delaware. SevOne's NPMD products, SevOne Performance Appliance Solution (PAS) v.5.4 and Performance Log Appliance (PLA) v.2.0 were evaluated for this research. These products are on-premises physical and virtual appliance offerings. SevOne PAS provides a scalable NPMD solution based on a distributed peer-to-peer architecture. PLA adds the ability to ingest unstructured data, which is a differentiator in the NPMD market. The tool demonstrates significant attention to visualizing and reporting on how network performance affects service and application delivery. Although PAS lacks native non-flow-based DPI collection and analysis, packet-based analytics are served through an OEM and largely nonexistent from standard troubleshooting workflows. Recent enhancements include foreign language localization and improvements in reporting. SevOne has an unique, guaranteed, 10-day certification for SNMP adapter for new devices for free. In Gartner's survey of end users, SevOne was rated strongest or joint strongest in terms of direction and vision, services and support capabilities, historical reporting and exporting, navigation and workflow ability to commit and deliver on requested enhancements, cost of implementation and cost of maintenance. Of the remaining capabilities, ease of implementation was the area that end users cited as requiring the most attention. Of the end users surveyed, 40% indicated that they had upgraded to the current release in 2015. This product is well-suited to large enterprises and service providers that need a flow and log analytics-oriented NPMD solution that can span a distributed and evolving environment. SolarWinds Performance Monitor 11.5 and Traffic Analyzer 4.1 Founded in 1999, SolarWinds is a public company headquartered in Austin, TX. SolarWind's NPMD products, Network Performance Monitor 11.5 and SolarWinds NetFlow Traffic Analyzer 4.1 were evaluated for this research. These products are on-premises software offerings. SolarWinds NPM and SolarWinds NetFlow Traffic Analyzer cover monitoring of network elements for fault, availability and performance. Packet analysis is included as part of the NPMD product, but requires the use of separately installed Packet Capture Sensors. These workflows offer limited Page 16 of 35 Gartner, Inc. G00276258
depth for troubleshooting, as compared with other NPMD solutions. Recent feature improvements include wireless heat maps, updated alerting engine and several automation/analysis functions (for example, capacity forecasting, automatic dependencies and automatic geolocation). In Gartner's survey of end users, SolarWinds ease of use and accessibility were cited as product strengths. Most of those surveyed reported no issues during the testing phase of the evaluation process. The same end users rated SolarWinds among the strongest vendors in terms of overall satisfaction. Those clients also reported satisfaction of SolarWinds product direction and vision. Of the end users surveyed, 40% indicated that they had upgraded to the current release in 2015. This product is well-suited to SMBs that require an SNMP monitoring solution for infrastructure monitoring, with the ability to expand with NetFlow and packet-based analysis. Viavi Solutions Observer 17 Founded in 1994, the Performance Management business unit of Viavi Solutions is headquartered in Minneapolis, Minnesota. Viavi Solutions is a public company. Viavi's NPMD offering, the Observer Performance Management Platform v.17 has been evaluated for this research. The platform comprises hardware and software products that are delivered as on-premises offerings. The Observer Performance Management Platform is a competitively priced, packet-based NPMD tool that focuses on network and application performance visibility. The Observer platform includes the Observer Apex, Observer Analyzer, Observer GigaStor, Observer Probes, Observer Infrastructure and Observer Management Server. The solution is oriented toward breadth and depth of packet capture and inspection. Viavi continues to design and manufacturing most of its own components and software. Latest updates include a software edition of Gigastor that delivers the same packet capture capabilities as the core product, with 250Gb and 1Tb capacity options. A solid-state drive (SSD)-based portable model of Gigastor was also released in 2015 that supports 10Gb packet capture and storage at full line rate. The last major update in September 2014 also included overhaul of the UI in the form of Observer Apex, an HTML 5 Web-based interface. Gartner's survey of end users indicated Viavi's NPMD tool was rated strongly in terms of ease of implementation. The same end users scored Viavi as the lowest-ranked vendor in terms of ease of upgrading and was joint lowest in terms of navigation and workflow. (This may be due to those end users that have older versions of the product.) Viavi was also rated among the lowest vendors in terms of overall satisfaction. Of the end users surveyed, 33% indicated that they had upgraded to the current release in 2015. This product is well-suited to large enterprises with deep and varied NPMD requirements to support network architects and network operations specialists. Context The goal for many IT organizations purchasing an NPMD tool is to provide efficient and effective visibility and understanding of network infrastructure, and to reduce the total cost of maintaining it. NPMD tools focus on identifying and reporting network performance issues that affect IT services in Gartner, Inc. G00276258 Page 17 of 35
the production environment, and assisting IT organizations to resolve those issues in a timely manner. Clients vary among those that have formed their NPMD solutions from a single product from a single vendor, multiple products from a single vendor and, more commonly, multiple products from multiple vendors combined to complete the NPMD feature set. Gartner outlines major use cases, details and functional areas, and rates each tool's ability to support a typical use scenario. These use cases are derived via hundreds of interactions with Gartner clients using the technology. There are many NPMD tool vendors in the market; some were unable to meet the inclusion criteria established, and are not covered in this research. This analysis complements the "Magic Quadrant for Network Performance Monitoring and Diagnostics" and focuses on product capability. The Magic Quadrant highlights a broad set of criteria that includes corporate visibility, vision, marketing and geographic focus. Gartner strongly recommends that organizations use this research in conjunction with the Magic Quadrant, inquiries with analysts, and other Gartner research to define requirements and select vendors that match their needs. In the course of this research, several key observations emerged that should be carefully considered during NPMD strategy formulation and solution selection, including: NPMD technology has seen incremental innovation over this review period, with core functionality across the vendors remaining largely stagnant. Normalization of UI and workflows across several vendor's toolsets has shown some progress, but much work must still be done. Ease of use remains an area that needs improvement and varies significantly across vendors and within solutions. Limited support exists within NPMD solutions for the monitoring of applications hosted in the private and public cloud, as well as SaaS, and for applications accessed via mobile devices. Although ITOA has not yet fully materialized, vendors have increasingly begun to leverage big data back ends or build operational analytics overlays to facilitate data analytics across all captured data. Support for monitoring software-defined network (SDN) environments remains absent in most NPMD solutions; however, most NPMD vendors now offer virtual instances of their products for instantiation in SDN environments. Flow protocol support and actual flow data utilization vary significantly across vendors. NPMD solutions have a primary data source (SNMP, flow or packet) and have a difficult time leveraging the benefits of each data source for the maximum value to the user. Pricing and product/capability packaging vary significantly across vendors. On-premises software and appliances are the dominant delivery models; however, SaaS offerings are slowly becoming available, although not adopted widely. Page 18 of 35 Gartner, Inc. G00276258
Integrations with other ITOM solutions (even other availability and performance solutions) are an afterthought for most, which encourages the use of these tools in the network silo. Although some vendors offer integrated APM and NPMD solutions, buying centers remain divided, with limited joint deals. Many NPMD solutions are assembled from multiple products, which can enable modular adoption of NPMD capabilities, but also can add significant complexity to procurement and ongoing maintenance. It remains imperative that organizations purchase tools that closely match their current maturity levels. Many network monitoring teams have yet to successfully make the leap from basic, reactive network availability management to proactive performance management. Although tool investment can play a part in this maturation, simply investing in NPMD tooling without similar investments in training, integration and processes will yield limited results at best. Gartner recommends that network teams assess their current state of maturity on a regular basis, individually and at the organizational level, to provide this perspective. To help, teams can use Gartner's ITScore for Infrastructure and Operations (ITSIO; see "How to Improve I&O Maturity by Using the ITScore"). Concurrent with this approach, NPMD tools should be evaluated to meet the growing demands from non-network infrastructure and operations (I&O) groups. Consider IT operations holistically in that it incorporates people and processes alongside technology. By reaching out to constituent lines of business (LOBs), it may be possible to see where data derived from the NPMD tool can prove valuable to other domains of the IT operations team. For example, those tasked with tracking configuration changes and the history of such changes would gain valuable insight from weekly network performance reports. This would help identify unauthorized updates to devices, systems or applications or changes made legitimately that had an adverse or inconsequential effect in terms of expected efficiency gains. Network capacity and utilization reports could be used by provisioning and procurement teams to help anticipate future demand for equipment and the cost of acquiring it. Product/Service Class Definition NPMD tools typically include the following primary capabilities: Passive-monitoring techniques analyze network traffic seen by network probes or appliances attached to mirror ports on network switches, or summarized from flow-based instrumentation embedded in network hardware, to provide a high-level summary of network conversations. The ability to inspect the contents of the network packet is essential to the forensics of network-based issues, especially in the context of voice-, video- and network-based applications. This means the product must have the ability to decode and present data on the application, not just expose the raw packet data. The ability to create, manage and report on synthetic test transactions leverages features normally embedded in the network hardware device's OS, such as Service Assurance Agents (SSA), IP SLAs and MediaNet. The ability to generate high-level dashboards and metric roll-ups involves overall network performance and health. Gartner, Inc. G00276258 Page 19 of 35
Predefined, out-of-the-box, static thresholds indicate warning and critical conditions for relevant metrics that are monitored, and escalation of threshold violations to event correlation and analysis (ECA) and notification management systems. These thresholds are userconfigurable. The approach organizations have taken for monitoring the network has been focused on network probes and via SNMP polling of network devices. SNMP polling provides valuable volume, error and utilization data, but lacks granularity by not breaking down the use of the bandwidth (see Note 1). Probe packet-based inspection was introduced to fill this gap; however, due to the cost of the devices, as well as management and implementation complexity, probes cannot provide visibility in all physical and logical network locations (see Note 2). This requirement for better data than SNMP polling, at a level that is more cost-effective than deploying a large number of packet inspection probes, resulted in the emergence of summarized flow-based techniques (see Note 3). Understanding how tool functionality has evolved for each data source, along with the caveats at each level, is crucial when determining which NPMD tool/tools are appropriate for your organization. This research examines five critical capabilities that differentiate NPMD products in various use cases: Endpoint/component/link monitoring Service delivery monitoring Diagnostics ITOA Integration These critical capabilities represent the most important and differentiated features and functions for NPMD tools. The capabilities in each of these areas were weighted for their respective use cases. Critical Capabilities Definition Endpoint/Component/Link Monitoring The ability to monitor, diagnose and generate alerts for endpoints (e.g., servers, VMs, storage systems or anything with an IP address), network components (including routers, switches and other network devices) and network links, which provide connectivity between network-attached infrastructure. Service Delivery Monitoring Service delivery dashboards help IT organizations understand their alignment with IT objectives, as shown through key performance indicators (KPIs). Reporting is assessed on the product's ability to provide trend reports that measure operational productivity across core processes. Dashboard and reporting capabilities must be tied to business value metrics, common critical success factors and KPIs. The tool must offer multidimensional charts that show how related Page 20 of 35 Gartner, Inc. G00276258
metrics affect each other. These would be based on the tools' ability to monitor, diagnose and generate alerts for dynamic end-to-end network service delivery, as it relates to end-user experience, business service delivery and infrastructure component interaction. Diagnostics The ability to facilitate or automate typical steps taken by an operator to troubleshoot, triage and diagnose issues. Aspects that enhance a tool's diagnostics include contextual drill-down, intuitive workflow and navigation aids (such as hover-over features) and simple search mechanisms. IT Operations Analytics The ability to leverage one or more ITOA techniques to distill, infer and/or create action-oriented advice from the dataset collected, which can include metric, time-series-based and unstructured data sources. ITOA capabilities include the ability to support model discovery, ascertain the root causes of performance problems, assess the impacts of multiple known root causes, predict events and recommend remediation. Model discovery is the basis of the other four use cases. Many NPMD tools claiming ITOA capabilities often only filter, aggregate and visualize data. Such tools do not perform inferences; at best, they only facilitate inferences taking place within a human mind (diagnostics). It is important to understand the distinction if automated model discovery, root cause analysis, cause impact determination, prediction or action recommendation is the desired requirement. Integration and Interoperability The tools' ability to be integrated with other third-party tools and the ability of those third-party tools to integrate with NPMD tools are increasingly important factors, particularly for organizations that license best-of-breed ITOM products, as opposed to products from a single vendor. Products are evaluated on the presence of out-of-the box integration capabilities (via APIs or connectors) that link to network fault management, network automation, ECA tools, application performance management, infrastructure performance management and unified monitoring tools. Use Cases Network Operator Responsibilities include troubleshooting network-related hardware/software issues, monitoring/ configuring network devices and making recommendations to network architects. Network operators are typically members of a team that's responsible for monitoring and maintaining the network infrastructure. Extended responsibilities may include analysis of network components for correct configuration (including patching), upgrades and making recommendations for network enhancements. In enterprises and government organizations, they may also serve as a Gartner, Inc. G00276258 Page 21 of 35
liaison with communication service providers to ensure the proper service delivery, operation and compliance of the provider's equipment, and ascertain delineation of responsibility where fault isolation is not immediately apparent. Network Architect Responsibilities include conceptual network design/planning, communicating architecture rationale to technical/business stakeholders, governance and escalation handling. Unlike generalists, engineers and operators, architects are largely proactive and work predominantly on planning, documentation and optimization, new projects and infrastructure strategies. Network architects often report outside the infrastructure and operations teams, typically working in an enterprise architecture or project management office. Network architects are relied on to provide functional, technical and/or process leadership. The formulation of best practices to ensure effective workflow and the monitoring of industry/technological trends, adoption and direction are common extended responsibilities. IT Operations Generalist Responsibilities include end-user support, fault validation, service desk ticket generation, troubleshooting broad IT issues and working with multiple domain specialists. IT operations generalists are typically members of a team that is responsible for the monitoring and support of the IT infrastructure. This can include endpoints, servers, applications and/or the network (including voice and video services). Given their broad remit, IT operations generalists normally interface with fault and availability monitoring tools and, in less-mature organizations, deal with issues reactively as they are being reported by end users. Collaboration with server, database, application support, application development, network, storage and security teams is frequently the case. As a result, fault identification and isolation is a frequent task as is problem hand-off. IT Operations Manager Responsibilities include team management, service compliance, escalation handling, cross-domain knowledge and experience, report generation and continual service improvement. IT operations managers typically oversee a team of IT operations generalists. As a result, they are often a point of escalation and require adequate high-level visibility as to the immediate health of business applications and services, as well as the underlying infrastructure that supports them. Historical and performance reporting assists IT operations managers in moving to more-proactive states whereby emerging issues can be anticipated and dealt with prior to end users being affected. Tracking and reporting of SLA compliance, and the identification of cross-domain constraints are also requirements that an IT operations manager would seek from any tool. Line-of-Business User Responsibilities include planning and reporting divisional and regional activities and performance in the service of business objectives (e.g., growth, cost and risk). Page 22 of 35 Gartner, Inc. G00276258
The LOB user may be a manager or a member of a team that is wholly or partially responsible for those applications or services that are perceived to be vital to running the enterprise or business unit. An LOB user will have a requirement to collect, identify and analyze business data to review and evaluate the organizations business performance. Identifying constraints to the delivery of the service or impediments to maximizing its efficiency and performance will be a priority. Adherence to any regulatory compliance, while reporting on KPI trends and patterns, is also vital. Integration with business reporting and online analytical processing (OLAP) tools will be a requirement for LOB users. Vendors Added and Dropped Added Flowmon, Paessler and SolarWinds Dropped AppNeta In addition to the above vendors, Viavi Solutions was formed from JDSU's network and service enablement (NSE) and Optical Security and Performance Products (OSP) business units. Inclusion Criteria Product-Related Criteria Vendors will be required to meet the following criteria to be considered for the 2016 NPMD Magic Quadrant and Critical Capabilities research: The ability to monitor, diagnose and generate alerts for: Network endpoints Servers, VMs, storage systems or anything with an IP address by measuring these components directly in combination with a network perspective Network components Routers, switches and other network devices, including SDN and NFV components Network links Elements that provide connectivity in the network-attached infrastructure The ability to monitor, diagnose and generate alerts for dynamic end-to-end network service delivery as it relates to: End-user experience The capture of data about how end-to-end application availability, latency and quality appear to the end user from a network perspective. This is limited to the network traffic visibility and is not within components such as what APM is able to accomplish. Gartner, Inc. G00276258 Page 23 of 35
Business service delivery The speed and overall quality of network service and/or application delivery to the user in support of key business activities, as defined by the operator of the NPMD product. These definitions may overlap, as services and applications are recombined into new applications. Infrastructure component interactions The focus on infrastructure components as they interact via the network, as well as the network delivery of services or applications. Support for analysis of: Real-time performance and behaviors Essential for troubleshooting in the current state of the environment. Analysis of data must be done within three minutes under normal network loads and conditions. Historical performance and behaviors To help understand what occurred or what is trending over time. Predictive behaviors By leveraging ITOA technologies, this provides the ability to distill and create action-oriented advice from the large dataset collected across the various data sources. Leverage the following data sources: Network-device-generated data Involves flow-based data sources, including NetFlow and IPFIX. Network device information Collected via SNMP. Network packet analysis Identifies application types and performance characteristics. The ability to support the following scalability and performance requirements: Real-time monitoring 10-gigabit (10G) Ethernet networks at full line rate. Ingestion of sampled flow records At a rate of 75,000 flows per second via a single instance of the product. Non-Product-Related Criteria Total NPMD product revenue (including new licenses, updates, maintenance, subscriptions, SaaS, hosting and technical support) must have exceeded $7.5 million for 3Q14 through 2Q15, not including revenue derived from security-related buying centers. The vendor must have at least 75 customers that use its NPMD product actively in a production environment. The vendor must have at least 10 customers located in at least two of the following geographic locations: North America, Latin America, EMEA and/or the Asia/Pacific (APAC) region that use its NPMD product actively in a production environment. Page 24 of 35 Gartner, Inc. G00276258
Table 1. Weighting for Critical Capabilities in Use Cases Critical Capabilities Network Operator Network Architect IT Operations Generalist IT Operations Manager Line-of- Business User Endpoint/Component/ Link Monitoring 23% 32% 10% 5% 0% Service Delivery Monitoring 10% 18% 15% 40% 60% Diagnostics 45% 20% 27% 5% 0% IT Operations Analytics 17% 13% 23% 20% 25% Integration and Interoperability 5% 17% 25% 30% 15% Total 100% 100% 100% 100% 100% As of January 2016 Source: Gartner (February 2016) This methodology requires analysts to identify the critical capabilities for a class of products/ services. Each capability is then weighed in terms of its relative importance for specific product/ service use cases. Critical Capabilities Rating Each of the products/services has been evaluated on the critical capabilities (see Table 2) on a scale of 1 to 5; a score of 1 = Poor (most or all defined requirements are not achieved), while 5 = Outstanding (significantly exceeds requirements).] Gartner, Inc. G00276258 Page 25 of 35
Table 2. Product/Service Rating on Critical Capabilities Critical Capabilities Automic Streamcore 6.3 and Sysload 6.0 CA Technologies Unified Infrastructure Management (UIM) 8.3 Cisco Prime Infrastructure 3.0, NAM 6.2 and Collaboration 10 Corvil Network Data Analytics Platform Tera (9.0) Flowmon Networks Flowmon 7.03 Fluke Networks TruView 10.0 and TruView Live 1.0 Genie Networks GenieATM 6.3.1 and VM HPE Network Node Manager i 10.01 and Real User Monitor 9.25 InfoVista VistaInsight for Networks 5.3 and 5View Suite NetScout ngeniusone 5.4 Niksun NetVCR Alpine 4.5 Paessler AG PRTG Network Monitor 15.3.17.29995/2996 Riverbed SteelCentral Suite SevOne PAS 5.4 and PLA 2.0 SolarWinds Performance Monitor 11.5 and Traffic Analyzer 4.1 Viavi Solutions Observer 17 Endpoint/ Component/ Link Monitoring 2.7 3.5 3.4 2.8 2.7 4.1 3.7 3.5 3.8 3.9 3.3 2.8 4.2 3.0 3.0 4.1 Service Delivery Monitoring 3.9 2.8 3.8 3.2 2.0 3.3 2.0 2.3 3.3 3.5 2.0 2.8 3.3 2.3 2.3 3.8 Diagnostics 1.0 3.5 2.8 3.3 1.5 4.0 2.0 3.1 2.5 3.0 3.0 3.3 3.0 2.9 2.5 4.0 IT Operations Analytics 2.5 1.8 2.0 3.5 1.0 2.5 1.1 2.9 1.0 2.0 1.9 1.0 2.5 2.9 1.0 2.2 Integration and Interoperability 1.0 1.5 1.5 3.8 2.0 2.7 1.0 1.0 2.3 2.2 1.0 3.8 2.0 4.3 1.3 1.3 Page 26 of 35 Gartner, Inc. G00276258
Critical Capabilities Automic Streamcore 6.3 and Sysload 6.0 CA Technologies Unified Infrastructure Management (UIM) 8.3 Cisco Prime Infrastructure 3.0, NAM 6.2 and Collaboration 10 Corvil Network Data Analytics Platform Tera (9.0) Flowmon Networks Flowmon 7.03 Fluke Networks TruView 10.0 and TruView Live 1.0 Genie Networks GenieATM 6.3.1 and VM HPE Network Node Manager i 10.01 and Real User Monitor 9.25 InfoVista VistaInsight for Networks 5.3 and 5View Suite NetScout ngeniusone 5.4 Niksun NetVCR Alpine 4.5 Paessler AG PRTG Network Monitor 15.3.17.29995/2996 Riverbed SteelCentral Suite SevOne PAS 5.4 and PLA 2.0 SolarWinds Performance Monitor 11.5 and Traffic Analyzer 4.1 Viavi Solutions Observer 17 As of January 2016 Source: Gartner (February 2016) Gartner, Inc. G00276258 Page 27 of 35
Table 3 shows the product/service scores for each use case. The scores, which are generated by multiplying the use-case weightings by the product/service ratings, summarize how well the critical capabilities are met for each use case. Page 28 of 35 Gartner, Inc. G00276258
Table 3. Product Score in Use Cases Use Cases Automic Streamcore 6.3 and Sysload 6.0 CA Technologies Unified Infrastructure Management (UIM) 8.3 Cisco Prime Infrastructure 3.0, NAM 6.2 and Collaboration 10 Corvil Network Data Analytics Platform Tera (9.0) Flowmon Networks Flowmon 7.03 Fluke Networks TruView 10.0 and TruView Live 1.0 Genie Networks GenieATM 6.3.1 and VM HPE Network Node Manager i 10.01 and Real User Monitor 9.25 InfoVista VistaInsight for Networks 5.3 and 5View Suite NetScout ngeniusone 5.4 Niksun NetVCR Alpine 4.5 Paessler AG PRTG Network Monitor 15.3.17.29995/2996 Riverbed SteelCentral Suite SevOne PAS 5.4 and PLA 2.0 SolarWinds Performance Monitor 11.5 and Traffic Analyzer 4.1 Viavi Solutions Observer 17 Network Operator 1.94 3.04 2.84 3.23 1.77 3.63 2.19 2.97 2.61 3.05 2.68 2.77 3.17 2.93 2.28 3.56 Network Architect 2.26 2.81 2.85 3.23 1.99 3.49 2.26 2.70 2.83 3.11 2.43 2.84 3.20 3.06 2.23 3.30 IT Operations Generalist 1.95 2.50 2.50 3.41 1.71 3.24 1.71 2.45 2.36 2.74 2.13 2.77 2.80 3.17 1.88 2.89 IT Operations Manager 2.55 2.28 2.68 3.43 1.81 3.04 1.61 2.13 2.53 2.81 1.80 2.77 2.78 3.09 1.79 2.76 Line-of-Business User 3.12 2.36 3.01 3.37 1.75 3.01 1.63 2.26 2.58 2.93 1.83 2.50 2.91 2.75 1.83 3.03 Gartner, Inc. G00276258 Page 29 of 35
Use Cases Automic Streamcore 6.3 and Sysload 6.0 CA Technologies Unified Infrastructure Management (UIM) 8.3 Cisco Prime Infrastructure 3.0, NAM 6.2 and Collaboration 10 Corvil Network Data Analytics Platform Tera (9.0) Flowmon Networks Flowmon 7.03 Fluke Networks TruView 10.0 and TruView Live 1.0 Genie Networks GenieATM 6.3.1 and VM HPE Network Node Manager i 10.01 and Real User Monitor 9.25 InfoVista VistaInsight for Networks 5.3 and 5View Suite NetScout ngeniusone 5.4 Niksun NetVCR Alpine 4.5 Paessler AG PRTG Network Monitor 15.3.17.29995/2996 Riverbed SteelCentral Suite SevOne PAS 5.4 and PLA 2.0 SolarWinds Performance Monitor 11.5 and Traffic Analyzer 4.1 Viavi Solutions Observer 17 As of January 2016 Source: Gartner (February 2016) Page 30 of 35 Gartner, Inc. G00276258
To determine an overall score for each product/service in the use cases, multiply the ratings in Table 2 by the weightings shown in Table 1. Acronym Key and Glossary Terms APM Application performance monitoring CSP EUE IM ITOA MPLS NPB NFM NFV SDN SNMP VPN Communications service provider End-user experience Infrastructure monitoring IT operations analytics Multiprotocol Label Switching Network packet broker Network fault management Network function virtualization Software-defined networking Simple Network Management Protocol Virtual private network Gartner Recommended Reading Some documents may not be available as part of your current Gartner subscription. "Magic Quadrant for Network Performance Monitoring and Diagnostics" "Technology Overview for Network Performance Monitoring and Diagnostics" "Market Guide for Network Packet Brokers" "How Products and Services Are Evaluated in Gartner Critical Capabilities" Evidence 1 Observations are based on the Magic Quadrant reference survey and more than 500 Gartner client inquiries and engagements with end users and vendors that have reported: Increased demand for network visibility from other aspects of the business Gartner, Inc. G00276258 Page 31 of 35
Initial immaturity of managing network environments that NPMD tools help address Extended requirements for NPMD tools to help justify network investments and aid capacity planning The need for tighter SLA compliance, especially in light of the increased adoption of IP-based voice, video and UC services, as well as HVD services Note 1 SNMP Polling as an NPMD Data Source Period polling is a method for quantifying the network use of network elements to gauge the requirements of the infrastructure. A polling-based approach can assess the existing environment and poll via the standard SNMP Management Information Base (MIB) to assess utilization on each network element. SNMP polling can also be used to gather error and volume data for instance, triggering an alert when a fault occurs or a predetermined threshold is breached. For example, it's possible to monitor the CPU, voltage and temperature of a device. If any of these parameters increases above expected levels, this may require investigation to prevent failure. In addition to error and volume monitoring, SNMP polling can be used to collect basic performance metrics to ascertain how much traffic is flowing through a network device. Based on the metrics gathered, the network team can estimate the delta between existing and required bandwidth needs on a per-location basis. A limitation of this method is the level of granularity that this approach offers. Data traffic is bursty in nature, which may be masked by polling intervals. The lack of application-aware segregation prevents correlating performance to specific applications. Take the following example. Location A has a constant 50% utilization during the 10-minute polling period. Location B has utilization of 100% in the first five minutes of the period and 0% in the second five minutes. A polling analysis would indicate that the two locations have a 50% average. The skewed data presents a misleading picture. Although Location A has sufficient contingency in terms of bandwidth, Location B is at full capacity for half of the time. Deploying new bandwidthintensive applications at Location B is likely to severely affect all services running at this site. An increase of polling frequency is an option; however, this adds to the overall overhead to the network, with every additional request for data and the response given from each element. Balancing the need for granularity with oversaturation of the network by management traffic is a fine line. Note 2 Packet-Based Technology as an NPMD Data Source Another approach is to examine the infrastructure in detail on a per-packet basis as opposed to sampling during a one- to 15-minute time frame. So instead of receiving a single data point every 10 minutes, as per the example in the SNMP polling subsection above, there would be data points for every packet traversed during the same 10-minute period, giving visibility into volatile traffic behavior from "bursty" application types. Page 32 of 35 Gartner, Inc. G00276258
If raw packet data is being gathered and examined, then the information collected is vendorindependent. Hence, the data has not been interpreted, summarized or manipulated in any fashion whatsoever. As a result, those relying on this information will have increased confidence in the accuracy of the data on which they're making decisions. This approach affords far greater insight; however, the offset is that a physical appliance or "probe" must be situated at each location. Otherwise, such detailed information cannot be gathered. These appliances have cost implications that can be excessive for an enterprise organization. Note 3 Flow-Based Technology as an NPMD Data Source A final method to provide further granularity is to take advantage of flow-based summarized data that provides the conversation between two network nodes (known as flow). Flow records also consume comparatively minimal storage space, compared with the raw packets that probe-based solutions leverage and may store. For example, as organizations move to 100GB networks, packet collection techniques become challenging, whereas flow-based methods remain a more-practical option. Flow-based data does not provide the detail down to a specific set of network packets going between the source and the destination and can have an impact on the devices where it is implemented. There are several flow collection standards. Vendor-derived standards are predominant and may not necessarily expose weaknesses to which the vendor may not wish to draw attention. However, they're well-integrated with the devices they're designed to monitor and are updated in anticipation of new hardware releases. The emergence of two standards (IPFIX and sflow) is a positive, because they are vendor-agnostic and, therefore, facilitate the management of a heterogeneous environment in a consistent manner. Comparing different vendors' devices is also facilitated. Such flow collection technology standards provide the same broad feature sets. Flow data collection is embedded in the network devices themselves. The device analyzes the network traffic moving from one interface to another, with the primary purpose of assessing bandwidth consumption and the level of data being sent and received among source and destination ports across the network. That data is then summarized into a stream of flow records that are sent to the monitoring tools collecting and assessing the flow records. Flow collection capabilities provide insight into which devices and applications are consuming bandwidth, how long the conversations are lasting and who is having these conversations. Because the data is summarized, a degree of detail is removed to simplify the ability to process and extract meaning from the actual network data. Critical Capabilities Methodology This methodology requires analysts to identify the critical capabilities for a class of products or services. Each capability is then weighted in terms of its relative importance for specific product or service use cases. Next, products/services are rated in terms of Gartner, Inc. G00276258 Page 33 of 35
how well they achieve each of the critical capabilities. A score that summarizes how well they meet the critical capabilities for each use case is then calculated for each product/service. "Critical capabilities" are attributes that differentiate products/services in a class in terms of their quality and performance. Gartner recommends that users consider the set of critical capabilities as some of the most important criteria for acquisition decisions. In defining the product/service category for evaluation, the analyst first identifies the leading uses for the products/services in this market. What needs are end-users looking to fulfill, when considering products/services in this market? Use cases should match common client deployment scenarios. These distinct client scenarios define the Use Cases. The analyst then identifies the critical capabilities. These capabilities are generalized groups of features commonly required by this class of products/services. Each capability is assigned a level of importance in fulfilling that particular need; some sets of features are more important than others, depending on the use case being evaluated. Each vendor s product or service is evaluated in terms of how well it delivers each capability, on a five-point scale. These ratings are displayed side-by-side for all vendors, allowing easy comparisons between the different sets of features. Ratings and summary scores range from 1.0 to 5.0: 1 = Poor or Absent: most or all defined requirements for a capability are not achieved 2 = Fair: some requirements are not achieved 3 = Good: meets requirements 4 = Excellent: meets or exceeds some requirements 5 = Outstanding: significantly exceeds requirements To determine an overall score for each product in the use cases, the product ratings are multiplied by the weightings to come up with the product score in use cases. The critical capabilities Gartner has selected do not represent all capabilities for any product; therefore, may not represent those most important for a specific use situation or business objective. Clients should use a critical capabilities analysis as one of several sources of input about a product before making a product/service decision. Page 34 of 35 Gartner, Inc. G00276258
GARTNER HEADQUARTERS Corporate Headquarters 56 Top Gallant Road Stamford, CT 06902-7700 USA +1 203 964 0096 Regional Headquarters AUSTRALIA BRAZIL JAPAN UNITED KINGDOM For a complete list of worldwide locations, visit http://www.gartner.com/technology/about.jsp 2016 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. or its affiliates. This publication may not be reproduced or distributed in any form without Gartner s prior written permission. If you are authorized to access this publication, your use of it is subject to the Usage Guidelines for Gartner Services posted on gartner.com. The information contained in this publication has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information and shall have no liability for errors, omissions or inadequacies in such information. This publication consists of the opinions of Gartner s research organization and should not be construed as statements of fact. The opinions expressed herein are subject to change without notice. Although Gartner research may include a discussion of related legal issues, Gartner does not provide legal advice or services and its research should not be construed or used as such. Gartner is a public company, and its shareholders may include firms and funds that have financial interests in entities covered in Gartner research. Gartner s Board of Directors may include senior managers of these firms or funds. Gartner research is produced independently by its research organization without input or influence from these firms, funds or their managers. For further information on the independence and integrity of Gartner research, see Guiding Principles on Independence and Objectivity. Gartner, Inc. G00276258 Page 35 of 35