Solution Guide: Application Performance Management

Transcription

1 SOLUTION GUIDE Solution Guide: Application Performance Management Making sense of the APM market jigsaw, vendor SWOT analysis, and Ovum APM 2012 Rainbow Map Reference Code: OI Publication Date: July 2012 Author: Michael Azoff SUMMARY Catalyst Application performance management (APM) today sits at a juncture that crosses many disciplines and domains: software development, applications in production, IT services, desktops, mainframes, web, mobile, cloud services, virtualization, application testing, network infrastructure, databases, and storage. APM vendors arrive in this space from different historical routes, but they all go under the same label. This report therefore aims to clarify the definition of an APM solution, and how vendors are taking their solutions forward. Ovum takes a representative group across new and established leading vendors, and provides a SWOT (strengths, weaknesses, opportunities, threats) analysis of their solutions. Ovum also conducted a comprehensive features assessment with the results summarized in the Ovum APM Rainbow Map. Ovum view APM tools can be categorized according to their target audiences. So development-oriented APM serves developers with the need to drill down to the code level, and provides the kind of information that programmers need and understand to solve code related problems. An operations-oriented APM tool will provide information about the production environment, and a single consolidated view of information meaningful to sysadmin and operations professionals. QA professionals will want details related to performance criteria that can be matched to the Ovum (Published 07/2012) Page 1

2 application's non-functional requirements. Furthermore, a new generation of APM tools monitor the end-to-end production environment, tracking distinct business transactions that cross multiple services and devices, and providing business transaction management. APM analytics is improving with new tools that can correlate thousands of metrics and identify patterns that provide early warning signs of impending trouble. The appearance of solutions that go beyond real-time monitoring and offer predictive analytics is another new market trend. Enduser experience monitoring is an APM category that focuses on how the user is experiencing performance. For example, green lights might be showing in production but the client side might be dead. This feature now needs to encompass applications running on smart mobile devices. Reporting to smart devices like tablets is also a new must-have. While some APM vendors aim for solutions that cover all monitoring and performance categories for the complete range of users, there are smaller players that are purely focused on operations and production. Ovum finds that the APM field is attracting new entrants with innovative solutions, so despite waves of industry rationalizations that occurred some years ago with all-encompassing solutions, the field is diverging again as more distinctly defined users are identified, such as DevOps, with their specific needs such as continuous deployment. APM in relation to cloud computing is a highly active area of solution enhancement by nearly all the vendors reviewed in this report. While some vendors can offer APM as SaaS, others are waiting to see how this market grows or are busy building such features. Taking APM into the cloud for private clouds is less of an issue than doing so on public clouds where there are limits on what can be deployed. Agents that accompany applications to public clouds is a common approach. APM for cloud service providers is another user category and an essential one as these providers look to differentiate themselves. APM solutions therefore need to keep pace not just with the changing nature of the data center, the transition to cloud services, use of server virtualization, messaging, virtual desktops, and so on, but also the categories of users who will use the performance information. Finally, we note in this report that given these changes taking place the applications can no longer be considered static but rather dynamic. Key messages The broad APM functionalities are converging in a single solution set for covering the data center and private cloud environments, but the key user groups are given targeted information. Ovum (Published 07/2012) Page 2

3 The key APM user groups are systems administrators and operators, application developers, QA, IT service management professionals, and senior managers and executives. APM analytics has become a major concern and active area of innovation, due to the sheer growth of data to be processed. Advanced techniques such as predictive analytics can quickly identify potential problems before they escalate and bring down systems. The cloud is impacting APM in a number of ways, from SaaS APM offerings, to solutions that can accompany applications to public cloud platforms, to tools that seamlessly track transactions across all tiers in a virtualized private cloud environment. Mobile devices are increasingly being used in the enterprise, and APM solutions must be able to monitor end-user experiences and app performance. Mobile devices are also becoming popular for displaying dashboards to APM user groups. The APM market is going through a revitalization as new vendors emerge and new generation solutions appear from established players. Ovum (Published 07/2012) Page 3

4 TABLE OF CONTENTS Summary 1 Catalyst 1 Ovum view 1 Key messages 2 APM Market Developments 7 Market view 7 Technology view 8 APM market trends 11 Solution Guide for APM 17 Competitive landscape 17 General Assessment 20 Capability assessment 20 Vendor Analysis 23 BlueStripe 23 CA Technologies 24 Compuware 27 ExtraHop Networks 28 HP30 IBM 32 itrinegy 34 Nastel Technologies 36 NEC 38 NetScout Systems 40 Netuitive 42 Opsview 44 Ovum (Published 07/2012) Page 4

5 OpTier 46 Quest 48 Appendix 51 APM Features Matrix and Ovum's Weightings 51 Methodology 57 Author 57 Disclaimer 57 Ovum (Published 07/2012) Page 5

6 TABLE OF FIGURES Figure 1: Ovum APM technology stack 10 Figure 2: Ovum Solution Guide for APM 2012: key vendors assessed 18 Figure 3: Ovum Rainbow Map 19 Ovum (Published 07/2012) Page 6

7 APM MARKET DEVELOPMENTS Market view Application performance management (APM) is currently going through a revitalization following cloud and DevOps technology evolutions (possibly revolutions), as well as innovation in areas such as analytics and real-time monitoring, so it is timely to define exactly what is meant by the term APM. Some four years ago the market appeared to be consolidating but the widespread adoption of virtualization and the recognition of the importance of end user-experience monitoring (EUEM) created new opportunities. The APM market today is seeing further diversification, so the modern, broad definition of APM is an application-centric performance monitoring and management solution that cross-cuts IT tiers including network infrastructure, physical and virtual infrastructure (server and OS), storage and databases, end-user experience with edge devices, and integration into general IT service delivery, with these tiers running in and across traditional data centers, cloud environments (private, public, hybrid), and SaaS providers. The modern APM solution is designed to address different IT cultures, their languages (view of the world), and needs. While movements such as Agile and DevOps attempt to break the walls between traditional silos such as development, QA, operations, network engineering, senior management, and others, there remains the need to address the particular information requirements of these communities. APM tools either provide across the board functionality with interfaces targeting the specific communities or they will be solely focused on one or two communities. APM for pre-deployment development and post-deployment troubleshooting therefore provides developers with source-code-level detail and statistics, whereas operationsoriented APM is concerned with live production where operators are concerned with end-user experiences and end-to-end service delivery, but for troubleshooting they only need details to method call levels. From a process viewpoint, operations also want to be able to have the right information that enables them to identify who the best team members are to solve the issue. Furthermore, those in operations care about the resource overheads incurred by deploying APM agents in production, and even low-overhead agents can in sufficient numbers drain production resources and also cause potential perturbations. So the type of solutions these communities want in place can differ markedly. In addition, there are two areas closely related to APM that Ovum believes will see increasing activity within the APM market: application performance testing and application security performance. The former is naturally of interest to QA and testing professionals, and APM tools that target this audience will include synthetic and real-user load-testing. Vendors include CA Ovum (Published 07/2012) Page 7

8 (which recently acquired ITKO, a pioneer in service simulation technology), HP, and IBM (which recently acquired Green Hat, another pioneer in service stubbing and virtualization). The second area, cyber security, has become an application-level concern as enterprise users increasingly work on devices that sit outside the firewall such as web browsers and mobile devices. This is an opportunity for the APM market to address. Finally, specialized APM solutions that target particular enterprise applications are available where the complexity of the application and the criticality to the business demands targeted tooling. GSX Solutions is one example of a company that provides a single software solution for monitoring Microsoft Exchange, Microsoft SharePoint, BlackBerry services, and Lotus software at the same time, using remote agentless technology. Unlike general APM solutions, GSX knows its targeted application deeply and can provide a level of analysis and reporting that administrators of these business applications will find immediately helpful in their daily maintenance activities. Another example is Knoa Software, which provides user experience monitoring and management for enterprise applications such as SAP and Oracle. How end users perform their daily tasks on these systems is not normally tracked unless the application is misbehaving, but Knoa is able to analyze usability performance in order to help users make optimal use of the often-complex application interfaces. Technology view Application performance management solutions on the market vary according to which technology direction the vendor has historically started from, such as applications or network. The technology categories covered by the solutions can therefore be split as follows (see Figure 1): Development-oriented: Root-cause APM satisfying needs of development teams with programming and testing support, through to lifecycle support in production with instrumentation of virtual machines and application servers for optimal code performance. Operations: A growth area with the emergence of DevOps, the tools are designed to be used by sysadmin and operators, talk their language and solve their production environment problems. Network infrastructure: This is network performance management (covering LAN and WAN) rather than APM, but the need for a unified view (application plus network) is leading to integrated solutions with operations performance management. In addition, unified communications (including video and VoIP) lead to increasingly sophisticated Ovum (Published 07/2012) Page 8

9 traffic (including video and voice-over-ip elements) with specialized performancemanagement needs. Business transaction management: Tracking a transaction view, as for example initiated by a business user of a service that crosses multiple applications and services. End-user experience monitoring (EUEM): covering client side, from desktop clients to web browsers and mobile devices. Strictly this covers availability, but aspects of EUEM overlap into website/application analytics and optimal behavior monitoring. Physical, virtual, and dynamic infrastructure, on cloud and traditional data center, including SaaS delivery channels. Storage and database: APM for storage and databases are niche specialist areas. While most APM solutions will provide traffic monitoring for data going in and coming out, only a few will instrument storage/database internals. Executive reporting and dashboards. A solution that provides a high-level dashboard for decision-makers who need easy-to-digest indicators such as traffic lights. Legacy systems, such as mainframes: These are still in use in industries such as financial services, and need to be tied in to ensure performance monitoring continuity. Many solutions simply measure the data going in and data coming out. Innovations in the APM space that cross-cut the above APM categories include: Advanced (Big Data) analytics: This is an increasingly sophisticated sub-topic within APM that aims to deal with the many hundreds of thousands of metrics generating data every second, and making sense of the mountain of data. Techniques tend to be either advanced statistics based, such as those based on correlations, or patternmatching. Always-on APM suitable for production monitoring in contrast with heavier instrumentation for pre-deployment testing and after failure event troubleshooting. End-to-end, 360-degree view solution that integrates many of the APM categories listed above. Increasingly this category also includes coverage of web application performance across geographical divides. Dynamic APM: The rise of new generation technology such as cloud services, virtualization, and messaging, means that applications are dynamic rather than static, and places greater demands on APM solutions to make sense of application behavior. Ovum (Published 07/2012) Page 9

10 Figure 1: Ovum APM technology stack Source: Ovum O V U M Increasing application dynamism needs multi-tier, end-to-end APM Modern multi-tier applications and services built for SOA environments are more dynamic than ever before, and this dynamism falls into three dimensions. First, the infrastructure on which applications run can switch in an instant across different cloud servers and virtual machines. Second, the applications may comprise mashups pulling in data from diverse sources, and rely on multiple and distributed components and services. Finally, applications are delivered faster and deployed more frequently than before with the rise of Agile methodologies, continuous delivery, and DevOps. For example, continuous delivery establishes a deployment pipeline that can trigger a code change to production use in a matter of minutes. All three dimensions demand more from an APM solution. Solution configuration should be easy to keep up with frequent Agile drops, it must recognize a distinct business transaction that cross-cuts multiple IT layers, it must follow a transaction across physical and virtual boundaries, and it needs Ovum (Published 07/2012) Page 10

11 to support the needs of different users from developers needing source code line detail to operators concerned with stability and service delivery in the production environment. User deployment approaches for APM There are various ways in which APM solutions can be deployed, dependent on the needs and roles of the users. The four main communities are operations, development, QA, and IT services management, with perhaps end users forming a new fifth grouping. Increasingly, a unified view that addresses the needs of all these communities is required by businesses that are heavily reliant on IT. APM solutions therefore need to be end-to-end, operating 24x7, and able to pre-empt problems before they escalate. In many businesses, point solutions are often brought in to deal with particular problems. For example, developers may need to instrument a Java Virtual Machine, or operations is finding that it needs to monitor end users that are suffering issues despite the server side apparently functioning normally. The collection of these point solutions cannot solve issues efficiently that transcend any one IT layer. Especially in a large enterprise environment the lack of a unified view can lead to significant time-wasting in resolving issues, but for any business reliant on mission-critical applications a comprehensive APM solution is essential. Another user category that will find APM essential is the cloud or SaaS provider, whose business model is based on a minimum up time with multiple nines reliability. APM market trends APM analytics has reached big data proportions In a large data center an APM solution can generate thousands of metric data points per second. This data avalanche will contain the health information about the IT environment, and the challenge is to manage the sheer scale of this data. One approach is to store it in a data warehouse and process it with sophisticated analytics tools that are designed to deal rapidly with the large volumes. An alternative approach is to treat the problem as Big Data and sample the data in real time. Both approaches have led to innovation in the APM market place. Advanced analytics tools are available to enhance legacy APM solutions A number of technology initiatives have led to a growth in the data generated by APM solutions. These tools are providing end-to-end application and service monitoring, covering every tier from network to business transactions. Also service-oriented architecture, which is not quite as dead as some might believe, results in an increase in data traffic to be monitored. Furthermore, virtualization and cloud computing are creating a dynamic environment with an increase in the complexity of metrics to be monitored. In order to keep pace with the sophistication of the IT Ovum (Published 07/2012) Page 11

12 environment the APM solutions started with static threshold-based monitoring but then progressed to event correlation, dynamic thresholding, and pattern matching. The next step to deal with the data overload problem has been tackled by specialist APM vendors such as NEC and Netuitive whose Predictive Analytics and Masterscope Invariant Analyzer respectively are designed to complement existing APM solutions. These tools are statistics-based and employ self-learning algorithms (using neural networks in the case of Netuitive) that process metric data in real time to make sense of the data avalanche and identify issues that need attention. They are based on multivariate correlation and regression and can also provide forecasts of issues that are likely to escalate and bring systems down. Another analytics approach is that taken by OpTier and Nastel, which have a complex event processing (CEP) engine to process data in real time. An alternative approach is to deploy a network appliance such as ExtraHop, which is passive and can monitor data flow in real time without the need for storage or indexing the data. With networks operating at 10 Gbps and data collections in excess of 100TB per day, many solutions resort to sampling rather than inspecting all the data, whereas a network appliance can deal with all data traffic. A start-up vendor that has just gone live with a new product recently is Boundary, which offers a SaaS solution for APM-related Big Data monitoring, charging users an hourly usage rate. Customers such as GitHub are discovering patterns of traffic at highly detailed granularity, with snapshots that are true real-time, to aggregates that help spot unusual behavior. Boundary stores the data as part of its hosted solution on Big Data databases and is ideal for monitoring in public cloud environments where an appliance is not possible. Leaders in the APM space such as CA, HP, and IBM have also created new-generation predictive analytics engines to complement their existing tools, enabling them to scale up their metricsprocessing capabilities. OpTier is also active in big transaction data processing, helping to capture unstructured data and converting it into structured data. The scale of this issue is fast becoming a Big Data problem, and large data center administrators will need to assess their existing APM solutions when handling these volumes of data. APM on and for the cloud Ovum invited the key report vendors to comment on the future of APM, and in particular what effect cloud (IaaS, PaaS, and SaaS) services will have on APM. This section is based on a selection of their responses. APM being offered as SaaS is available from many of the APM vendors, and others are busy enhancing their solutions in the expectation that this market will grow significantly. Ovum (Published 07/2012) Page 12

13 On the question of whether IaaS and PaaS providers should offer an APM solution embedded in their service, IBM feels that customers do not want to deal with multiple APM solutions, but instead prefer one solution that spans cloud, virtual, traditional, and hybrid environments. However, IBM believes that APM as part of a PaaS offering will eventually become a requirement, although there are security considerations to consider, and customers today prefer a single APM solution as mentioned, to span everything. Compuware believes that customers expect APM solutions to be an embedded part of XaaS services, so while the burden is on the service providers to provide proactive APM reporting back to consumers, Compuware also believes that many customers will self-procure APM solutions because it is ultimately their brand and business opportunity, and SLA penalties from the service provider will not mitigate the impact to the business. For IBM, the migration of application workloads to cloud platforms via IaaS, PaaS, or SaaS models means that the APM solution must be able to discover application workloads on private and public clouds, deploy the monitoring solutions to those infrastructures, and be able to send data and alerts back to the monitoring solution. This requires flexible data gathering techniques, such as non-agent application data collection, and the cloud dynamic also requires new ways of visualizing APM data. Therefore a new breed of cloud administrators will require a high-level view of application health and cloud resources consumed, and software application stakeholders need to be concerned with the way application workloads are running in the cloud as much as with user experience. IBM is addressing these issues with its IBM Tivoli SmartCloud offerings. HP sees APM as an integral part of cloud computing, recognizing that the cloud model demands a new mindset about how applications are developed, deployed, monitored, and managed. For example, on the cloud you may be trying to monitor resources you may not own, so it is essential to have rigorous monitoring practices to identify the exact root cause of problems that occur. It can also be very difficult for developers to diagnose and fix cloud-based application issues without performance data and end-user information from IT operations monitoring teams. Compuware believes that enterprise adoption of the cloud as a deployment platform will continue to accelerate. Cloud applications are built from assets provided by publicly shared third-party service providers, private clouds, and data center assets, and increasingly powerful end-user clients. In order to monitor the health of these applications, broad visibility across the application delivery chain is critical, spanning publicly shared third-party services, enterprise private clouds, enterprise data centers, and mobile, tablet, and browser-based clients. There is a general consensus among the APM vendors that XaaS providers will in time be expected by their customers to provide detailed application-performance metrics as part of their Ovum (Published 07/2012) Page 13

14 services. As Netuitive points out, this should ideally be driven by common industry standards, but as Netuitive also warns, it could end up being driven by one or two large vendors such as Microsoft or VMware. Ovum sees Amazon AWS as creating a de facto industry standard in cloud API. This would be an opportunity for Amazon to drive a cloud APM API, but there is no sign of this happening. CA recently commissioned a study that explored the intention of organizations in the medium-tolarge-enterprise market segment it targets, to adopt end-user experience monitoring via SaaS. According to the study, 14% of respondents purchase end-user experience monitoring via SaaS, and this number is expected to increase to 24% within the next two years. CA views SaaS as giving users greater flexibility around budgets for ongoing operations, while removing upfront and capital costs. This increased flexibility also relates to the deployment model. In addition, the speed with which a service can be implemented and the ease with which it can be used is important. Customers want what they want, when they want it, and they only want to pay for what they use if and when they use it. CA has made a strong move to address this market trend by acquiring Watchmouse and rolling out CA APM Cloud Monitor. Similarly, CA believes the advent of the cloud-connected enterprise (CCE) will transform the way in which organizations consume service-assurance solutions because they would typically favor managed services or SaaS. While SaaS-based network/event management solutions are expected to grow rather slowly, APM-as-a-service is quickly becoming a reality, which according to CA is contributing to 9% of the total APM market, and is expected to grow to 17% by Therefore, while there is no significant pressure for network/event management-as-a-service, there is strong pressure for APM-as-a-service. CA believes the long-term future of full-featured APM, not just the remote synthetic monitoring capability, lies with the delivery of a SaaS-based model. CA recognizes that APM for the data center requires a convergence and integration of the various segments that have traditionally been separate disciplines, namely APM, infrastructure management, and services operations, to yield end-to-end APM with comprehensive integration across multiple tiers. However, CA sees the demands of CCE as pulling in the opposite direction, with service providers owning the infrastructure and back-office applications, and enterprise IT owning the differentiating bespoke application logic and the management of the underlying service providers. CA therefore sees the CCE model as creating a convergence challenge that has already been addressed in the data center model. CA has addressed this by offering the same CA APM whether in virtual and cloud environments or non-virtual and data center environments. Quest addresses APM for the cloud by ensuring that its Foglight solution is designed to support both public and private cloud-based architectures. Foglight agents communicate over well-known Ovum (Published 07/2012) Page 14

15 ports as well as data-collection, storage, and sending techniques. However, for cloud environments that are restricted in what they allow, Foglight can function without requiring local agents to collect information. Quest sees APM-as-a-service as gaining in importance over the next two to five years. Foglight is available as a managed service, and Quest is validating the need for a broader cloud-based APM offering for the future. Optier sees the use of APM SaaS as slowly benefiting larger parts of the market. As long as most large enterprises keep many applications off the cloud for various reasons, they will also be slow to adopt APM in the cloud. OpTier's product is already cloud-ready. Its out-of-the-box capabilities help clients plan, migrate, and operate in a cloud environment, as well understand change impact, resource consumption, cost of transactions, and more, to assist with accurate provisioning and chargeback. Optier has no immediate plans for a SaaS offering, but is considering it for the future. SaaS APM does have the advantages of lower costs (pay as you use) and flexibility, and new entrants such as Boundary are pure SaaS offerings. Ovum expects this space to be one of the hottest in APM in the years ahead. APM for mobile devices Smart mobile devices are having an impact on enterprise staff because the consumer-led pattern of adoption is crossing over to the workplace, led by the top C-level executives and by staff on the ground and field. This is affecting the APM market in two ways. First, there is a need for APM solutions to monitor app performance on smart mobile phone and tablet devices, whether through synthetic or real user experience monitoring. It is important here to consider the challenges of monitoring a given app when accessed on different devices at different times. Second, the APM reporting and dashboards need to be geared to deliver performance metrics and results to mobile devices, for both executives and operations administrators. In this review, all the key vendors offer mobile device APM except BlueStripe and Nastel, and all vendors offer a mobile device dashboard, except BlueStripe, Opsview, and Quest, and these vendors are likely to enhance their offerings in these respects in the near future. The extent of coverage of mobile device APM features may be further ahead in some than others. For example, synthetic monitoring may be available, but real-user experience monitoring may not. APM innovation in web performance monitoring Gaining maximum return on investment from an APM solution can require technical expertise to interpret the health reports generated by the solution. If this expertise is in limited supply within the organization this reduces the solution's benefit. A new startup that launched its first product in December 2011 is Lacuna Systems, whose solution Indico, which is designed to be quick to install, Ovum (Published 07/2012) Page 15

16 immediately provides health warnings by targeting a sweet spot in the IT infrastructure. Lacuna believes it has no rivals to compete with, at least not yet. All data centers of any meaningful size make use of application delivery controllers (ADCs) to remove load from web servers, and through which web applications are deployed. The ADC hardware market is essentially ruled by four manufacturers: F5, Cisco, Citrix, and Radware, and these controllers produce a wealth of statistics, such as throughput, volume of connections, and connection rates, grouped by servers and applications. Indico with a single deployment can remotely access ADCs in any number of data centers and can alert issues at an early stage of development before a system goes down. Indico uses its intelligent thresholding technology, leveraging Lacuna s native Big Data analytics engine, to baseline and define normal application performance so as to proactively alert in real time when performance falls outside of expected norms. Ovum (Published 07/2012) Page 16

17 SOLUTION GUIDE FOR APM Competitive landscape Ovum takes a close look at the competitive landscape of relevant technology solutions. The assessment is a quantitative and qualitative representation of Ovum's views and opinions about the competitive APM market environment. The list of included vendors is not intended to be exhaustive, but representative, offering readers an in-depth analysis of the leading vendors in the context of the specific technology area. The final mix of selected vendors provides a good balance between leading established players and newer solutions from smaller vendors. The key vendors included in this solution guide are listed in Figure 2. Of the selected enterprise IT management vendors (BMC, CA, HP, and IBM) only BMC is missing, having pulled out mid-stream. Ovum believes this is because BMC is in the process of launching a new solution and is not yet ready to go public. It is advisable that readers consider other solutions not covered in this report, for example AppDynamics and Splunk, and in the purely APM analytics space Prelert and Boundary (launched during the writing of this report). Ovum asked the reviewed vendors to which segments of the APM coverage areas their solutions were applicable, where the segments were itemized as follows: Business processes (multi-applications/services) Application-level transactions Network infrastructure Performance monitoring and management Performance testing Predictive IT performance analytics Two of the vendors, NEC and Netuitive, offer solely predictive analytics solutions that are designed to work with and require APM solutions in place. For this report these two vendors solutions are considered complementary APM solutions and are not included in the Ovum APM Rainbow Map, but a SWOT analysis is provided. Of the APM vendors reviewed, NetScout and Opsview did not offer performance testing, and BlueStripe and Nastel did not cover network infrastructure, otherwise the vendors covered every Ovum (Published 07/2012) Page 17

18 segment in the above list. Where the above-mentioned vendors do not cover a space they will integrate with third-party solutions. Looking at APM from a TCP/IP layer perspective, the Open Systems Interconnection (OSI) model defines the abstraction layers of communication systems, where a layer serves the layer above it and is supported by the layer below it. The OSI model is briefly defined as: Application, presentation, session layers: L5-L7 Transport layer (flow control): L4 Network layer (data packets): L3 Data link and physical layer (switches and binary transmission): L1-L2 The APM solutions reviewed all covered L3 to L7, with the exception of Nastel, which used a third party for L4. However, with L1 and L2 this is where the main divide between network performance management and APM occurs, so only some vendors offered solutions here. These were CA, Extrahop, HP, IBM, NetScout, Opsview, and Quest. All the vendors reviewed here offer software APM solutions except ExtraHop, which only provides an appliance (combining software and hardware). NEC and Netuitive provide analytics solutions that are designed to complement APM and other IT management solutions, and are not included in the Ovum APM Rainbow Map, which is designed for full APM solutions. Figure 2: Ovum Solution Guide for APM 2012: key vendors assessed Source: Ovum O V U M Ovum (Published 07/2012) Page 18

19 Figure 3: Ovum Rainbow Map Source: Ovum O V U M Ovum (Published 07/2012) Page 19

20 GENERAL ASSESSMENT Capability assessment The vendor solutions were assessed against a set of broad APM features. Vendors were invited to provide feedback on the structure and content of the features matrix (FM) and were then asked to complete the final version. The weightings that Ovum used in assessing the different sections are revealed in the appendix, and it should be kept in mind that customers will vary with their particular requirements and should not simply select vendors on the basis of the scores achieved in this solution guide. For example, customers may have legacy APM tools in place and may be looking to fill gaps in their coverage. The final scores of the FM were aggregated by section and plotted on a heat-style map, except we refer to it as the Ovum Rainbow Map given that the rainbow sequence of colors are used to rate the scores. The final Rainbow plots are shown in Figures 3 and 4 with the color key shown in Figure 5. In the following sections the broad APM features assessed are described qualitatively. Application performance optimization This is the core of APM, and covers features such as application performance monitoring, and application availability. Ovum weighted two features in particular highly here: the capability to capture all transactions 24x7 end-to-end from browser through web and application tiers to database and back, and to provide deep visibility into transactions down to code level. Application platform coverage This section assesses the platforms covered, in particular whether the solution is applicable to applications in virtualized and cloud environments, and mobile applications. Topology and change impact analysis The auto-discovery of transactions flow topology is considered important, as are dependency mapping and automated features for transaction visualization and bottleneck detection. End-user experience monitoring End-user experience monitoring from the user's perspective, such as from inside the user's browser, is a hot topic in APM. This set of features can also overlap into website usability, such as tracking visitor satisfaction and web page landing statistics. The deep tracking of users optimal Ovum (Published 07/2012) Page 20

21 understanding of enterprise application GUI interfaces is also covered here. Most highly rated features were: monitoring across all devices, all browsers, capture click paths and Web 2.0 page actions; and trace logic in Web 2.0/Ajax-based end-user clicks into the server side. Application performance testing Performance testing is an important activity in QA, and for organizations with IT development on site this will be a must-have feature. Solutions will need to run performance testing for synthetic users and real users, as well as website testing from diverse locations. Network performance management Network performance management overlaps with broadly defined APM, and here Ovum assesses infrastructure performance optimization and network modeling capabilities. In the former case key features are network performance monitoring, network availability, network traffic management including load management, and being able to identify and pinpoint network bottlenecks between application components. Network modeling includes capabilities such as network scenario and capacity modeling and simulation, and end-user response time modeling. Business transaction management and SLAs BTM is the capability to trace business transactions, looking at services holistically, with all components that make up a service, and assess business impact of issues. It has become a hot area in APM, providing a business-level understanding of the impact of problems arising deeper down in the application and infrastructure layers. The need to manage service-level agreements is also assessed here. Unified communications (UC) performance management Unified communications applications such as for voice, video, video conferencing, desktop video conferencing, and telepresence, are an increasingly important use-case scenario for organizations, and APM solutions need to be able to monitor these systems. Diagnosis and root-cause analysis This is the core analytics functionality, offering event correlation, source code level identification troubleshooting, data packet-level analysis, and with more advanced solutions, automated triage and repair. Ovum (Published 07/2012) Page 21

22 Predictive IT performance analytics Predictive analytics is at the cutting edge of advanced APM, with self-learning modeling and behavior learning engines, automated problem pattern detection, CEP engines, and other techniques in anomaly detection and prediction. Database and middleware support Many APM solutions will cover databases by monitoring delays in signals going in and coming out of the database, but some solutions perform deep dive database optimization. Good coverage of diverse middleware is also a necessity for large data centers. Cyber security Many applications are now accessed via a browser so the traditional firewall is no longer the frontline in protecting the data center from malware. Web applications are allowed to tunnel past the firewall and the applications therefore need to be secure from within. Cyber security is about detecting weak spots in applications and the infrastructure, proactively identifying issues before security breaches occur. Admin, reporting, and alerts APM solutions have in the past been difficult to configure and administer. The aim is near-aspossible zero-configuration out-of-the-box deployment, and auto-discovery of applications, devices, transactions, and users. Dashboard metrics and health reporting, both real-time and historical, and reporting to smart mobile devices are assessed here. Alerts cover KPI-based alarming and policy-based alarming. Ovum (Published 07/2012) Page 22

23 VENDOR ANALYSIS Ovum conducted briefings with vendors and produced a SWOT analysis in each case. BlueStripe Profile The co-founders of BlueStripe are ex-ca Wily executives who brought a number of Wily developers over to help create a new APM product. BlueStripe FactFinder is a new-generation product (the company talks about APM 2.0) that is operations-friendly and targets the growing need to manage transactions across multiple tiers and also identify where problems are located. The real-time topology dependencies map discovers servers, networks, and linked devices. Agents sit on servers and track transactions across multiple layers, including cloud and virtual environments (the vendor quotes low agent overhead in region of 0.1% to 0.4%). BlueStripe conducted an IT operations survey and found that respondents rated the (long) time it takes to identify the cause of business-impacting outages and slowdowns as the number-one issue. Reducing this delay is the key aim of the vendor, as well as addressing the needs of operations staff to deal with issues without the delays ensued by appealing for assistance from developers. Administration is simple with one person sufficient to support over 100 servers. The tool integrates with IBM Tivoli Application Dependency Discovery Manager, BMC Atrium, and HP Change and Configuration Management Database. FactFinder's data is gathered from operating-system level agents that monitor all TCP-connected processes on a given system, including all connection activity, transaction performance, and underlying system resource utilization. SWOT analysis Strengths The company's go-to-market strategy of targeting IT operations executives is finding a responsive audience because a major issue is being able to link failing business transactions to failing infrastructure. FactFinder has strong dependency technology matched with hybrid analytics for performance problem resolution, the Correlatron feature. One thousand servers yielding 35 million performance metrics can use dependency analysis to reduce the problem to 10 servers in the direct application path, reducing the scale of the problem to 5,000 metrics, followed by correlation analytics that further reduce the scale to 10 metrics that matter. Correlatron can isolate problems Ovum (Published 07/2012) Page 23

24 to core issues. The tool is aimed primarily at operations staff with real-time time transactionmonitoring needs. Weaknesses BlueStripe is best considered as a solution to complement existing APM tools because it lacks some essential APM requirements such as end-user monitoring and source code line problem identification. This may be a disadvantage in a green-field setting, but most organizations have legacy APM solutions so the company does have a market opening. BlueStripe is not targeted at developers and QA professionals. Opportunities Organizations with legacy APM tools will have the option to retain these tools and through BlueStripe be able to improve problem resolution using its FactFinder Correlatron solution. The vendor is considering a SaaS offering. Threats The APM market has many vendors and rationalization is likely to take place. BlueStripe may well become an acquisition target of a broader solution APM vendor looking to strengthen its dependency analysis and correlation analytics. Recommendation for enterprise IT customers Ovum believes BlueStripe should be evaluated for real-time performance monitoring and rapid analytics-based resolution to complement legacy APM solutions in the data center. CA Technologies Profile CA Technologies is a member of the Big Four in the enterprise IT management space, with a portfolio of role-based solutions offering IT service assurance. CA APM is the core of and driving force behind the company s Service Assurance solutions that link applications, real users, transactions and services with the underlying IT infrastructure to provide a comprehensive, unified understanding of how applications and infrastructure deliver business services across physical, virtual, cloud, and mainframe environments. Transactions form the core around which the strategy and the company s APM products are based, with end-to-end measurement of every transaction, deep monitoring of key software Ovum (Published 07/2012) Page 24

25 components that make up a transaction, and mapping of physical or virtual systems to transactions. This enables an understanding of end-user experiences across the enterprise and cloud, and the establishing and monitoring of SLAs. It also proactively detects issues, and provides problem triage and the diagnosis of application, network, and infrastructure root causes. CA APM is powered by an APM transaction model that discovers and maintains the dynamic relationships between business services, transactions, and their logical application structure. This data is consumed by the Unified Service Model (USM), the centerpiece of the CA Technologies Catalyst integration and automation platform for delivering a complete 360-degree view into the IT environment. Catalyst uses open standards web services communications, so any tool can create a connector into Catalyst, including third-party products. The underlying data unification in CA APM ensures for example that end-user experience monitoring is linked to application problem diagnosis. In the recent 9.1 release, CA took this a step further by unifying end-user experience and network performance monitoring on a single appliance that provides a single source of truth for how network behavior impacts the end-user experience. This also helps organizations to understand how infrastructure components affect service quality and how effective the network is at delivering applications to users. The CA APM solutions also share a common transaction model, so that a common business transaction can be defined and monitored, such as, for example, login and transfer funds transactions in banking applications. In addition to CA APM, Service Assurance includes several integrated modules. Infrastructure Management provides comprehensive monitoring, analysis, and reporting into the infrastructure. Service Operations Management correlates and analyzes information from infrastructure, application performance, and other management. It visualizes service components end-to-end, and dynamically calculates how they impact services or put them at risk. Executive Insight offers real-time access to high-value metrics that measure the pulse of business. It is designed specifically for the mobile platform. CA Technologies recently acquired Watchmouse, which has been rebranded CA APM Cloud Monitor. The solution offers synthetic transactions to monitor performance, and a geographic knowledge of end-user performance using more than 60 monitoring stations in 40 countries. For cloud-based applications outside the enterprise s control, CA APM Cloud Monitor observes applications that are delivered by SaaS and managed service providers to quickly and easily check Ovum (Published 07/2012) Page 25

26 the availability and performance of cloud applications, manage SLAs proactively, and prove that SaaS vendors and MSPs are meeting service delivery commitments. SWOT analysis Strengths CA APM, the lead product in the company s Service Assurance portfolio, is able to offer each IT stakeholder a solution relevant to their needs with a unified underlying data integration framework feeding these solutions. Important in keeping up with trends, CA Technologies can offer a cloudmonitoring solution that content delivery providers, for example, will want. CA Technologies continues to be a major vendor in this space and can offer advanced solutions in the areas examined. Weaknesses CA Technologies has few weaknesses. One area that Ovum would like to see an improvement in is integrated cyber security with APM, which is normally reserved for specialist consultants and tools, and which Ovum believes should be automated as much as possible within a broader APM offering. Ease of tool administration is another area where CA Technologies can improve. Opportunities CA Technologies is well placed to continue its leadership in the APM space with plenty of opportunities for further acquisitions that can fill out any emerging gaps as technology continues to evolve. Threats Complacency is perhaps the only threat, but from all evidence CA Technologies is keeping abreast of the latest technology challenges. Recommendation for enterprise IT customers Ovum's recommendation is to ensure that the Big Four are on your shortlist, which includes CA Technologies. It is always a good policy to maintain some degree of tool heterogeneity in your data center in order to maximize bargaining leverage when negotiating with vendors. CA Technologies data integration framework and ease of connecting to it are a definite plus. Ovum (Published 07/2012) Page 26

27 Compuware Profile Compuware has in recent years made APM its key strategic offering, and acquired Gomez and dynatrace to bring that aim to fruition. Married with its existing product, Vantage, the company continues its integration roadmap, and its new product branding is Compuware APM under which dynatrace and Gomez form the enterprise and cloud offerings respectively. Integration between the two offerings ensures unified coverage of application delivery. Vantage, which covers business services management features and is aimed at IT executives, has been integrated into Compuware APM. Compuware's integration roadmap will see completion by end of Customers will be able to choose individual solutions from within the Compuware APM suite should they wish. Compuware APM dynatrace represents a new-generation APM solution that provides end-to-end tracing of transactions, from single to aggregates, and drill-down to source line code detail for troubleshooting. At the core of dynatrace is its patented PurePath technology, which deploys lowoverhead agents across the IT environment. These agents capture real-time performance data for production monitoring, with sufficient detail for QA and developers to analyze the root causes of issues. Compuware APM Gomez is the cloud, web, and mobile APM solution. The rise of enterprise mobile device usage necessitates APM cover, including with synthetic monitoring (performance testing). Gomez can now bring dynatrace PurePath technology into play across cloud and mobile networks. Gomez also features new packet error and quality-of-service metrics. Compuware's cloud APM is designed for managing applications from inside and outside the cloud, ensuring enduser satisfaction, operational monitoring of dynamic (virtual machine) cloud infrastructure, and cloud costs by monitoring server usage and return on investment. The cloud offering for private and public clouds includes Amazon Web Services, Microsoft Azure, and Rackspace. Most recently, coverage of Big Data database Cassandra has been added to the APM coverage. Compuware dynatrace's Data Center Real User Monitoring has flexible packages aimed at small web, medium web, SAP, and non-web usage. SWOT analysis Strengths Compuware's two decisive acquisitions, Gomez and dynatrace, combined with its existing Vantage product form the bedrock of Compuware APM, providing advanced capabilities covering Ovum (Published 07/2012) Page 27

28 most user scenario needs. The solution's PurePath technology is not only ahead of the curve in satisfying the source code level details needed by developers, but can be exploited across the APM usage scenarios for feeding into a variety of operations and network monitoring requirements. Weaknesses The lack of network modeling features is the only main capability gap in an otherwise comprehensive solution. The vendor is still on an integration roadmap, with the end of 2012 as the designated unification completion. Opportunities Compuware continues to innovate and improve its APM offering, and Ovum identifies unified communications, advanced performance analytics, and cyber security as areas where Compuware should enhance its solution. Threats The APM market is fiercely contended but Compuware is well positioned to ensure it remains a key player. Recommendation for enterprise IT customers Ovum's recommendation is to shortlist Compuware APM for its advanced solution covering comprehensively the most important of traditional APM (in its broadest definition) with the new trends in cloud, web, and mobile usage. ExtraHop Networks Profile ExtraHop Networks was founded in 2007 by the lead architects of F5's key platform products in order to put into the market a disruptive solution specifically built for IT operations teams. ExtraHop offers network-based APM appliances that can monitor up to a sustained 10Gbps of network traffic per device, with three key feature areas: proactive early warning, triaging and troubleshooting, and mapping and measuring. Legacy APM tools can be difficult to deploy and maintain. In addition, they may perturb the system through the use of agents, they can be costly when licensed per server, they can skew timing measurements within virtual environments, and they can have limited network performance Ovum (Published 07/2012) Page 28

29 visibility. The company therefore created a solution that is passive on the network and autodiscovers and auto-classifies devices and applications, providing visibility across the network, web, database, and storage tiers. It also offers real-time health and performance metrics down to individual transactions. ExtraHop offers a virtual appliance that can monitor up to 1Gbps of sustained network traffic, making it easy to deploy across geographically distributed environments, such as, for example, monitoring branch offices. In addition, this virtual appliance is ideal for monitoring virtual environments, and analyzing transactions crossing virtual machines on the same physical host. As a network-based solution ExtraHop provides TCP/IP L2 L4 level network performance metrics, as well as full L7 (application layer) content analysis for critical web, database, and storage protocols. The approach allows for database performance tuning by identifying tables with missing indices, web application analysis with tracking of HTTP methods and status codes, memcache performance monitoring through key analysis, and troubleshooting slow logins and load times per user and per session in Citrix VDI deployments. SWOT analysis Strengths The solution is a self-contained appliance that requires no configuration, agents, or probes, and can be operational in 15 minutes into deployment. The appliances support a wide range of protocols and offer APM with network analysis down to L2 L4. It is suitable for continuous realtime monitoring of dynamic environments supporting numerous applications. Weaknesses For the needs of developers the solution offers limited visibility into system internals such as CPU and memory, and therefore needs to be complemented with other tools to provide a complete solution. End-user experience monitoring is also limited, although ExtraHop did recently add opensource Boomerang browser scripts and inspection triggers. Opportunities ExtraHop set out to be disruptive, and its network-based APM provides advanced capabilities in a neat self-contained solution. Ovum sees opportunities in partnering with other APM vendors for pre-production and forensic analysis. Ovum (Published 07/2012) Page 29

30 Threats As a small player with an innovative approach in a competitive market, there is a possibility of acquisition. Recommendation for enterprise IT customers Ovum suggests evaluating the product for APM requirements where other tools either lack sufficient visibility across tiers or cause environment overheads to be too high for continuous realtime monitoring. Recommendation for enterprise IT customers Ovum suggests evaluating the product for APM requirements where other tools either lack sufficient coverage or cause environment overheads to be too high for continual real-time monitoring. HP Profile HP APM is positioned as part of the HP Business Service Management (BSM) solution, which includes Systems Management Center and Network Management Center (NMC). Common to these solution sets is an event-management and topology-based event-correlations tool called HP Operations Manager i, which rests on the BSM platform, and which is fed by the Run-Time Service Model (RTSM), which in turn feeds on the HP CMDB solution HP Universal CMDB. It is not necessary to have either UCMDB or any other HP products in place in order to run HP APM, because a self-contained subset of RTSM is bundled with it, and RTSM will integrate with other CMDB products. The benefit of a connection to a CMDB is that the HP APM solution is then able (through RTSM) to have access to a living topology map of the IT infrastructure, and is able to link business processes and transactions to IT components. RTSM was first introduced in 2010 and is now a key part of HP's IT Performance Suite strategy. RTSM visual maps the connections (in a dashboard) between infrastructure, software, applications and services, and business enablement. HP APM comprises six software products: HP Service Level Management (SLM) measures, tracks, and reports on IT performance to ensure alignment with business objectives and to meet demand. HP Business Process Monitor (BPM) provides synthetic end-user monitoring of application performance and availability. Ovum (Published 07/2012) Page 30

31 HP Real User Monitor provides real user experience monitoring with visibility into user experience, transactions, and behavior. HP Diagnostics provides actionable information about application problems for rapid resolution. HP TransactionVision provides continuous real-time transaction tracking, spanning distributed and mainframe environments. HP Business Process Insight (BPI) monitors business processes and identifies problems and quantifies business impact using key metrics. The first four products are bundled into a package known as HP APM 360. Related products are Mobile Application Monitoring, HP APM on SaaS, Service Intelligence, and System Management. Service Intelligence is available as three products: HP Service Health Analyzer (predictive analytics), HP Service Health Optimizer (dynamic capacity management for virtualized environments), HP Service Health Reporter (consolidates reports on resource metrics, response time data, and business service topology data across dynamic virtualized IT infrastructure). HP Service Health Analyzer is a predictive analytics tool feeding on RTSM data and able to correlate metric abnormalities against topology. The advanced analytics allows forecasting of problems issues and prioritization of issues based on business impact. SHA analyzes historical data to learn what is normal behavior based on advanced statistical research developed at HP Labs. It can detect trends and seasonality patterns, such as day-of-week variations, then automatically set thresholds. HP's runtime anomaly detection system engine is based on multivariant analysis of metrics. The tool enables action to be taken before service is impaired. New issues are matched against previous anomalies using HP's Historical Anomaly DNA technology, and when a match is found the tool offers immediate solutions based on previous resolutions. This technology is also good at detecting noise that can be ignored, reducing the false alarm rate. Finally, HP offers advanced network performance management tools under the NMC brand, of which Automated Network Management is a new-generation solution that integrates HP's best-ofbreed products. SWOT analysis Strengths HP has a long history in the business services management space and APM is a major offering in its portfolio. There are products to fit all the needs of operations professionals managing the largest data centers. HP runs its own research labs from which advanced analytics solutions are Ovum (Published 07/2012) Page 31

32 created, and it owns a number of patents in predictive analytics. In terms of APM coverage, HP scored top in the Ovum APM Rainbow Map. Weaknesses While a large portfolio of products allows HP to fine-tune a solution for a customer, it does place a burden on a new customer trying to make sense of it all, and also puts HP at a disadvantage compared with vendors that offer one box. However, the solutions HP offers are designed for the most demanding IT infrastructure environments. Opportunities APM is in a growth phase at the moment, the result of transitions to new technologies, spanning web, mobile, cloud, and virtualization. HP has the capability and opportunity to address these trends. Threats HP is a stable company, one of the enterprise IT management group of four, and does not face any threats. Recommendation for enterprise IT customers Ovum's recommendation for enterprise customers is to shortlist HP APM as one of the top-end solutions on the market, geared for complex IT infrastructure environments from medium to the largest enterprises. IBM Profile IBM identifies a number of trends driving APM: lines of business need greater business process and application performance availability; the transformation of the data center by cloud and virtualization adoption; and data centers need faster, agile response times with reduced resources and skills availability. Solutions therefore need to be easier to deploy and configure and more intuitive to administer. IBM Tivoli APM solutions are designed to address these challenges and take a five dimensional view of APM: discovery and modeling, end-user experience monitoring, transaction tracing, diagnostics, and analytics. IBM Tivoli APM includes real end-user experience monitoring using an agentless approach, as well as robotic (synthetic) monitoring, for providing early warning signals. IBM Tivoli APM offers Ovum (Published 07/2012) Page 32

33 both agentless and agent-based transaction tracking. Agentless monitors are deployed for realtime monitoring of network traffic, including across private and public cloud segments. Agentbased monitors can be deployed for detailed instance-level transaction tacking, or real time for high-risk, mission-critical domains. IBM SmartCloud Monitoring is a solution that provides infrastructure-as-a-service (IaaS) optimization and cloud monitoring. Cloud capacity optimization helps reduce infrastructure costs and aligns usage with organizations technical and business policies. Analytics capabilities cover workload management, what-if analysis, recommendation and benefits summary, and optimization plans. Integrations across IBM business solutions sees IBM Tivoli integrating its APM and BPM solutions for an end-to-end performance management solution. There is also a program to support DevOps with application diagnostics, as IBM Rational and IBM Tivoli integrate their solutions to provide the seamless development-to-operations continuous delivery that DevOps addresses. The data center metrics explosion has become a real problem of data management that traditional manual analysis techniques cannot cope with. There is a need for automated, self-learning analytics technology which IBM is able to deliver. IBM delivers powerful trending and baselining tools integrated with its APM monitoring solutions to detect emerging capacity and performance issues. Soon to be delivered is an advanced set of behavior-learning analytics tools that not only learn normal behavior and alert operations for abnormal behavior, but also integrate daily and seasonal application behavior into the threshold assessments. This capability stems from integrating IBM Tivoli Monitoring with SPSS Advance Correlation capabilities for projecting higher confidence capacity and performance trends using time-tested correlation methods that can be automatically applied. IBM can therefore offer analytics in the top bracket of capabilities available on the market. As organizations make increasing use of cloud services and facilities the applications that they deploy on them will run across all the tiers within the organization, so it is essential that APM tools are able to cross these boundaries seamlessly. This plays to IBM's strengths because it is able to bring a wide portfolio of solutions to enable this vision. The integration work across the IBM tools is taking place and Ovum believes that IBM will continue to be a force in APM. Ovum (Published 07/2012) Page 33

34 SWOT analysis Strengths IBM is able to focus attention on the latest trends in APM to deliver advanced solutions, with particular progress in cloud APM and BPM-integrated APM. IBM Tivoli is able to deliver a wide portfolio of solutions to bear on the broadest scope of APM. Weaknesses IBM needs to fill gaps in its end-user experience monitoring, but otherwise has one of the most accomplished solutions on the market today. Opportunities IBM maintains leadership through home-grown solutions, resulting from extensive research, and acquiring new innovators that periodically appear, and when doing so accelerates alignment with strategic initiatives around its key business solutions. Threats APM is going through a resurgence, with new entrants appearing and community editions of products pushing the boundaries. While IBM mainly targets larger enterprises that have complex and large-scale problems and multi-tier complexities, there may be market erosion from secondtier and smaller players. Recommendation for enterprise IT customers Ovum's recommendation for large enterprises is to shortlist IBM Tivoli APM, especially where APM across cloud infrastructure is a requirement, and where enterprise IT management integrations are strategic. While single product solutions are often initially perceived as good-enough solutions they can prove unable to provide value as cloud adoption becomes more pervasive. IBM's solution can be easily deployed while seamlessly allowing additional features to be brought online as management needs become more complex. itrinegy Profile itrinegy's AppQoS is described as a network application performance management appliance, with the emphasis on the network. The appliance offers real-time analysis of network traffic and Ovum (Published 07/2012) Page 34

35 application performance and is non-intrusive, using taps, mirrors, and other technology. The solution is designed to be used in continuous real-time mode, and because it is passive it places no burden on the IT infrastructure. It will monitor virtualized as well as physical environments, WANs, cloud-based services, and mobile apps. The appliances are deployed across the IT environment, including remote locations, and will link together to form a complete picture of network traffic and APM. The installation process is designed to be easy, taking minutes for the appliances to configure and connect to each other. AppQoS uses advanced technology to virtualize the network interface, avoiding the need for port aggregation hardware. The solution will analyze all data passing through layers 3 and 4 in the TCP/IP stack, and it will also examine layer 7, the application layer, for some protocols. The realtime data gathered is analyzed to provide application performance monitoring via a rich web interface dashboard. Monitoring of WAN or local sites can be combined on single or multiple consoles, and views can be customized as required. While the default dashboard displays the last 15 minutes of traffic and performance, the menu-driven interface allows a user to create custom dashboards for displaying information particular to their needs. Widgets on the dashboard can be drilled to reveal further depth of information on performance. The interface can also generate reports with full charts for management purposes. Each appliance can process network traffic running at over 10 Gbps and 1.5 million packets per second, and there is no limit to the cluster size of networked AppQoS appliances. In cases where the data gathered is especially large there is the Data Offload appliance that can be linked in and which can handle high-volume reporting tasks. The application performance metrics are correlated by the solution across the whole cluster of appliances monitoring the environment. The only traffic generated by the appliances are alerts and requests for data generated by the dashboard. For security and data-management purposes, linked appliances can be restricted to operate in one direction. The current offering is in its second generation and has won awards from TechWorld and Network Computing. SWOT analysis Strengths An appliance solution that is easy to deploy and configure, providing a scalable clustered solution that has minimal effect on the environment that is being monitored and therefore suitable for continuous real-time use. Flexible and cost-effective device options are available from VM guest, Ovum (Published 07/2012) Page 35

36 portable 20Mbps, to multi-port 10GB versions. It offers full-strength APM with deep-level packet inspection across protocols such as HTTP, FTP, CIFS, and VoIP. Weaknesses itrinegy is not in the business of competing with the Big Four APM vendors with their wide portfolio of solutions. It has a number of gaps in its range of offerings, including no (internal) database monitoring, no business transaction monitoring, and no predictive analytics. This solution either meets the particular needs of a customer, or will need to be complemented with additional solutions. Opportunities itrinegy is a specific offering in the broad APM spectrum and has the opportunity to grow out its solution in the areas identified by Ovum. As a single solution that is easy to install and at a relatively low cost compared to competitors in the field, the vendor has good prospects. Threats The competition in APM is fierce, with mature organizations battling new startups, so itrinegy faces multiple threats. In the context of network APM quality of service, it has sufficient momentum to hold its own. Recommendation for enterprise IT customers Ovum's recommendation for enterprise customers is to consider itrinegy AppQoS against their specific needs, with a view to solving APM pains at a cost-effective level. While this solution does not offer a comprehensive set of APM features, it has strengths that can be matched with individual network APM requirements. Nastel Technologies Profile Nastel's APM product is AutoPilot M6, an application and transaction performance monitoring solution. Suitable for distributed and mainframe platforms, the solution covers transaction management, business activity monitoring, middleware management, and APM, with a complex event processing (CEP) engine at the core driving the monitoring and correlating activity. Key to Nastel's approach is making sense of messages and business transactions that span multiple Ovum (Published 07/2012) Page 36

37 tiers, while being able to track individual messages, requests, and other detailed activity at high granular levels. This approach allows a business service quality of service (QoS) to be provided. One of the major challenges facing the use of APM is the data explosion that is reaching Big Data proportions. The problem has grown in the past decade with the increasing complexity of the IT environment and its multiple tiers including physical, virtual, cloud, web, mobile, application components, databases, and storage, coupled with improved use of metrics gathering across the infrastructure and end-user devices. In order to make sense of this data in as near to real time as possible CEP is proving to be an ideal technology because it can identify the root cause of a complex sequence of events. Nastel's CEP engine, which scales to handles millions of events per second is home-grown and embedded in the solution with no tuning or expertise required to configure it by an administrator. Nastel s solution is particularly strong in covering middleware technologies. It will discover transactions running across application components such as Java or.net objects, WebSphere MQ, Tibco, DataPower, Solace, and CICS, and model the transaction flow. The CEP engine correlates the events detected with data flowing in from Nastel and third-party tools to provide instant root-cause analysis, including difficult to detect gradually occurring trends. The administration of the tool allows for the segregation of duties, ensuring compliance with regulatory concerns, while also monitoring how compliant the activity is in the data center. SWOT analysis Strengths CEP-based APM analytics drives the core root-cause analysis and issue predictions. Nastel is one of a new generation of vendors that goes beyond generating and collecting data and into understanding events and provide guidance to the user with actionable information. In addition, areas of strength include BTM, topology discovery and change-impact analysis, middleware support, reporting, and ease of use in administration. Weaknesses The areas of weakness such as end-user experience monitoring, network modeling, unified communications, and cyber security coverage are best met by partnering with other solutions. Ovum (Published 07/2012) Page 37

38 Opportunities Nastel has core strengths and also gaps in its solution and therefore needs to establish strategic alliances with other vendors in the APM space with which it can work in order that customers have full APM coverage. Threats As one of the smaller vendors in the APM space the larger players pose a threat with their greater marketing reach. Nastel's best strategy is to partner and foster an ecosystem in which its CEP approach lends strength. Recommendation for enterprise IT customers Ovum believes that enterprise customers should evaluate Nastel when their existing solutions are not able to cope with the deluge of metrics data. Nastel can help enterprises become more proactive in their response to problems by using AutoPilot as a single point of APM analysis for their existing tooling. Going further they could use it to de-duplicate existing overlapping middleware monitoring tools in order to reduce costs and complexity. NEC Profile NEC MasterScope Invariant Analyzer is a new-generation statistical and algorithmic analysisbased solution for system performance management. APM solutions generate hundreds of thousands of data points across an IT infrastructure, spanning multiple applications, services, and hardware, physical and virtual. Making sense of this data deluge is a serious challenge. Performance degradation is a problem for businesses, but many issues may remain undetected and invisible under conventional APM analysis. While end users might complain of poor service, IT operations have the task of making sense of multiple metrics and logs. Once the problem is identified the next challenge is to locate the root cause and fix it. NEC's approach is to identify invariant relationships between the performance metrics. During initialization relationships are detected based on analyzing a historical backlog of data, and when any relationships are found to hold over time they are grouped in a model as invariants. Having identified these special relationships, the next step is to monitor whether they continue to hold in time. Any deviations are immediately detected as indicators of potential problems. An anomaly is identified when many deviations are detected for a given performance metric to enable robust detection and cause location of potential problems. This approach goes beyond thresholding of Ovum (Published 07/2012) Page 38

39 individual metrics because how metrics relate to each other conveys richer information than absolute metric values, even if the thresholding is dynamic. Using MasterScope Invariant Analyzer makes it unnecessary to set thresholds because the invariant relationships become the key indicators. A system such as a data center and the applications running across it can be rapidly analyzed without needing expert knowledge configuring various thresholds, and MasterScope Invariant Analyzer automates the whole process. The invariant indicators that are found to deviate from the model can help identify root causes of problems. The benefit of this approach is in the reduction of silent failures, issues that are too low-level to be picked up by conventional tools until they start to escalate, leading to early detection of issues and rapid analysis and location of problems, followed by recommended solutions before system and user impact occur. The solution dashboard has a ranked list of all the indicators used and provides a visual anomaly map of the indicator relationships, flagging those that are broken. Detailed views of the indicators over time are displayed in a separate graph, and problem events are marked. Special pie charts are used to visually determine the failure's root cause. The outer part of the pie chart indicates which part of the system is most likely to be failing, and the inner part shows which specific component is most likely to be failing in the identified most likely part of the system. Multiple pie charts are produced covering web and databases, for example, and a master pie chart is then created by balancing the differences across the charts. The master pie chart is then able to indicate the most likely cause of failure. Invariant Analyzer is designed to integrate with existing APM tools and workflows. It has closed loop integration with APM solutions such as CA APM (formerly Wily Introscope) that provides automated analysis of data collected by APM tools and alert escalation to the APM tool consoles to enable familiar user experience and workflows. SWOT analysis Strengths Developed by the laboratories of NEC, the patented technology behind MasterScope is the result of deep research in statistics-based analysis. The solution is complementary to existing tools and can enhance legacy systems by adding greater predictive capabilities and providing increased guidance to root-cause analysis, as well as helping cope with the Big Data scale of metrics generation in large-scale data centers. According to NEC, the solution's invariant analysis approach is unique in APM. Ovum (Published 07/2012) Page 39

40 Weaknesses Invariant Analyzer is positioned as an add-on analytics engine to existing performance-monitoring tools so it is not a standalone APM solution by itself. As such, customers will usally purchase Invariant Analyzer as part of an APM solution from NEC partners. Opportunities Invariant Analyzer can also be applied to IT domains other than APM, including infrastructure management and network management. NEC could sell the product directly, however it is not generally geared for such sales and marketing operations in all geographies. Threats The vendor is stable and faces no particular threats. The core activity of NEC is computer manufacturing and communications. Recommendation for enterprise IT customers Ovum's recommendation is to evaluate the APM vendors, such as CA Technologies, that have a partnership with NEC for MasterScope Invariant Analyzer. NetScout Systems Profile The NetScout ngenius Service Assurance Solution provides a holistic view into the user experience of applications and services by integrating the monitoring of application elements, network traffic, and how their inter-relationships impact one another. NetScout's unified service delivery management (USDM) approach spans APM, network performance management, unified communications management, and cybersecurity management. USDM therefore provides APM in the broadest sense discussed in this report, covering end-to-end performance; services, applications and protocols; physical and virtual infrastructure; voice and video traffic; and detecting cyber threats and forensically investigating cyber attacks. NetScout ngenius Service Assurance solution provides a wide range of key service delivery management capabilities that spans service visualization, early warning performance management and analysis, service and policy validation, service and network optimization, forensic analysis, and reporting and trending. The foundation capabilities of the ngenius solution are: visibility using passive, non-intrusive data capture and real-time analysis of IP traffic; correlation and analysis Ovum (Published 07/2012) Page 40

41 based on an adaptive data model-driven correlation of service flows; and visualization and reporting revealing interrelationships and dependencies and providing actionable intelligence. At the root of these capabilities is NetScout s Adaptive Session Intelligence (ASI) technology, a realtime deep-packet analysis and data mining engine that dynamically tracks, captures, and analyzes complex service delivery and application transactions across multi-domain IP networks for comprehensive real-time analysis of the performance of applications and services across physical and virtual environments. NetScout solutions can instrument a private cloud to track and extract performance metrics for applications executing in the cloud. In order to monitor applications in the public cloud there are two options. If the cloud provider has instrumented its services and makes the information available then this data can be leveraged. Otherwise, ngenius Virtual Agent can be taken into the cloud with the application and deployed on the same virtual server. All the performance meta-data, analytics, and KPIs gathered will then be delivered back to the customer's ngenius infrastructure deployed on-premise. For end-to-end response times NetScout is finding that customers, originally financial traders but increasingly enterprise customers, are seeking granular hop-by-hop latency and performance metrics. This can be achieved by deploying ngenius InfiniStream Appliance for deep-packet capture in each venue or transformation point and ngenius Enterprise Intelligence can provide the hop-by-hop user session transaction analysis with per-hop performance metrics that include granular latency measurements. NetScout's solution is an integrated system providing contextual linkage between each analysis module. These modules share the same common data and metrics and enable a customer to buy what they need with the flexibility to add incremental capabilities in future. The advantage of NetScout's packet-flow approach is that all the data is captured, and a given analytics module then uses the appropriate data it needs. SWOT analysis Strengths NetScout ngenius takes a totally passive, packet-flow approach with the benefit of no agents to deploy or manage (or perturb) the environment, providing a perspective of service delivery that traditional APM tools cannot provide. Ovum scored NetScout well in the application and infrastructure performance management segments, unified communications, all aspects of analytics, middleware support, and cyber security. The unified aspect of the solution coupled with advanced analytical and forensic capabilities is a key strength. Ovum (Published 07/2012) Page 41

42 Weaknesses There are gaps in the solution in end-user deep-dive application-level monitoring, and automated network modeling. There is no application performance testing, and no application dependency mapping. However, NetScout does provide a representation of servers and networks, but it is not automatically created. Elements are discovered, but the system user needs to construct the path, though once constructed, it provides a view of the delivery environment. Opportunities NetScout has clear strengths on the network side of APM, with core functionality for L2-L7, though the most recent tools further extend NetScout s L7 capabilities. Partnering to fill the missing gaps such as end-user monitoring should be an important strategy for NetScout. The vendor is also on a roadmap of enhancing its solution. Threats NetScout has a solid base in the network aspects of APM but faces stiff competition from vendors in other areas of APM. Its strategy is to enhance USDM, organically or through acquisition. Recommendation for enterprise IT customers Ovum's recommendation for prospective customers is to consider NetScout solutions as part of a broad APM deployment (with the benefits of passive and agentless monitoring), especially where you need to focus on the management of a holistic view into service delivery and gain deep-dive visibility into intra-server application transactions. Netuitive Profile Netuitive is a relatively new APM analytics vendor, incorporated in 2002, but has grown rapidly with over 100% year-on-year sales growth. The company started by OEMing its technology, then moved to an enterprise focus with the release of its first enterprise product in 2007, with its greatest success in the financial services sector. Netuitive s Predictive Analytics is an advanced analytics layer that sits on top of data-gathering solutions for APM, infrastructure, and BAM, and performs self-learning analytics. Large enterprises with complex applications have a Big Data problem with APM. They may have many tens of APM tools in place and the metrics data generated can become overwhelming. The growth in metrics Ovum (Published 07/2012) Page 42

43 collection has reached the point where manual, human-based analysis is no longer possible to analyze all the data, and much of it remains unused. Consequently there is a risk of outages. The solution is automation, and Netuitive's approach to automated analytics has three key advantages: it is self-learning, reducing the labor of configuration; it is holistic in combining data from multiple silos and making sense across physical, virtual, and cloud tiers; and it is predictive, being able to process massive data loads in realtime and able to detect anomalies proactively before they can turn into cascading failures. At the core of Netuitive is the patented behavior-learning engine, which feeds on all the metrics data gathered across the data center. The automated self-learning based on proven statistical multivariate regression analysis is key to Netuitive's business proposition. The solution learns what is normal behavior and can therefore detect unusual behavior, without an administrator having to set thresholds and configure the system. The presentation layer of the solution provides composite health scores, diagnostics, Netuitive's branded Trusted Alarms, and reports. Trusted Alarms is both a real-time and forecast alert feature made possible by Netuitive's automated contextual analysis, and can be integrated into existing monitoring consoles and defectticketing systems. The alerts are generated using an accumulated score of real-time, contextual, and forecasted deviations, taking into account the number, frequency, and severity of each deviation, and feeding in results from multiple real-time and forecasted calculations. A Trusted Alarm can be generated for an individual system issue or an end-to-end business service. According to Netuitive the feature reduces false alerts by between 90% and 99%. The advantage of having an analytics engine running on top of a heterogeneous APM tool environment is that it becomes possible to perform deep and rapid correlations across all the APM tools in the IT environment and provide the results in a unified dashboard. This enables end-to-end problem isolation across silos, and proactive notification of impending performance issues. SWOT analysis Strengths Nine patents in statistical multivariate regression analysis, taken to new levels of scalability based on techniques taken from neural network research, protect the approach that Netuitive has perfected. Superior rapid analytics processing capabilities are linked to a performance management database. An open API allows easy integration between the predictive analytics platform and third-party tools including APM, monitoring tools, event-management consoles, and ticketing. Ovum (Published 07/2012) Page 43

44 Weaknesses It requires APM data-gathering tools to be in place, so this is a complementary solution, and because APM tools generally already have some in-built analytics features, Netuitive is worth considering when the data analysis becomes unmanageable without advanced automation and analytics. Opportunities Netuitive has been successful in the OEM space and is now gaining traction and direct market share in the enterprise APM space. Threats The company's technology may become increasingly attractive to a larger APM vendor given the way in which APM data processing is turning into a Big Data problem. Recommendation for enterprise IT customers Ovum's recommends that medium-to-large enterprise customers with data-analysis problems should shortlist Netuitive for their APM analytics. Netuitive will be a good fit for enterprises interested in correlating large volumes of real-time performance and application data from a multitude of data sources, enabling proactive management of critical applications. Opsview Profile As the vendor name implies, Opsview offers a view into IT operations, with an operations-centric approach to monitoring data centers that makes it ideal for system administrators, providing a onestop shop for operations monitoring across networks, servers, and applications. Opsview is an open-source product available in three editions: Core, Pro, and Enterprise. Core (formerly Community edition) is available for free download and provides core-monitoring capability. The subscription-only Enterprise edition is certified and ready for mission-critical systems, with full 24x7 support. It extends Core s features with enterprise modules such as an advanced dashboard and reports, Opsview Syncmaster, which allows seamless migration of configuration settings between staging environments, SMS messaging, Netaudit for network infrastructure configuration and security monitoring, and Service Desk Connector for an integrated IT service. Ovum (Published 07/2012) Page 44

45 The Core features include distributed point-and-click system-monitoring, triggers, and multichannel alerts, SLA tracking, performance charts, web-based configuration, cloud services support, a native Android mobile app, and many web-enabled views, a dynamic dashboard, and drill-down commands. Opsview has begun to offer dashboard on smart mobile devices (Android), with ios devices to follow. Opsview Pro is a new offering aimed at the SME market that will be launched at about the time of this report's publication. The original Opsview project was started in 2003 and went open source in 2009 with the launch of the Community and Enterprise editions. The architecture is based on the open source Nagios core framework and is cross-platform, supporting Windows, Linux/Unix, SNMP, LAMP, and agent and agent-less monitoring. The free Core product is a first rollout bed for new features that are firmed up in the Enterprise edition. The solution uses open RESTful/web services based APIs, making customization easy. The Enterprise edition subscription is available in Bronze, Silver, Gold, and Platinum offerings differentiated by factors such as the number of incidents supported and the number of technical contacts. SWOT analysis Strengths Opsview has a clear target market focused on operations end users and service providers managing their end users, with tools providing the information this community requires. The solution covers on-premise, cloud, and hybrid system monitoring, and has an attractive opensource model with competitive pricing. Weaknesses It does not offer code-level troubleshooting of interest to developers, and the modeling capabilities are limited to trend extrapolation. Some of the aspects of virtual environment support is still in the pipeline, and WAN, storage, and security features are limited. There is no unified communications performance management, such as video and VoIP. Opportunities In a mature market Opsview with its open-source model is able to offer a competitive solution at a lower cost than proprietary solutions. DevOps has placed greater emphasis on operations for driving collaboration and improving software delivery, and Opsview with its operations-centric APM solution is well aligned with these trends. Ovum (Published 07/2012) Page 45

46 Threats Competition in the market continues with new entrants targeting the DevOps space, and there is therefore some overlap between release-management solutions and some of Opsview's sweet spots. APM cloud SaaS solutions can become the next wave in cost reduction to erode the onpremise solutions market. Recommendation for enterprise IT customers Ovum believes that any organization can benefit from trialing Opsview Core in its data center, although of course the free license is only one aspect of the total cost of ownership, with training and support the other key factors. The Enterprise edition would be the next step if the Core product is found to be suitable. Enterprise adds extra guarantees on the feature set, making it suitable for mission-critical environments, as well as offering extensions suitable for larger environments. Despite the gaps, the offering is well-targeted at its core audience. New features are driven by customer demand, and in 5% of cases driven by a customer-vendor collaboration that is then rolled out into the main product lines. OpTier Profile OpTier first entered the APM market as a business transaction management (BTM) vendor, coining the term, or at least bringing it to prominence. The need to link different services, applications, and components that are part of a single business transaction is what OpTier realized was important for the business. OpTier has since realigned its position as an APM vendor and has grown its solution to cover other aspects of the APM stack, in particular, analytics. However, business-centric concerns remain a core coverage area that includes health of business processes, identifying process bottlenecks, SLAs, and business impact. OpTier also recognizes the Big Data problems with the sheer volume of silo data generated by traditional monitoring tools. Its CEP engine and patented business transaction technology can automatically capture and process billions of cross-silo transaction data per day to bring immediate business context and value, reported to the various stakeholders including business users, IT operations, DevOps, application owners, and CIOs. OpTier refers to this as Big Transaction Data. The current offering is differentiated by the following aspects. It is business transaction-driven, provides a living topology of devices and applications (the continually updated dependency Ovum (Published 07/2012) Page 46

47 mapping is a key feature), it is fully automated and housed in a single product, deployment can be agentless or with agents, its is cloud-ready, it offers capacity management, and it links into CMDB, QA, and testing, and business services and lifecycle management systems. The company focus is enabling an always-on business through its APM solution. As part of the marketing realignment, OpTier has named its product Always-on APM, which brings together into a single solution a number of suite components that were separate products, including end-user experience-monitoring, application diagnostics, but not Business Events, which is now branded Always-on APM Advanced Analytics and available as an optional module. The underlying technology is called the BTM platform. OpTier Always-on APM analytics provides OLAP, alerts, log analysis, business process flow, and optimization reports, and the Advanced Analytics module offers complex event processing. The solution covers all tiers in the data center from middleware and mainframe through to cloud. It includes an SDK and customizable dashboards. The end-user monitoring uses JavaScript injection to track browser-based user activity, as well as client, delivery, and network times, and includes users on mobile devices. This monitoring can be used for example track how third-party content impacts end users. Business processes are monitored on a dashboard where users can select from any number of processes listed. The process steps are identified in a process diagram and traffic-light-colored depending on health. Steps can be drilled into revealing detailed performance metrics. Problems are diagnosed from a business perspective, identifying business impact. SWOT analysis Strengths There are two distinguishing aspects of OpTier Always-on APM. The solution is a single product that covers a wide range of APM areas, including OLAP analysis, with the business-level process monitoring reducing the time to understand the business impact of an issue. For more complex environments, the CEP engine can be added. Weaknesses The only serious gaps in the solution are unified communications and cyber security. There is also minimal support for application performance testing. This solution is less geared for the QA and testing part of the development lifecycle and more toward the operations and business services management end of APM. Ovum (Published 07/2012) Page 47

48 Opportunities OpTier created a wave when it entered the market with BTM. Since then it has realigned itself to an APM positioning with BTM as a core differentiator. The single-box solution should provide a strong marketing position, and the vendor is positioning itself to enhance legacy solutions with its BTM approach. In addition, the rich transactional data generated by the solution can be leveraged as a data source for further analytics reporting using either OpTier or third-party products. Threats As a relatively new player with a pioneering approach, the threat of acquisition will occur during periods of market rationalization. Rivals are also reconfiguring their tools to offer BTM, which while recognizing OpTier's innovation, will dilute its market impact. Recommendation for enterprise IT customers Ovum's recommendation is to shortlist OpTier where a single APM solution is required to complement existing monitoring tools for cross-silo transaction visibility or replace some of the more traditional APM tools. Because the solution is able to dig down to source-code level it may also appeal to developers and will fit well in an organization adopting DevOps. Quest Profile For many years Quest was the first port of call for database administrators that needed deep database-monitoring solutions. The company has over the last decade diversified into APM for the data center, and its flagship product is Foglight. Foglight is a single solution built in a modular way that allows users to select the APM segments they need. The solution covers: User experience: synthetic user record and playback, and real-user performance monitoring and session capture and replay. Application server monitoring: component granularity, transaction tracing, middleware (message queue) monitoring, and web server monitoring. Network performance monitoring: in-depth traffic analysis, advanced alerts and autoremediation, and VoIP monitoring. Service level monitoring: service definitions, service and operations-level agreement policies, dependency mapping, component topology discovery, role-based dashboards, and third-party integrations. Ovum (Published 07/2012) Page 48

49 Physical and virtual server monitoring: operating systems, virtual servers, capacity management, and virtual machine storage monitoring. Database server monitoring: database performance monitoring, real-time database diagnostics, database workload analysis, and database tuning. Applications today cross multiple tiers across the IT infrastructure, and Foglight is able to track an application across these domains. At same time Foglight is able to take a single user's view and monitor multiple applications that the user may use during a session. Quest recognizes a broader definition of APM that encompasses networks, virtual and physical infrastructure, and databases. The latest core Foglight release has a number of new capabilities. Remote monitoring of OS and Database performance without requiring local agent install on the monitored server. Google-like search capability to find any object or health topic. Data model extensions to include business data such as cost-center attributes, business unit associations, or configuration and application management metadata. Reporting with external accessibility for reporting data and analysis, and create, run, or manage reports from any dashboard. End-user data visualization in the service operations console, and dependency mapping with automatic recognition of services. New Java and database monitoring features, and deeper network monitoring including dependency mapping periodically updated in Foglight. In addition, network and VoIP monitored traffic are converged. Quest introduced a Conversion Funnel process diagram to provide visibility into user transactions, where as each step in a process is tracked, any issues are immediately flagged. Problems can be drilled into and a triage operation initiated. There is support for Java Enterprise Edition with a dashboard highlighting Java EE server health including JVM availability, heap usage, and Java Transaction API rollback. Similar capabilities are available for.net applications. Looking forward Quest is investing in cross-domain capabilities for triaging and fault isolation, enduser experience management for web 2.0, and business impact-based alerting. Ovum (Published 07/2012) Page 49

50 SWOT analysis Strengths Quest has a single product with integrated modules that a customer can choose according to requirements, with the option to grow capability. There is excellent overall coverage of APM features and the solution is a good fit across development, QA, and operations. Foglight is also available as a SaaS offering. Weaknesses The topology and change impact analysis is a new feature and needs further enhancement. Similarly, unified communications is a recent addition and has room for further enhancement. Opportunities Quest is growing Foglight year-on-year so Ovum expects to see continued feature enhancements. The company has a solid reputation among the technical operations community and has the opportunity to expand into the more business-oriented areas of APM. Threats Quest is stable and growing in its space so Ovum does not perceive any market threats, but consolidation action by a larger rival cannot be ruled out. Recommendation for enterprise IT customers Ovum's recommendation for enterprise customers is to shortlist Quest for its deep-dive capabilities across the breadth of the wider APM space. The modular nature of the solution allows customers to incrementally try out and add capabilities as needed. Ovum (Published 07/2012) Page 50

51 APPENDIX APM Features Matrix and Ovum's Weightings The following sections describe the features matrix that vendor solutions were scored against, with the weightings that Ovum used. The section results were aggregated and rainbow color scheme mapped to produce Figure 3, the Ovum APM 2012 Rainbow Map. Application performance optimization Application performance monitoring - 2 Application availability - 2 Application server-side performance and management: application server management - 2 Track individual transaction data packet - 2 Capture all transactions 24x7, end-to-end from browser through web and application tiers to database and back - 4 Provide deep visibility into transactions down to code level - 4 Application transaction management: multi-tier single application view- 2 Interrelationships between application, network, and enablers - 1 Application traffic management: data packet level application routing optimization - 1 Measure real end-user traffic through content delivery networks - 1 Record all environment settings, transactions and application data prior and during a fault for replay by developers - 1 Detection of jitter - 1 Application platform coverage Covers Java platform - 1 Covers.NET platform - 1 Covers applications in virtualized environment - 2 Covers applications in cloud environment - 2 Covers mobile applications - 2 Ovum (Published 07/2012) Page 51

52 Topology and change impact analysis Auto-discover transaction flow topology for all transactions - 1 Automatically map and visualize the flow of transactions in distributed environments - 1 Automatically detect bottlenecks (specific components introducing latency or causing errors) - 1 Automatically detect flow topology changes (for example, different routes, components added or removed) - 1 Compare environment at two time periods (for example, compare new and current application deployment) - 1 Application dependency mapping - 1 End user experience monitoring Monitor end user experience, from the user's perspective inside the browser - 3 Monitoring across all devices, all browsers, capture click paths and Web 2.0 page actions - 3 Track visitor satisfaction (for example, satisfied/tolerating/frustrated) - 1 Compare end-user experience and performance against industry leaders and competitors using dynamic baselines - 1 Automatically detect and monitor the performance of landing pages - 1 Trace logic in Web 2.0/Ajax-based end-user clicks into the server side - 2 Application performance testing Simulated user and transaction (synthetic) load and performance testing for performance analysis - 1 Website testing from diverse locations - 1 Web services testing - 1 Stress testing - 1 Real user performance monitoring for web-based applications - 1 Ovum (Published 07/2012) Page 52

53 Network performance management: Infrastructure performance optimization Network performance monitoring - 2 Network availability - 2 Network traffic management including load management - 2 Identify and pinpoint network bottlenecks between application components - 2 Capacity management/planning - 1 Server management - 1 Router and switch performance management - 1 Storage performance management - 1 WAN performance management - 1 Cloud performance management - 1 Virtualization network performance management (within hypervisor) - 1 Remote datacenter monitoring - 1 Network modeling Network scenario and capacity modeling and simulation - 1 Modeling includes cloud environment - 1 Modeling includes mobile network - 1 End-user response time modeling - 1 Business transaction management and SLAs Trace business transactions (look at services holistically, all components that make up a service) - 1 Business impact management - 1 Service modeling - 1 Alert on business anomalies (for example, incomplete or malformed business processes) - 1 Service delivery management - 1 Ovum (Published 07/2012) Page 53

54 SLA definition, automatic performance baseline - 1 SLA definition, manually define service-level objectives for key transactions - 1 SLA breach alerting with customizable tolerance rules - 1 Service-level performance visibility - 1 Unified communications (UC) performance management Unified communications applications (voice, video, video conferencing, desktop video conferencing, telepresence) - 1 VoIP performance monitoring - 1 VoIP call playback for historical analysis - 1 Video and video conferencing (at application and network level) - 1 Telepresence (support multi-screen and multi-audio) - 1 Measure all UC services side-by-side - 1 Measure UC services along with the network, other applications and services - 1 User experience management for UC services - 1 Diagnosis and root cause analysis Event correlation: pattern matching and understanding of multiple events for rootcause analysis - 1 Event correlation: identifying event(s) at root of sequence of events - 1 Source code level identification troubleshooting - 1 Knowledge base and collaboration: annotating logs and sharing knowledge - 1 Application-level fault management - 1 Network level fault management - 1 Data packet-level analysis - 1 Auto-detect application errors in transactions - 1 Automated triage and repair - 1 Automated memory diagnostics - 1 Ovum (Published 07/2012) Page 54

55 Application Web 2.0 type UI and distributed objects/components deep-dive analysis - 1 Web application transaction analysis (JavaEE,.NET ) - 1 Historical deep-level analysis - 1 Predictive IT performance analytics Self-learning modeling or behavior learning engines - 2 Predictive analytics and performance benchmarking - 2 Cross-domain (infrastructure, networks, servers, applications, IT business services) analysis - 2 Automated problem pattern detection (for example, too many SQL executions, high CPU executions, hidden exceptions) - 2 KPIs and aggregated metrics for multiple dimensions (for example, OLAP): transactions, users, locations, applications, business processes - 1 Complex event processing (CEP) for advanced analytics. Correlates infrastructure events and business events for smarter rules alerts - 1 Log parsing - 1 Analyze load balancing and show flow anomalies (for example, load is not balanced evenly between cluster nodes) - 1 Highlight excessive chattiness between components - 1 Database and middleware support Database monitoring - 4 Database performance optimization - 4 HTTP/S - 1 SOAP - 1 CICS - 1 Tibco - 1 Websphere MQ - 1 Oracle Service Bus - 1 Ovum (Published 07/2012) Page 55

56 Websphere Message Broker - 1 RMI and RMI-IIoP - 1 T3/S - 1 LDAP - 1 Cyber security Detect potential of cyber security threats - 2 Policy-based security alerting - 1 Security breach forensic analysis - 1 Network-based anomaly detection - 1 Security information and event management (SIEM) integration/data sharing - 1 Deep packet capture and analysis - 1 User session replay capabilities - 1 Admin, reporting, and alerts Zero-configuration for ease of deployment and use - 2 Auto-discover applications, transactions, and users and auto-adapt to application changes - 2 Real-time reporting -1 Historical trending -1 Integrated reporting and dashboard -1 Mobile device dashboard - 1 Compare transaction performance in two time periods or two different environments - 1 Real-time service dashboard - 1 Real-time reporting - 1 Historical and trending reporting - 1 User-defined reports - 1 KPI-based alarming - 1 Ovum (Published 07/2012) Page 56

57 Policy-based alarming - 1 Methodology Ovum analysts use data collected through primary and secondary research to determine the leading vendors in the APM Solution Guide across all categories. Ovum believes the vendors selected represent the main options open to enterprises. The capability analysis required all participating vendors to review an assessment of features available in the solution. Product briefings with vendors were also conducted. Author Michael Azoff, Principal Analyst [email protected] Disclaimer All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of the publisher, Ovum (an Informa business). The facts of this report are believed to be correct at the time of publication but cannot be guaranteed. Please note that the findings, conclusions, and recommendations that Ovum delivers will be based on information gathered in good faith from both primary and secondary sources, whose accuracy we are not always in a position to guarantee. As such Ovum can accept no liability whatever for actions taken based on any information that may subsequently prove to be incorrect. Ovum (Published 07/2012) Page 57