Extending APM. Managing Application Performance Inside and Outside the Corporate Firewall. December 2010 Jeffrey Hill. ~ Underwritten, in Part, by ~

Transcription

1 Extending APM Managing Application Performance Inside and Outside the December 2010 Jeffrey Hill ~ Underwritten, in Part, by ~

2 Page 2 Executive Summary The traditional data center-enclosed application infrastructure now extends beyond the corporate firewall to applications running in the cloud and increasingly to a mobile base of end users. In response, companies are placing emphasis on managing application performance across the evolving application infrastructure. A study by Aberdeen Group in November 2010 surveyed 94 organizations, and results provided solid evidence that companies that manage application performance have fewer complaints from end users and faster time to repair application issues. They also can build applications that are easier to use and require fewer IT resources to support. Research Benchmark Aberdeen s Research Benchmarks provide an in-depth and comprehensive look into process, procedure, methodologies, and technologies with best practice identification and actionable recommendations Best-in-Class Performance Aberdeen used the following three key performance criteria to distinguish Best-in-Class companies: 1.6 hours mean time to repair critical application issues 99.6% average availability of critical business applications 17% reduction in the number of calls to the help desk Best-in-Class companies also experienced gains in two key IT metrics: customer satisfaction improved by 61% and employee productivity improved by 67% Competitive Maturity Assessment Survey results show that the firms enjoying Best-in-Class performance shared several common characteristics, including: 67% are able to associate user transactions with business processes 44% have the ability to monitor application performance across multiple browsers or platforms 33% have integrated application performance management with enduser experience monitoring Required Actions In addition to the specific recommendations in Chapter Three of this report, to achieve Best-in-Class performance, companies must: Integrate the monitoring of application performance and the enduser experience Implement end-to-end monitoring and management of application performance Use end-user experience to inform application design or changes This document is the result of primary research performed by Aberdeen Group. Aberdeen Group's methodologies provide for objective fact-based research and represent the best analysis available at the time of publication. Unless otherwise noted, the entire contents of this publication are copyrighted by Aberdeen Group, Inc. and may not be reproduced, distributed, archived, or transmitted in any form or by any means without prior written consent by Aberdeen Group, Inc.

3 Page 3 Table of Contents Executive Summary...2 Best-in-Class Performance...2 Competitive Maturity Assessment...2 Required Actions...2 Chapter One: Benchmarking the Best-in-Class...4 Business Context...4 The Maturity Class Framework...5 The Best-in-Class PACE Model...7 Best-in-Class Strategies...8 Chapter Two: Benchmarking Requirements for Success...10 Competitive Assessment...11 Capabilities and Enablers...12 Chapter Three: Required Actions...17 Laggard and Industry Average Steps to Success...17 Best-in-Class Steps to Success...19 Appendix A: Research Methodology...21 Appendix B: Related Aberdeen Research...23 Featured Underwriters...24 Figures Figure 1: Pressures to Improve Application Performance...4 Figure 2: Best-in-Class Improve Application Performance Metrics...7 Figure 3: Current Best-in-Class Strategies...8 Figure 4: Tools for Monitoring Application Performance...14 Figure 5: Adoption of APM Tools...16 Figure 6: Monitoring and Managing External Applications...17 Figure 7: Increased Monitoring Improves Best-in-Class Performance...19 Tables Table 1: Top Performers Earn Best-in-Class Status...6 Table 2: The Best-in-Class PACE Framework...7 Table 3: The Competitive Framework...11 Table 4: The PACE Framework Key...22 Table 5: The Competitive Framework Key...22 Table 6: The Relationship Between PACE and the Competitive Framework...22

4 Page 4 Chapter One: Benchmarking the Best-in-Class Business Context The traditional idea of an application infrastructure neatly enclosed in a data center is evolving in response to external forces that didn't even exist 15 years ago. Virtualization of key data center components such as servers and storage made it possible for applications to run independently of the underlying hardware, while the increased speed of inter-networking allows applications to provide acceptable performance for users outside of the corporate firewall. Cloud computing may be a logical extension of a virtual infrastructure, but cloud also places new demands on IT to monitor and manage application performance. Similarly, an increasingly mobile workforce equipped with smartphones, netbooks, or tablets changes the traditional models of application deployment, performance, and the end-user experience. As a result of these changes, traditional transaction-oriented measurement methods are evolving into an end-to-end view of application performance that takes into consideration end user experience as well as the performance of critical business processes. Fast Facts 80% of companies use response times as the primary indicator of application performance 89% of companies consider to be their most critical business application Pressures to Improve Application Performance The need to reduce the number of complaints from end users about slow application response times is the predominant pressure (65%) compelling companies to improve application performance (Figure 1). A related pressure, complaints from users about application usability (24%), is another indicator that application performance needs to be addressed. Taken together, these two pressures should be a call to action for IT to take a close look at both applications and the underlying infrastructure. Figure 1: Pressures to Improve Application Performance End users complain about response times 65% End users complain about application usability 24% Customer satisfaction low er than desired 22% Infrastructure doesn't support planned grow th 22% IT costs are higher than they should be 19% 0% 20% 40% 60% 80% Slow application performance has a direct impact on user productivity and may cause users to become frustrated. Applications that are perceived by users as difficult to use place an additional, and often unnecessary, burden

5 Page 5 on IT resources such as end-user training or help desk resources. Both pressures highlight the need for IT to build and deploy applications that make it easier for end users to get their work done. Customer satisfaction remains an important measure of IT performance. Lower than desired customer satisfaction is a signal for the need to examine and possibly alter current IT practices. Twenty-two percent (22%) of respondents reported lower than desired customer satisfaction. Along with user complaints, customer satisfaction provides a powerful incentive for IT to make improving application performance a top priority. Business growth depends not only on expanding market opportunities and products or services that appeal to consumers but also on a business infrastructure that supports growth. In addition to infrastructure components such as servers, storage, and networks, applications may also need to be updated to accommodate additional transactions or users. Twenty-two percent (22%) of responding companies cited their current infrastructure's inability to support planned growth as a top pressure for adopting application performance management. Nineteen percent (19%) of companies reported that higher than expected IT cost was a top pressure. Even with careful planning, it is difficult to anticipate all of the events that might cause IT costs to exceed budgets, but most IT organizations invest in technologies and maintenance to prevent or mitigate disruptions to IT services. Managing application performance proactively is one way to prevent application issues from disrupting critical business processes or affecting user productivity. The business cost of application issues may not be as obvious as server or storage system failures, but frequent problems with applications can consume IT resources as quickly as any infrastructure problem. The Maturity Class Framework Aberdeen used three key performance criteria to distinguish the Best-in- Class from Industry Average and Laggard organizations: Mean time to repair critical application issues Average availability of critical business applications (excluding maintenance and planned downtime) Reduction in the number of calls to the help desk Companies with top performance in these criteria earned Best-in-Class status (Table 1). Additional information about the Aberdeen Maturity Class Framework can be found in Appendix A.

6 Page 6 Table 1: Top Performers Earn Best-in-Class Status Definition of Maturity Class Best-in-Class: Top 20% of aggregate performance scorers Industry Average: Middle 50% of aggregate performance scorers Laggard: Bottom 30% of aggregate performance scorers Mean Class Performance 1.6 hours mean time to correct critical application issues 99.6% average availability of critical business applications (excluding maintenance and planned downtime) 17% reduction in the number of calls to the help desk 18.6 hours mean time to correct critical application issues 96.9% average availability of critical business applications (excluding maintenance and planned downtime) 6% reduction in the number of calls to the help desk 27.9 hours mean time to correct critical application issues 95.9% average availability of critical business applications (excluding maintenance and planned downtime) 9% increase in the number of calls to the help desk Improvements in Other Application Performance Metrics Best-in Class companies also showed improvement in other metrics related to application performance (Figure 2). The ability to find issues before applications are deployed is essential to avoiding a host of problems once the application is in production, including slow performance, loss of user productivity, or even disruptions in business processes. On average, Best-in- Class companies were able to find 69% of application issues prior to deployment, compared to 40% for other companies. Regardless of whether application issues are found before deployment or in production applications, the faster issues can be fixed, the less likely they are to affect end-user productivity. Best-in-Class companies were able to fix 66% of application issues before they affected end users, while other companies fixed 47% of issues. Overall, end users reported fewer application issues in Best-in-Class companies (32%) than in other companies (37%). The lower percentage for Best-in-Class companies can be attributed to better performance in finding application issues prior to deployment and fixing them before end users are affected. While there is no set rule about what percentage of transactions should be monitored, most companies concentrate their efforts on monitoring and

7 Page 7 analyzing the transactions of applications that are considered "critical" to business operations, such as , database, and CRM applications. For Best-in-Class companies, the percentage of transactions monitored and analyzed is 38%, while other companies tracked 50% of application transactions. Figure 2: Best-in-Class Improve Application Performance Metrics 80% 60% 40% 20% 0% 69% 66% 40% Percentage of issues identified before deployment 47% Percentage of issues fixed before end users are affected 32% 37% Percentage of issues reported by end users 38% 50% Percentage of transactions monitored and analyzed Best-in-Class All others The Best-in-Class PACE Model Best-in-Class companies use a combination of strategic actions, organizational capabilities, and enabling technologies to ensure success with initiatives for application performance management. Table 2 summarizes the characteristics of these companies. Additional information about the Bestin-Class PACE Framework can be found in Appendix A. Table 2: The Best-in-Class PACE Framework Pressures Actions Capabilities Enablers End users complain about application response times Improve application usability based on end-user experience Integrate performance management inside and outside the corporate firewall Implement quality of experience (QoE) improvement program Ability to associate user transactions with business processes Problem resolution data is available for future reference Ability to measure session abandonment rate Centralized management of application performance Ability to monitor applications in virtualized environments Monitor and analyze network performance using software, a service, or an appliance (83% Best-in- Class adoption) Monitor and analyze server performance using software, a service, or an appliance (83% Best-in- Class adoption) Real-time alerts for applications and business processes (74% Best-in-Class adoption) Track business-critical transactions in real-time (72% Best-in-Class adoption) Monitor and analyze transactions using software, a service, or an appliance (67% Best-in-Class adoption) Monitor performance across multiple operating systems (61% Best-in-Class adoption) Traffic probes for measuring application performance (59% Best-in-Class adoption)

8 Page 8 Best-in-Class Strategies Complaints about application usability are the leading pressure driving companies to adopt performance management solutions, and as Figure 3 shows, improving usability based on understanding the end-user experience is the leading strategy of Best-in-Class companies (37%) to address this pressure. End users are especially sensitive to usability issues; an application that is perceived as difficult to use may affect productivity or increase the need for additional training and support from IT resources. A second strategy of Best-in-Class companies is to integrate application performance management both inside and outside the corporate firewall (32%). Integrating performance management across the firewall recognizes that the traditional idea of a monolith application infrastructure is evolving and addresses the need to manage applications regardless of where they or the end users are located. From a practical standpoint, managing applications across the entire infrastructure enables both IT and business managers to view all the components of performance with a "single pane of glass." "It s not just that applications are more complex, we ve come depend on performance management tools as a way to keep up with the speed of business on the Internet. The bank can t afford delays in fixing problems, it just costs too much money to be down. ~ IT Director, Retail Banking Institution Figure 3: Current Best-in-Class Strategies 40% 37% 32% 20% 23% 20% 26% 11% 11% 7% 0% Improve application usability based on end user experience Integrate APM inside and outside corporate firewall Implement quality of experience improvement program Evaluate ROI from application deployments Best-in-Class All Others Twenty-six percent (26%) of the Best-in-Class implemented a formal Quality of Experience (QoE) improvement program as part of their application performance initiatives. Successful QoE programs recognize the importance of including end user experience in application design. The result should be applications that are easier to use and increase the productivity of end users. A final strategy of Best-in-Class companies is to evaluate the ROI of application deployments (11%). In companies with a large number of end users, even incremental changes in applications may involve substantial cost and use of IT staff and resources. Evaluating the ROI for application deployments ensures that IT is employing appropriate methods to maximize

9 Page 9 the effectiveness and minimize the cost and disruption to end users and to business processes. Aberdeen Insights Strategy Best-in-Class companies outperformed their peers in application availability, time to repair issues, and calls to the help desk, but these gains come at a cost: Best-in-Class companies had 14% higher budgets for application performance management than their peers. The question is, are the performance gains worth the increased expenditure? Even without knowing actual budget amounts, it is possible to answer this question by looking at the opportunity cost of managing application performance. In this case, the opportunity cost consists of the expenditures that were avoided by investing in an application performance management solution. For example, a critical application issue that affects order processing has an number of costs associated with find and fixing it, such as: IT support and engineering time or labor to locate the problem Application engineering and quality control time or labor to repair and test the fix IT support time or labor to deploy an updated application to end users Any lost or delayed revenue due to the issue with order processing Any IT projects or tasks that were delayed while IT staff worked on finding and fixing the application issue The sum of all of these costs represents the opportunity cost of not investing in an application performance management solution, which primary purpose is to reduce or avoid these costs. If the opportunity costs for application issues exceed the costs of an application performance management solution, then there is ample justification to consider investing in a performance management solution. A further component of the formula is the work involved in monitoring transactions and sifting through the resulting data to identify and address the root cause of latency or downtime. Best-in-Class organizations mitigate this cost by monitoring a lower percentage of the total traffic. While some IT managers might presume that the most effective approach is to monitor 100% of transactions, our research shows that the "sweet spot" is to monitor about 35% to 40% of transactions. As shown in Figure 2, the Best-in-Class monitor one-third fewer transactions than all others, giving them the information they need at a lower cost. In the next chapter, we will see what the top performers are doing to achieve these gains.

10 Page 10 Chapter Two: Benchmarking Requirements for Success The use of application performance management as part of IT and business practice can have a profound impact on business processes, end-user productivity and how applications are designed for the organization. A successful application strategy must also integrate with business processes in such a way that the business, customers and stakeholders benefit. Case Study - AppMix AppMix is a Boston- and Mumbai-based IT consulting services provider specializing in application development reengineering, and support for the education, airline, and banking industries. Because IT consultants comprise AppMix s main workforce, most employees can resolve simple software issues on their own; AppMix's internal help desk handles mostly hardware and network problems. AppMix's CEO / CIO Kini Shah believes that focus on Application Performance Management (APM) gives AppMix a unique perspective on the challenges of providing application support services for its clients. AppMix clients were losing business and found that end-users were not happy with the performance of the application. In an end result clients were not becoming repeat customers. Our primary goal was to keep our customer s happy. At that point, AppMix brought in a comprehensive performance management application system. Weekly reporting of the system ensured that a continuous update was sent to all concerned parties, said Kini Shah CEO/CIO, AppMix. AppMix found that adding one full time resource to an existing group for up-keep and maintenance of the system is helpful to performance management. Shay said, In our first month we realized that there was a pattern in users who did not come back, further analysis lead to better system design and performance. In addition system up-time (outside of scheduled updates) after the first two months went from 98% to 99% a significant improvement and AppMix hope s to be at five nines (99.999%) by the end of the first six months after the implementation of the APM system. Shah truly believes that, An APM system is a must for all client facing applications. An APM system is reportable and accountable and also helps the business in analysis of where the users are spending majority of the time on the system. This data is then translated to additional sales/advertisements which easily augments the cost of the APM system. continued Fast Facts 58% of all responding companies reported that cloud computing will have a major impact on future IT projects or initiatives 70% of responding companies monitor performance inside the corporate firewall, 32% monitor performance outside the firewall

11 Page 11 Case Study - AppMix AppMix has seen differences in their organization since deployment of the APM system. They have received less support calls, which alone saves in support costs, and have obtained happy customers in the APM transition as well. Shah s final words were in what to look for in an Application performance management system; They all have some strength in certain areas. The key is to marry the technology you are using and the goals you want to accomplish with the new APM system. Once you take these into consideration there is usually a strong contender which has everything you are looking for along with costs considerations. Competitive Assessment Aberdeen Group analyzed the aggregated metrics of surveyed companies to determine whether their performance ranked as Best-in-Class, Industry Average, or Laggard. In addition to having common performance levels, each class also shared characteristics in five key categories: (1) process (the approaches they take to execute daily operations); (2) organization (corporate focus and collaboration among stakeholders); (3) knowledge management (contextualizing data and exposing it to key stakeholders); (4) technology (the selection of the appropriate tools and the effective deployment of those tools); and (5) performance management (the ability of the organization to measure its results to improve its business). These characteristics (identified in Table 3) serve as a guideline for best practices, and correlate directly with Best-in-Class performance across the key metrics. Table 3: The Competitive Framework Process Organization Knowledge Best-in-Class Average Laggards Ability to monitor applications in virtualized infrastructures: 47% 26% 12% Ability to monitor applications in cloud infrastructures: 33% 17% 4% Role-based dashboard to provide the results of application performance, analytics, and end-user experience: 50% 21% 4% Ability to associate user transactions with business processes: 67% 30% 20% Actionable performance intelligence is provided to key stakeholders: 33% 25% 12%

12 Page 12 Technology Performance Best-in-Class Average Laggards Real-time alerts for applications and business processes: 74% 45% 13% Track business-critical transactions in real-time: 72% 30% 13% Monitor application performance across multiple browsers or platforms: 44% 23% 15% Application performance management is integrated with end-user experience monitoring: 33% 21% 12% Ability to monitor application performance from different geographic locations: 53% 31% 8% Capabilities and Enablers Based on the findings of the Competitive Framework and interviews with end users, Aberdeen s analysis of the Best-in-Class shows that end-user experience monitoring must be combined with more traditional application performance metrics to form a consistent strategy of building applications that meet both corporate and end-user requirements. Process Best-in-Class companies (47%) have the ability to monitor application performance in virtualized infrastructures, compared to 26% of Industry Average companies and 12% of Laggard companies. Virtualization of the application infrastructure is an important component of current IT strategy and is a prerequisite for making the best use of cloud computing. Virtualization enables companies to consolidate application servers, increase the utilization and availability of storage, reduce the costs associated with data backup systems, reduce the overall space requirements and energy and cooling requirements for data centers. For many companies, the biggest change wrought by virtualization is philosophical: applications are no longer bound to specific hardware configurations or even operating systems. Virtualization allows applications to run virtually anywhere and this has liberated IT strategy from being bound to a physical infrastructure. Nevertheless, virtual application infrastructures may actually require more application monitoring and management than physical infrastructures because a poorly designed virtual infrastructure can cause reduced application performance and inefficient use of computing resources. Cloud-based applications share many attributes with applications running in a virtual infrastructure, including abstraction from the underlying hardware environment. From an IT standpoint, one of the major differences is one of "It s never easy to go back and ask for more money from management especially if they don't understand the problem. Using APM we re able to solve application faults more quickly and management is finally getting a picture of what we do." ~ IT Manager, Retail hardware chain in the Northeastern U.S.

13 Page 13 control: applications running in a corporate virtual infrastructure are still under the control of IT, even though they may run outside of the traditional data center environment. However, a cloud infrastructure is often owned and operated by a third-party provider that supplies the required hardware and operating system necessary to run the application. As companies consider moving applications into the cloud, the need to monitor application performance is just as important as it is with virtual or physical infrastructures. For this reason, 33% of Best-in-Class companies have the ability to monitor applications running in cloud-based infrastructures, as opposed to 17% of Industry Average companies and 4% of Laggards. Organization Role-based dashboards are a highly effective way to report on the various components of application performance, the end-user experience, and the analytics associated with managing the application infrastructure in real-time. Most dashboards display a variety of views customized to a person's role in the organization or degree of interest in technical detail. Summary or highlevel views of information are intended for business managers who need to understand the impact of application performance on key business processes, while IT personnel might be interested in a detailed view of transaction times and failure rates to determine the root cause of poor application performance. The ability to get needed information in real time to executives, business managers and IT staff enables them to make informed decisions more quickly. Fifty percent (50%) of Best-in-Class companies used role-based dashboards to provide the results of application performance, analytics, and end-user experience, while only 21% of Industry Average companies and 4% of Laggard companies possessed this capability. Knowledge Management The ability to associate user transactions with business processes is a cornerstone of understanding application performance, because it links end user actions to business processes, allowing analysis by both IT and business managers to locate critical application issues as well as ongoing analysis of application efficiency. Some of the ways this knowledge might be used include: By an IT systems analyst to correlate transactions generated by end users with the amount of time it takes to a business process to complete, identifying bottlenecks or choke points in the flow of transactions By an application developer to support application design changes to increase application efficiency By network engineers to increase the transaction processing capabilities or speed of the network Sixty-seven percent (67%) of Best-in-Class companies have the ability to use this knowledge, more than twice as many as Industry Average companies (30%) and almost four times as many as Laggard companies (20%).

14 Page 14 Best-in-Class companies (33%) exceed Industry Average companies (25%) and Laggard companies (12%) in having the ability to provide actionable performance information to key stakeholders. For example, storing transactions and root-cause information in a repository is only useful if this data drives an increased understanding of how an application fails or how its performance can be improved. In this example, the stakeholders might include IT staff, application designers, and business managers, all of whom need to understand how application performance affects their part of the organization. Technology Successful management of application performance includes two related capabilities of application management systems. The first is the ability to track business-critical transactions in real-time. A real-time view of critical transactions keeps both IT staff and business managers informed about the current state of critical applications and are essential to spotting issues that may have a disruptive impact on businesses processes. Seventy-two percent (72%) of Best-in-Class companies track critical transactions, while only 30% of Industry Average companies and 13% of Laggard companies have this capability. A related capability is the use of real-time alerts for applications and business processes which is used by 74% of Best-in-Class companies. Real-time alerts allow IT and business managers to respond to issues before they affect processes that are critical to the operation of the business. Only 45% of Industry Average companies and 13% of Laggard companies used real-time alerts. An increasingly diverse set of application performance monitoring tools allows companies to examine performance at many different points in the processing of business transactions, from the end user's desktop to the server that processes data. Figure 4 shows some of monitoring tools used by Best-in- Class companies to better manage application performance across the larger application infrastructure on both sides of the corporate firewall. Figure 4: Tools for Monitoring Application Performance 60% 50% 50% 53% 40% 20% 0% -20% 24% 9% Monitor performance across multiple browsers 34% 9% Monitor performance across multiple O/S 26% 12% 0% Monitor performance on mobile devices 38% 31% Es tablis h application performance thresholds Best-in-Class Average Laggards 22% 12% 0% Monitor enduser experience in the cloud

15 Page 15 For example, tools that that monitor application performance in multiple browsers or platforms are increasingly important to application designers, especially since the browser has become the preferred method of user interaction with applications. The building blocks of web pages (HTML and CSS) are standardized, but the application and execution of scripting languages, such as javascript and PHP, are not. For this reason, different browsers present application functionality differently; in some cases scripts may not function at all. This is not only irritating to end users; it may cause users to abandon a site in favor of one that works correctly. The ability to test and monitor the application performance in multiple browsers becomes a best practice when revenue is at stake. Fifty percent (50%) of Best-in-Class companies have the capability to monitor the end-user experience across multiple browsers, while only 24% of Industry Average companies and 9% of Laggard companies do. Performance Management The integration of application performance management with end-user experience monitoring provides IT with a complete picture of how applications perform under real conditions. Performance is not simply the ability to deliver transactions reliably to servers; performance also includes how end users interact with the application. A unified view of application performance also helps designers to build better applications. Thirty-three percent (33%) of Best-in-Class companies manage application performance and the end-user experience together, nearly twice that of Industry Average companies (23%) and more than six times that of Laggard companies (8%). The ability to monitor application performance from different geographic locations is important because the performance of the Internet is not a constant. Network performance not only varies from region to region, it also varies within a region, based on such factors as current network traffic, server outages, and time of day. Application performance can suffer greatly under these changing conditions so it is important for applications to be designed to perform correctly under a variety of conditions. To the extent possible, applications need to present end users with an acceptable experience regardless of their location and network conditions. For these reasons, 53% of Best-in-Class companies have the ability to monitor application performance from different geographic locations, compared to 31% of Industry Average companies and 8% of Laggard companies.

16 Page 16 Aberdeen Insights Technology A look at the tools that companies currently use to support application performance and the tools they plan to use (Figure 6) provides insight into the technologies that companies consider important for the future. Figure 5: Adoption of APM Tools 60% 40% 20% 16% 46% 49% 43% 39% 11% 12% 10% 22% 53% 0% Monitor application performance in the cloud M onitor enduser experience in the cloud Monitor application performance on mobile devices Tools to accelerate application delivery to mobile devices Unified platform to monitor application performance Currently use Plan to adopt The first two tools reflect the growing impact of cloud computing on future IT strategy, if not the increasing use of cloud as a way to supplement or even replace some parts of the internal infrastructure. Applications running in the cloud need to meet the same performance and user experience as those running in the internal infrastructure to be an acceptable replacement. Mobile devices present a challenge to IT to maintain application performance in spite of rapidly changing platforms, screen sizes, and device capabilities. Even so, the continuing adoption of mobile technologies by end users will eventually force companies to deliver applications that provide similar performance to ones running on the desktop. Companies plan to adopt performance monitoring (43%) and tools to accelerate application delivery (39%) to maintain performance on mobile devices. The adoption of a unified performance monitoring platform (53%) reflects the need for IT to consolidate the various tools that monitor application performance. A unified platform is easier to learn and operate than a collection of monitoring tools and presents a consistent display and user interface to both IT and business managers.

17 Page 17 Chapter Three: Required Actions Whether a company is trying to move its performance in application performance management from Laggard to Industry Average, or Industry Average to Best-in-Class, the following actions will help spur the necessary performance improvements: Laggard and Industry Average Steps to Success Increase the monitoring of virtualized and cloud infrastructures. Even though virtualization is considered an IT best practice by the computer industry, server virtualization still leads the list of projects that are consuming IT resources (67%), followed closely by cloud computing (58%). As Figure 6 shows, a significant percentage of Best-in-Class companies monitor application performance in virtualized and cloud infrastructures compared to Industry Average and Laggard companies. Virtualization is relatively easy to implement, but the difficulty of managing virtual applications increases rapidly as applications proliferate. In production environments, monitoring application behavior and performance is essential to maintaining throughput and the efficient use of computing resources and storage. Cloud computing infrastructures present the same challenges as virtualization, except that the underlying computer infrastructure is provisioned and managed outside of IT's direct control. Even with SLAs in place, monitoring performance in the cloud is one way of making sure that applications are performing to expectations. As Industry Average and Laggard companies consolidate their application infrastructure, or if they are considering cloud computing, they will need to maintain control of performance by increasing their use of monitoring. Fast Facts 47% of companies that manage application performance reported an increase in employee productivity 42% of these companies also reported an increase in customer satisfaction Figure 6: Monitoring and Managing External Applications 80% 60% 40% 20% 0% 58% 58% 47% 33% 26% 28% 26% 28% 13% 17% 19% 4% Monitoring applications in virtualized infrastructures Monitoring applications in cloud infrastructures Management of external applications Management of external user experience Bes t-in-class Average Laggards

18 Page 18 Implement performance management outside the corporate firewall. Best-in-Class companies lead both Industry Average and Laggard companies in their ability to manage application performance and user experience outside the corporate firewall (Figure 6). Running applications outside the traditional data center infrastructure raises challenges to performance that simply don't exist on an internal network. The quality and speed of the external network is highly variable which means that applications may or may not perform as designed. A good user experience is vital for applications on the external network; users have a low tolerance for slow applications or applications that are perceived as difficult to use. The solution is to manage applications so that they function well under all network circumstances and to design applications that are efficient on the network and which provide end users with quick performance and ease of use. The message is clear: Industry Average and Laggard companies need to look beyond the internal infrastructure to the applications running on the external network. Companies that make managing their external applications a priority mitigate the risk of losing of customers and revenue. Define baselines for acceptable performance. What constitutes acceptable performance for an application will be a matter of guesswork unless companies first establish baselines against which performance can be evaluated. For example, the amount of time it takes for an application to process and complete a transaction may be perfectly acceptable for business purposes, but end users may perceive that the application "takes too long" to respond, a very subjective assessment of performance. In large companies, IT professionals often compare the performance of a specific application to similar applications in the organization; in companies with fewer applications it may be easier to compare an application's performance to accepted industry values. Fifty percent (50%) of Best-in-Class companies have defined baselines for application performance, compared to 30% of Industry Average companies and 28% of Laggard companies. Use end-user experience to inform application design or changes. Applications may perform flawlessly in the purpose for which they were designed and still deliver unacceptable performance to end users or customers. For example, an application designed to collect patient information at a doctor's office may be work exactly as designed but users may find it difficult to use. Ignoring the end user experience may result in increased complaints to support or requests for additional training, or in severe cases, loss of end-user productivity. Using the experience of users as part of application design or redesign is likely to result in applications that meet end-user requirements for usability and response and still accomplish the tasks for which they were designed. Forty-seven percent (47%) of Best-in-Class companies use

19 Page 19 user experience as part of the application design process, while only 27% of Industry Average and 16% of Laggard companies do. Best-in-Class Steps to Success Increase the use of application monitoring. While Best-in- Class companies experienced a lower time to repair critical issues, higher application availability, and fewer calls to the help desk than other companies, there are some areas where Best-in-Class companies could improve their performance (Figure 7). Best-in- Class companies led other companies in their ability to monitor applications in virtual infrastructures, but less than half (47%) of them had that capability. The increasing use of virtualization in production environments means that applications need to perform as well or better than applications running in physical infrastructures. Monitoring application performance in virtual infrastructures ensures that performance is maintained throughout the application infrastructure. Cloud infrastructures present similar challenges to virtual infrastructures except that applications are generally running outside of a company's direct control. Only 33% of Best-in-Class companies monitored application performance in the cloud. Finally, only 22% of Best-in-Class companies monitor application performance on mobile devices. Mobile devices are rapidly becoming the application delivery platform of choice as end user adoption rates soar. The low switching cost for end users to move to new applications means that companies need to provide satisfactory performance and user experience on applications deployed to mobile devices or risk losing fickle end users. Figure 7: Increased Monitoring Improves Best-in-Class Performance Applications in virtual infrastructures 47% Applications in cloud infrastructures 33% Applications on mobile devices 22% 0% 20% 40% 60% Integrate application performance monitoring with enduser experience monitoring. Only 33% of Best-in-Class companies have integrated performance monitoring with user experience monitoring, a low percentage considering that 58% of

20 Page 20 Best-in-Class companies have centralized the management of application performance. The value of integrating the monitoring of these two key facets of application performance is that it provides a more complete picture of application behavior from the end user to the server. Consolidating the transactional view of performance with the user's view makes it easier to analyze applications under a variety of traffic and use conditions. An integrated view also speeds the process of locating application issues and provides detailed information for both IT staff and business managers. Aberdeen Insights Summary For early corporate adopters, the traditional idea of an application infrastructure running in a data center has already evolved into a mix of internal and external applications, server and storage. The decision about where applications should reside is still based on performance and cost, but these companies have recognized that alternatives such as cloud computing and mobile devices are not only viable, but are already impacting capital and operating expenditures in a positive way. For companies that are in the mainstream, extending the application infrastructure across the corporate firewall may seem both risky and insecure compared to a tightly managed internal infrastructure, but they are being slowly driven to adopt these technologies in response to shrinking IT budgets and staff in a time of business recession. As alternatives to a concentrated application infrastructure become more widely used it becomes easier to justify using them. What will finally motivate companies to adopt these technologies is the ability to keep business processes running at a lower cost and the recognition that revenue depends on satisfying the expectations of customers and end users.

21 Page 21 Appendix A: Research Methodology Between November and December 2010, Aberdeen examined the use, the experiences, and the intentions of more than 90 enterprises using APM in a diverse set of enterprises. Aberdeen supplemented this online survey effort with interviews with select survey respondents, gathering additional information on APM strategies, experiences, and results. Responding enterprises included the following: Job title: The research sample included respondents with the following job titles: Manager (34%); EVP / SVP / VP (12%); Director (11%); Consultant (11%); CEO / President (10%); Staff (9%); CIO (5%); and other (8%). Department / function: The research sample included respondents from the following departments or functions: IT manager or staff (50%); sales and marketing staff (17%); product development or engineering (7%); senior management (8%); operations manager (6%); and other (12%). Industry: The research sample included respondents from a wide cross section of industries. The sectors that saw the largest representation in the sample were: IT consulting or services (29%); telecommunications equipment and services (18%); software (15%); government and public sector (9%); healthcare, medical or dental devices and services (8%); and financial services (8%). Geography: The majority of respondents (63%) were from North America. Remaining respondents were from the Asia-Pacific region (15%), Europe (13%), and the Middle East and Africa (9%). Company size: Twenty-eight percent (28%) of respondents were from large enterprises (annual revenues above US $1 billion); 32% were from midsize enterprises (annual revenues between $50 million and $1 billion); and 40% of respondents were from small businesses (annual revenues of $50 million or less). Headcount: Forty-nine percent (49%) of respondents were from large enterprises (headcount greater than 1,000 employees); 21% were from midsize enterprises (headcount between 100 and 999 employees); and 30% of respondents were from small businesses (headcount between 1 and 99 employees). Study Focus Responding executives [retail] utives completed an online survey that included questions designed to determine the following: The degree to which RFID is deployed companies in are their actively retail using operations application performance and the financial implications management of or the are in the technology process of implementing it The What structure metrics companies and are effectiveness using to establish of existing and then RFID assess implementations their progress in application performance Current management and planned use of RFID to aid operational and promotional The benefits, activities if any, that have been derived from The application benefits, performance if any, that have been management derived initiatives from RFID initiatives The study aimed to identify The emerging study best aimed practices to identify for emerging application best performance practices for [RFID] management, usage in and [retail], to provide and to a provide framework a framework by which readers by which readers could assess could their assess own their own management capabilities.

22 Page 22 Table 4: The PACE Framework Key Overview Aberdeen applies a methodology to benchmark research that evaluates the business pressures, actions, capabilities, and enablers (PACE) that indicate corporate behavior in specific business processes. These terms are defined as follows: Pressures external forces that impact an organization s market position, competitiveness, or business operations (e.g., economic, political and regulatory, technology, changing customer preferences, competitive) Actions the strategic approaches that an organization takes in response to industry pressures (e.g., align the corporate business model to leverage industry opportunities, such as product / service strategy, target markets, financial strategy, go-to-market, and sales strategy) Capabilities the business process competencies required to execute corporate strategy (e.g., skilled people, brand, market positioning, viable products / services, ecosystem partners, financing) Enablers the key functionality of technology solutions required to support the organization s enabling business practices (e.g., development platform, applications, network connectivity, user interface, training and support, partner interfaces, data cleansing, and management) Table 5: The Competitive Framework Key Overview The Aberdeen Competitive Framework defines enterprises as falling into one of the following three levels of practices and performance: Best-in-Class (20%) Practices that are the best currently being employed and are significantly superior to the Industry Average, and result in the top industry performance. Industry Average (50%) Practices that represent the average or norm, and result in average industry performance. Laggards (30%) Practices that are significantly behind the average of the industry, and result in below average performance. In the following categories: Process What is the scope of process standardization? What is the efficiency and effectiveness of this process? Organization How is your company currently organized to manage and optimize this particular process? Knowledge What visibility do you have into key data and intelligence required to manage this process? Technology What level of automation have you used to support this process? How is this automation integrated and aligned? Performance What do you measure? How frequently? What s your actual performance? Table 6: The Relationship Between PACE and the Competitive Framework PACE and the Competitive Framework How They Interact Aberdeen research indicates that companies that identify the most influential pressures and take the most transformational and effective actions are most likely to achieve superior performance. The level of competitive performance that a company achieves is strongly determined by the PACE choices that they make and how well they execute those decisions.