Common Pitfalls in Implementing Application Performance Management

Common Pitfalls in Implementing Application Performance Management

Introduction On an ever-increasing basis, the typical organization s core business processes rely on a combination of applications and the supporting IT infrastructure. One of the implications of that situation is that if those applications, or the infrastructure that support them, are not performing well, then neither are those core business processes. In most cases, when a core business process is not performing well, the company loses revenue and the perceived value of the IT organization is diminished. As described in Application Performance Management: New Challenges Demand a New Approach 1, the traditional approach to application performance management is fundamentally flawed. One of the primary manifestations of the flaws is that in the majority of cases when the performance of an application begins to degrade, the degradation is noticed first by the end user and not by the IT organization. The goal of this white paper is to describe seven common pitfalls that IT organizations need to be aware of and to avoid when implementing application performance management solutions. To achieve that goal, this paper will incorporate recent market research, a study of the ROI that is associated with using application performance management in a pre-production environment and interviews with IT professionals involved with improving application performance. The Seven Pitfalls 1. Lack of Scalability One measure of scalability of an application performance management solution is the sheer number of transactions that the solution can support. Given the ongoing growth in the number of business transactions that IT organizations need to support, application performance management solutions need to be able to scale to support increasingly large numbers of transactions. The scalability of a system also refers to the ability of the system to support growth without incurring undo delay, complexity or expense. One of the factors that limit the scalability of an application performance management solution is if the solution requires that all packets be stored. This type of solution requires a large amount of storage and if the number of transactions were to double, an equally large amount of storage would have to be added. This results in increased cost and management overhead. A more effective solution is to provide the option to store all of the packets for the small number of situations for which that is a requirement. However, in all other situations, the solution should be able to filter out the vast majority of packets. This type of solution requires notably less storage and also means 1 http://www.webtorials.com/content/2012/09/apm-new-challenges-demand-a-new-approach.html 2

that an IT organization can support a doubling in the number of transactions by adding only a relatively small amount of storage. 2. Lack of a Comprehensive Solution In April 2012, 199 IT professionals were given a survey in which they were asked to indicate the type of outcome that typically occurs if one or more of their company s business critical applications are performing badly. They were allowed to indicate multiple outcomes. The outcomes that were mentioned the most often are shown in Table 1. Impact Percentage The Company Loses Revenue 62.0% IT Teams Are Pulled Together 59.8% Company Loses Customers 45.1% CIO Gets Pressure from his/her Boss 45.1% Harder for IT to get Funding 44.6% CIO Gets Other Pressure 42.9% Table 1: Impact of Poor Performance The fact that as shown in Table 1, in the majority of instances in which a business critical application is performing badly that IT teams are pulled together is further evidence that there are pitfalls in the selection of application performance management solutions that IT organizations need to avoid. That follows because if the current approach was effective, IT organizations would be able to rapidly identify the cause of degraded application performance and wouldn t need to pull teams together to do so. One of the primary reasons why it is so common to have to pull IT teams together to identify and resolve performance issues is that it is common for IT organizations to have separate tools to manage the application, the network and the other components of the infrastructure. In almost all cases, these tools are not integrated and so the IT professionals who are familiar with them have to go to what is commonly referred to a war room and try to piece together the root cause of the problem while acting as human interfaces between the disparate management systems. This manual approach to integrating and analyzing disparate sources of management data tends to elongate the amount of time and increase the amount of resources that it takes to do troubleshooting. The time and resources that are consumed by war room troubleshooting can be either eliminated or dramatically reduced if IT organizations implement a comprehensive 3

application performance management solution one that effectively integrates application, infrastructure and network management. 3. Paying for Multiple Solutions As discussed above, it is common for IT organizations to have separate tools to manage the application, the network and the various components of the infrastructure and that in virtually all instances, these tools are not integrated. One downside to this approach was discussed above it results in elongating the amount of time and increasing the amount of resources that it takes to do troubleshooting. Another downside of this approach is that it significantly increases the cost associated with ensuring acceptable application performance. This includes the cost of various management tools and the infrastructure that is necessary to run the tools. It also includes the cost of the IT professionals who support each tool. The following paragraph discusses yet another negative impact of not having a comprehensive solution. 4. Complexity Another negative impact of not having a comprehensive solution is that it increases the overall complexity of management. This follows because the previously discussed war room approach to management forces humans to perform the complex tasks of integrating and analyzing disparate sources of management data. In addition, in some instances, the lack of integration between management systems results in IT organizations having to make-up or contrive data for one management system even though production data is available in another management system. A more effective approach is to implement an application performance management solution that integrates with other management systems, such as a capacity management solution. This would enable the capacity management solution to leverage the production data that is in the application performance management solution and hence avoid having to do capacity planning based on contrived data. In addition, the application performance solution should be integrated with other key management solutions, such as an infrastructure management solution. In that way, IT organizations can quickly determine if the source of the problem is the application or the infrastructure. This type of integration reduces the time it takes to do root cause analysis and it also eliminates the complexity and frustration that is associated with the war room approach. 5. Having a Focus Just on the Production Environment One of the major pitfalls associated with application performance management is that it is typically implemented in the production environment, but not in the application development environment. A Performance Engineer at a major financial institution highlighted the advantages of using APM in the development environment when he stated that there is a lot 4

of focus within his company on setting and meeting SLAs. He gave the example of where the SLA of an application that was recently under development called for a five second response time, but that the pre-production testing they did showed that there would be a nine second response time. This data was brought to their business partners because It is the business partners that have to determine how they want to spend their funds. They could require code changes or they could require broader architectural changes. The Performance Engineer went on to say that the pre-production test data that they bring to their business partners helps the business partners decide how much of their funds they will spend on the functionality of a new application and how much they will spend on the performance of that application. The financial institution conducted a study to quantify the ROI that is associated with their using application performance management tools in a pre-production environment. The results of that study, which are shown in Table 2, highlight the significant savings that are associated with using application performance management in a pre-production environment. Best Case Most Likely Worst Case Simple ROI 393% 304% 197% Payback (months) 6 Months 6 Months 8 Months Internal Rate of Return 280% 225% 156% Table 2: Summary of the ROI Analysis 6. Lack of a Business Focus As noted in the introduction, when applications, or the infrastructure that support them, are not performing well, then neither are the core business processes that are supported by those applications. However, in most organizations, the entire focus of application performance management is technical; e.g., the applications and the infrastructure that support those applications. In addition, in most organizations, application performance management tools are used exclusively by the IT organization. IT organizations will be much more successful with application performance management if the solution they implement links the performance of end-user transactions to business outcomes. This approach enables IT organizations to prioritize problems by business impact to assure that high-value services and end-users receive the highest services levels in alignment with business objectives. In addition, IT organizations can build strong bonds with the company s business managers if they implement application performance management tools that can provide business managers with valuable insight into the transactions that support their key business processes. This insight can either come directly 5

from the application performance management tools or from these tools used in conjunction with other business intelligence (BI) tools. 7. Lack of Support for Business Accountability As noted in Table 1, in the majority of instances in which one or more of the company s business critical applications are performing badly, the company loses revenue. Because of the impact of poor application performance on revenue, IT organizations are held accountable for application performance in general, and for the performance of business critical applications in particular. While ensuring acceptable application performance is always a challenge, it is even more challenging if the application is accessed in total or in part from a public cloud provider because in those situations the IT organization loses control over the underlying application delivery infrastructure. In order for an IT organization to satisfy management s expectations of application performance, it needs to deploy application performance management solutions that can provide visibility into performance, help ensure service level-agreements are met, warn of degrading application performance and quickly pinpoint problem root cause. The solution must provide the capability to deploy management agents at a public cloud provider s sites in order to get visibility into the end-to-end performance of every transaction and to be able to discover potential problems and perform rapid root cause analysis. The application performance management solution should also provide the capability to use synthetic transactions to monitor Web application performance from outside the firewall; i.e. from the end-user s point of view. Summary and Call to Action The vast majority of IT organizations have already implemented application performance management. However, due to a flawed approach to application delivery management, only a small minority of IT organizations has realized the promised benefits. The inability of IT organizations to fully realize the promised benefits of application performance management diminishes the perceived value of the IT organization. This inability also has a significant business impact because in the majority of instances in which the performance of a business critical application is performing badly, the company loses revenue. There are seven major pitfalls that IT organizations that are either just deploying application performance management solutions for the first time, or that are implementing new solutions must avoid. Those pitfalls are: 1. Implementing a solution that doesn t scale to support the number of transactions that must be supported without a significant increase in cost and complexity. 6

2. Implementing a fragmented solution that results in the majority of performance problems being resolved in a war room environment with humans acting as the interface between the disparate management systems. 3. Paying for multiple fragmented solutions and still having to resort to a expensive, timeconsuming war room approach to troubleshooting. 4. Implementing a set of management solutions that require that IT professionals perform the complex and frustrating task of acting as the interface between the disparate management systems. 5. Deploying application performance management solutions just in a production environment and failing to realize the benefits of using these solutions in a preproduction environment. 6. Implementing solutions that focus just on technology, and not on business transactions and failing to use those solutions to provide business managers with insight into the operations of their businesses. 7. Implementing a solution, whether that is in a traditional IT environment of one that leverages a public cloud provider, that doesn t enable the IT organization to satisfy management s expectations of application performance. 7