1 EMA Advisory Note: IT Speaks: Practices and Trends in Enterprise Application Management (Summary) Sponsored by: More than 200 IT professionals with direct responsibility for managing business applications were surveyed regarding practices, products, and challenges. The results provide useful guidance for enterprise management vendors and enterprise IT professionals alike. Written by Julie Craig Enterprise Management Associates (EMA ) Senior Analyst Practice Lead, Application Management Practice Managing Today s Business Applications: A New Imperative Deploying business applications, such as customer relationship management (CRM), enterprise resource planning (ERP), and vertical industry applications is complex and timeconsuming. Managing them is even more challenging. EMA research has found that while enterprise IT organizations are very good at deploying and maintaining infrastructure, they are still struggling to effectively deploy and manage the business services and applications that infrastructure supports. At the same time, one of IT s biggest stated challenges is aligning with the business. One of the best ways to achieve such alignment is to put tools in place that provide a clear picture of the links between infrastructure and applications in essence, a model of IT technology as it aligns with business imperatives. With this in mind, EMA published cross-functional research in early 2008 outlining a revolutionary new paradigm for managing enterprise applications. That paper, entitled Managing Applications in 2008: What Does End-to-End Really Mean?, details the product capabilities required to develop a holistic, endto-end view of business applications in context with underlying infrastructure. This broad, deep perspective is a fundamental requirement for managing performance, availability, and quality of experience (QoE), and for managing change. This research builds on the ideas outlined earlier in the year with a follow-up survey conducted in June, More than 200 IT professionals with direct responsibility for managing business applications were surveyed regarding practices, products, and challenges. The results provide useful guidance for enterprise management vendors and enterprise IT professionals alike. Vendors will likely find IT s concerns and challenges of value in making decisions regarding where to focus research and development (R & D) budgets. This report also highlights significant changes in the composition of support groups, and this is useful information for building and training sales teams. A new constituency is driving application-related initiatives, and the traditional emphasis on selling to IT Operations may no longer be the best strategy. Today s business applications are being managed primarily by cross-functional
2 infrastructure management and/or architecture teams, professionals with the cross-silo skills necessary to troubleshoot extremely complex, tiered applications. For enterprise IT executives, this research represents current best practice. In conjunction with the Application Management Semantic Model first proposed in the above-referenced paper, it offers guidance about the kinds of capabilities required to develop a businessfocused management perspective. The goal of best practices is to generate organizational maturity by optimizing both processes and technology. While best practice libraries provide extensive guidance on the process side, research highlights the technology requirements for managing business services as they become increasingly complex and to do so at reasonable cost. Key Findings Application Support Is Being Done By Specialists with Cross- Functional Skills Traditionally, IT support has been the sphere of Operations teams consisting largely of administrators with skills in one or more technology silos. This study found that the primary responsibility for managing enterprise applications now lies with IT architecture and/or infrastructure services groups. Respondents indicating that data center personnel, development, network operations teams, and line of business IT were doing this function were in the minority. Since application management requires a much more cross-functional support approach, this is not surprising. This does, however, represent a fundamental shift in support processes and practices that accounts in part for rising IT costs. Specialists with these skills are expensive, and enabling them with tools sets the stage for significant return on investment (ROI) over time. Custom Applications Predominate Over Packaged Applications This study found definitively that custom applications are the most common type of enterprise application. Seventy-eight percent of those surveyed reported that they are supporting custom applications, while 66% reported packaged applications. This statistic is of interest primarily because many of the application management products in the market place most notably application discovery and dependency mapping products are geared primarily towards packaged applications. The most common way IT finds out about application-related problems is (still) via telephone calls from users, and this percentage is up approximately 10% from two years ago. IT Is Becoming Less, Rather Than More, Proactive In Terms Of Application Management The most common way IT finds out about application-related problems is (still) via telephone calls from users, and this percentage is up approximately 10% from two years ago. IT Personnel At All Levels Have Made The Connection Between Change And Problems IT professionals report that weak change management processes and/or change governance are the primary factors limiting their effectiveness in supporting applications. They also report that, in addition to contributing to problems and downtime, change fallout also impacts business and customer satisfaction.
3 IT professionals report that weak change management processes and/or change governance are the primary factors limiting their effectiveness in supporting applications. 1. Consolidated event correlation 2. Change tracking, verification, auditing 3. Root-cause analysis Best Practices Best practice adoption continues to grow, but, in the U.S. at least, companies leveraging best practices are still in the minority. Fortyfive percent of those surveyed reported use of best practices. Of these, the IT Infrastructure Library (ITIL) was the most common, followed by the Capability Maturity Model Integration (CMMI), Six Sigma, and Control Objectives for Information and related Technology (COBIT) An extrapolation of these findings reveals many companies using multiple frameworks. Most Wanted Product Capabilities The three product families on IT s most wanted list include: Consolidated event correlation is number one on IT s most wanted list, and root-cause analysis capabilities are number three. However, they are actually two sides of the same coin. Root-cause analysis requires very robust event correlation capabilities, and the findings detailed in this report support the premise that such correlation has to be at the business service level. In other words, just as applications are supported by heterogeneous, multi-vendor technology, next generation correlation engines must be robust enough to understand and correlate information coming in from multi-vendor solutions supporting applications, transactions and heterogeneous infrastructure. Without such capabilities, IT specialists are flying blind when it comes to root cause analysis. This statement is borne out by the finding that the majority of application-related problems are still resolved with tribal knowledge as opposed to application management products. Change is a primary contributor to application outages, and IT gets the value of being able to track, verify, and audit changes. Key Recommendations Investments In Sophisticated Cross-Functional Correlation Capabilities Pay Off For Both Vendors And Consumers. An increasingly automated and sophisticated approach to isolating the root cause of application-related problems is key to supporting n-tier and complex transactions. There is high demand for products that translate this requirement into reality. Troubleshooting requires correlation, and in lieu of automation, this is a primarily manual process. While most IT organizations have already put infrastructure monitoring solutions in place, few have access to a single, comprehensive, business-focused perspective on the relationships among infrastructure, applications, transactions, and application health. Correlating information across technology silos, multi-vendor solution sets, and from multiple vantage points generates the detailed information needed for rapid root cause analysis, and also forms the foundation for increasingly automated problem remediation capabilities.
4 While traditional consoles and frameworks offer these capabilities at some level, most focus on a bottom-up approach that generates this view via metrics from infrastructure instrumentation. While this is certainly of value in supporting the data center, it is of limited value in supporting applications. Managing today s complex applications requires both broad and deep visibility that can be generated by combining bottom-up metrics with top-down transaction information. What IT is really saying with this finding is that an either/or approach perspective to infrastructure health or transaction health is self-limiting. A correlated combination of the two is today s leading edge, and an absolute requirement for correlating execution problems to underlying root cause within complex tiered applications. This transaction-focused capability can be achieved with multiple types of products. Synthetic and real transactions, network sniffing, transaction observation, and similar technologies provide transaction-level visibility. Combined with information on infrastructure performance and health, and ideally with application dependency mapping and an accurate CMDB, this provides a new kind of correlation capability that enables better management of today s complex business services. Best Practice In Application Support Is Trending Towards Cross- Functional IT Architecture and Infrastructure Specialist Teams, and Away From Siloed Support Teams While support has traditionally been the responsibility of Operations, this study found that the task of supporting today s applications has been shifted to IT infrastructure experts with cross-functional, cross-domain skills. 46% of the IT personnel responding to this study were part of either architecture or infrastructure planning teams, and this finding has multiple ramifications. One is that vendors may need to reconsider who their prospective customers are. In many cases, sales teams are trained to speak to the problems of IT Operations. The strong focus of these findings on the need for cross-technology correlation capabilities demonstrates the fact that today s sales efforts will likely require that architects and/or infrastructure R&D specialists become part of the sales discussion as well. Another ramification is that emerging best practice appears to indicate that application support requires personnel with higher-level cross functional skills. Specialists with a good grasp of multiple technologies, as well as business applications, are typically more experienced than those with silo skills, and this uncovers an additional takeaway. This is likely one of the major reasons why the cost of IT administration is so high, averaging approximately 75% of budget. In lieu of automation that intelligently correlates cross-enterprise infrastructure and application information, this task falls to IT infrastructure specialists. Such specialists are both scarce and expensive. Moving IT As An Industry From Reactive To Proactive Application Management Requires Significant Investments From Both Vendors and Consumers While best practice adoption stresses the need for a proactive, continually improved approach to managing business services, IT is struggling to make this a reality and losing. Over the past two years, the percentage of application-related problems reported by users (versus detected by technology) has risen from 43% to 54%. Without products
5 that can improve these percentages, managing business applications remains a primarily reactive activity. While it is obvious that IT is becoming much more capable at deploying complex technology, including SOA and Web services, it is also true that for the average IT shop, the management function appears to be an afterthought. IT personnel are working overtime and tell us that, nevertheless, project backlogs continue to grow. Getting a handle on virtualized, distributed composite transactions, black box integrations, custom applications that have evolved over eons, and connections with services running external to the organizational firewall, is not a simple feat. Leading-edge solutions will eventually help to turn this tide, and some excellent solutions are already available, but in the meantime IT as an industry appears to be losing ground. This paper will be followed in Quarter Four, 2008 with a landscape paper detailing products with capabilities in each area of the EMA Application Management Semantic model. Configuration Management and Change Governance Products Build A Foundation For Proactive Application Management Five years ago, best practices, and particularly change management processes, were being driven by executive leadership. Change governance was seen by many IT administrators as an impediment to the problem resolution process. What a difference a few years makes. IT line staff and managers alike have now made the connection between change and problems, with this study finding that change tracking, verification, and auditing were second on the IT wish list, after correlation. Vendors have been investing in products with these capabilities for several years, and multiple vendors are well-positioned with seasoned and robust solutions. On the IT side, it appears that strategic investments in change management processes and products can yield a harvest of low-hanging fruit in terms of efficiency and effectiveness. This is particularly true for those that are already focused on best practices, and where a Configuration Management System is already in place. Best Practices Are A Good Investment The industry at large is gravitating to best practices because of their promise to improve IT effectiveness. Although most best practice thought leadership revolves around processes, one of the most tangible, and most useful, outcomes of best practice efforts has been with Configuration Management systems, embodied in the federated Configuration Management Database (CMDB) envisioned by ITIL. Research conducted across healthcare IT organizations in the spring of 2008 found that 28% of companies surveyed already had a CMDB in place, 33% were in the process of putting one in place, and 39% were in the planning stages. Significantly, 0% have no plans for a CMDB. EMA has been beating the CMDB drum for over five years, and the main reason why is because it is impossible to manage what you don t understand. Correlation, Change Management, Release Management, Incident Management almost every tactical or strategic IT activity requires a clear picture of the relationships among the multiple elements comprising each business service. Configuration Management Systems provide logical models of the IT ecosystem that serve as the starting point, not just for change control, but also for all of the other activi-
6 All large organizations have pockets of silos, and with money as tight as it is, it is extremely difficult to work in an environment that has no standards or where people don t adhere to standards. Foglight elevates Quest Software into the realm of application management vendors and effectively addresses almost every application type, including SOA. ties related to managing technology-based business services. In addition, since garnering the benefits attributed to best practice initiatives requires a cross-discipline and even a cross-business communication and planning strategy, they tend to build and strengthen these cross-business links. Best practices promote the kinds of cross-functional, cross-business efficiencies that are the hallmark of successful Service Oriented Architecture deployments. Leading-edge adopters tell us that the organizational challenges involved in successfully deploying both SOA and CMDB are greater by far than the technical challenges. Best practices build bridges between cost centers, departments, and teams, and efficiency in delivering IT services requires this collaboration. One of the focal interview subjects for this project, a VP of Global Service Level Management for a worldwide financial services company, said it very well. He states, Automation is definitely one of the keys, and it is needed especially for companies of our size. However, automation alone doesn t cut it. All large organizations have pockets of silos, and with money as tight as it is, it is extremely difficult to work in an environment that has no standards or where people don t adhere to standards. In today s regulatory environment, compliance and regulatory requirements have to be adhered to even within silos. Without standards, we can t assure that this is happening. About Quest Foglight is Quest Software s flagship end-to-end application management solution, incorporating Service Level Management (SLM) and End User Management as well as Application Performance, Database, Infrastructure and Virtualization Management capabilities. Foglight s broad technology coverage includes support for Java and.net, Siebel, PeopleSoft, SAP, and Oracle Enterprise applications. It also covers end-user monitoring for Apache, Microsoft IIS, Citrix, and Rich Clients via real performance analytics as well as synthetic transactions. At the heart of Foglight is a recently updated SOA-based architecture that enables comprehensive application management at significantly decreased time-to-value. Built around that architecture is a foundation that provides the essentials to support Foglight s extensive set of out-of-the-box technologies. This elevates Quest Software into the realm of application management vendors and effectively addresses almost every application type, including SOA. This also positions Quest well to respond quickly to technology changes without affecting customer installations. By correlating hundreds of relationships and thousands of metrics into a single source of application insight, Foglight enables IT organizations to proactively detect, diagnose to the root cause, and resolve performance and availability problems in a priority that makes sense to the business.
7 Distinctive features include: Application Management Monitoring and correlation from the business services layer to the supporting infrastructure components Transaction monitoring and correlation from end-user to database code level Application dependency modeling Fast detection, root-cause diagnosis and resolution of issues Continuous performance management for Java development and QA using JProbe Service Level Management Cross-domain service-modeling Role-based dashboards Service-chain discovery and mapping Prioritization of problems based on business impact End User Management Real user session recording builds a transaction-based view of production business services and includes Tivo-like replay of problem transactions HTTP traffic analysis extracts business-focused information such as daily order totals from HTTP traffic. This is a key leading-edge technology that only a few vendors are offering, and it is garnering considerable attention for its potential to build bridges between IT the business. Pulling business-focused information out of management products is the wave of the future, and one that will likely improve the value proposition of Foglight, making it an easy sell to business-focused executives. Synthetic and real user performance analytics at network and application layers Infrastructure Management OS, Network Device and Middleware management Ability to integrate existing 3 rd party monitoring tools Virtualization Management Role-based views to help users quickly understand performance for each resource across each layer of the environment Workload migration modeling helps determine if planned moves will have unfavorable impact on resources, applications and end users before they are made Asset and configuration tracking provides insight into what happened, when it happened and why to identify impact on application performance and availability as well as on other virtual machines Database Management Monitoring, Diagnostics & Analytics SQL Optimization Database Capacity Management Storage Optimization Toad Development/Admin
8 With a staff of four, we don t have time to go through logs to make sure things are working correctly. said Bob. Foglight manages system logs and notifies us when it sees something abnormal. Notification is set via a rules-based manager and problem events pop up on the console. (Bob Heist, Lead Administrator, JT3) Case Study: Quest Software s Foglight Supports Testing of Defense Equipment with Advanced Root Cause Correlation Lisa Gray and Bob Heist are the Chief Information Officer (CIO) and Lead Administrator respectively for JT3, a joint venture company created for the purpose of providing engineering and technical support to the J-Tech (Joint Range Technical Services) test and training ranges. They are part of a 15 year defense contract supporting range operations and maintenance for large military and defense projects. Their high level mission is to ensure that JT3 projects stay on track, on time and on budget by proactively monitoring availability and performance of all applications that support development and testing efforts. At JT3 all projects undergo exhaustive testing throughout the development process, and project examples include the F35 and the cruise missile. Two key applications supporting this process are the asset management and maintenance applications that support project planning, management, and completion. As a government contractor JT3 is compensated solely on performance as measured by the overall success of the flight missions they support. Key performance indicators include application up time and availability, Help Desk calls answered, Help Desk tickets closed, and overall cost of providing services. Application down time impacts their earnings, as do projects that do not meet budget, time, or deliverable requirements. JT3 uses Quest Software s Foglight to monitor and manage the user experience of these and other critical web-based applications including Service Oriented Architecture (SOA) and web services, custom integrations and Application Program Interface (API)-based transactions among others. Prior to Foglight, Bob s team flew by the seat of our pants. There was no way to know when there was a problem, or what that problem was and JT3 managed production issues reactively, with a user on the telephone providing the first inkling of an application problem. When a user called, support teams had to track down the source of the problem, which typically involved the time-consuming process of sifting through logs and manually checking device consoles. JT3 specifically chose Foglight because of its end-to-end monitoring capabilities. During product evaluation, they did look at several other solutions, but Foglight came out on top as some competing products did not support J2EE-based web applications, while others lacked the capabilities necessary to monitor and manage JT3 s complex and mission-critical applications. Once they decided to go with Quest Software, consultants were on-site for approximately one week to do the Foglight installation. Quest Software s support and program management personnel continue to work with JT3 on an ongoing basis to help with configuration and training. JT3 has used Foglight for two years, and is pleased with the results. Foglight has enabled them to take a much more proactive
9 Integrating Foglight reduced our mean time to repair application issues from 2 hours to 10 minutes by capturing and correlating critical application performance data. (Bob Heist, Lead Administrator, JT3) management stance and they no longer have to wait for customer calls to determine when an application problem occurs as Foglight automatically tracks both current and normal application response times. With a staff of four, we don t have time to go through logs to make sure things are working correctly, said Bob. Foglight manages system logs and notifies us when it sees something abnormal. Notification is set via a rules-based manager and problem events pop up on the console. Another benefit is that Foglight has essentially taken over the monitoring of back end systems resulting in an approximate 50% time savings and enabling Bob s team to devote more time to actively managing servers. Foglight not only does application monitoring, it also monitors the entire infrastructure as well. It is not limited to Windows and Unix, but also has expansion capabilities-- if we want to write our own interface to an AS/400 or other OS type, we can do that. JT3 also likes the fact that new product versions are easy to install and require no additional consulting effort. Foglight has also simplified our root cause analysis efforts as well. Instead of poring through logs, Foglight provides visibility into each transaction. It reports the time required for each step to execute, including time spent at the user end, on the network, on the server, accessing the data base, and sending processed information back to the user. If a problem occurs, we can find the problem in two clicks, Bob reports. He continues, Integrating Foglight reduced our mean time to repair application issues from 2 hours to 10 minutes by capturing and correlating critical application performance data. It also provides helpful views into the application and allows us to look into SOA-based, distributed, and web transactions. Before Foglight, we monitored applications using point products, with minimal perspective to the overall transaction. With Foglight, application related problems became obvious and are resolved quickly. We have also been very happy with the Quest support organization. It has been far above the support we get from other companies, which is one reason why we continue to invest in Quest. Enterprise Management Associates, Inc Central Avenue, Suite 105 Boulder, CO Phone: , Fax: , Web: EMA, ENTERPRISE MANAGEMENT ASSOCIATES, and the mobius symbol are registered trademarks or common-law trademarks of Enterprise Management Associates, Inc.