Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com WHITE PAPER Managing Web Applications Infrastructure with IBM Tivoli Monitoring Sponsored by: IBM Corporation Tim Grieser August 2003 INTRODUCTION IBM Tivoli software has established a strong reputation for providing comprehensive software products that enable IT organizations to manage the performance and availability of multiple systems and applications to meet desired service goals. Initially aimed at managing client/server environments, IBM Tivoli software performance and availability management software has been continuously extended to address the ever-growing applications landscape, especially with respect to multitier and Webbased applications. Given the increasing complexity of distributed applications architectures, performance and availability management software must be able to address the particular requirements of optimizing specific components, such as Web servers and J2EE-based application servers, as well as managing multiple server operating environments (Unix, Linux, Windows, z/os) across multiple physical locations. THE GROWING IMPORTANCE OF WEB APPLICATIONS In today's rapidly evolving business and technology environments, IT is increasingly becoming a direct provider of online services to end users, with growing requirements to support customer and employee access to applications over the Web. In this environment, managing the performance and availability of Web-based applications is an increasingly important responsibility. This task becomes especially critical for ebusiness applications because the successful completion of revenue-generating financial transactions is highly dependent on the infrastructure managed by IT. In such cases, slowdowns or outages can result in significant revenue losses not to mention bad publicity and even a negative impact on corporate stock valuations. Of course, the exact cost or revenue loss due to poor performance or system outages will depend on the specific application and on the corporate business model. For example, significant downtime in high-value financial services applications, such as banking or online stock trading, can result in millions of dollars in lost revenue. Slowdowns or outages affecting inward-facing applications, such as sales management or HR, can result in a significant decrease in user productivity. The increasing importance of Web-based applications means that more than ever before, IT needs software tools that can monitor and manage the infrastructure to meet required applications service objectives especially high availability under heavy loading conditions and fast response times for end users.
WEB APPLICATIONS COMPONENTS Applications architecture has long since evolved beyond the classic single-server mainframe environment to include distributed approaches such as network-linked client/server components (especially for Unix and Windows platforms) and multitier deployments with application servers and databases. Often, multiple instances (or images) of each tier are realized in the form of clusters or server farms, adding to the complexity. In many cases, heterogeneous platforms must be supported where distributed application components can reside in differing OS environments. Adding to this complexity, the Web brings new components to applications infrastructure, including Web servers, such as Apache and IIS, and J2EE-based applications servers, such as WebSphere and WebLogic. Figure 1 illustrates typical components found in today's Web-based applications. FIGURE 1 TYPICAL WEB-BASED APPLICATION COMPONENTS RDBMS Edge Server Web Server Application Server Middleware Web Application Server Legacy Systems MANAGING WEB APPLICATIONS Critical requirements for managing Web applications include ensuring that they are up, available, and responding well from an end user's viewpoint. Applications performance and availability are often managed from two perspectives: the "individual element," or "resource," viewpoint and the "transaction," or "end-to-end," viewpoint. The individual element viewpoint focuses on optimizing the performance and availability of each application infrastructure component, such as J2EE application servers and Web servers. The "transaction" viewpoint aims at managing the overall performance of an application transaction, starting from the time the transaction is initiated by an end user and following the transaction through all the infrastructure components it must visit to receive required service, such as a Web server, an application server, and back-end database. The overall transaction performance is 2 #3737 2003 IDC
determined by accumulating the performance measures received at each infrastructure component in the chain between the end user and the completed transaction. Clearly, Web infrastructure elements such as Web servers and application servers are key components in determining whether an application's overall performance and availability are acceptable or fall within desired service objectives. THE IBM TIVOLI SOFTWARE APPROACH TO INFRASTRUCTURE MANAGEMENT IBM Tivoli software has adopted a three-layer approach to systems and applications infrastructure management. The foundation is a base layer of individual component resource monitoring together with local problem resolution, often at the individual server level. The second management layer moves up to a multicomponent view, which includes cross-domain event correlation to determine root causes and automated action-taking to resolve problems. The highest layer, business service management, is based on analyzing data stored in the IBM Tivoli Data Warehouse to facilitate decision support and to provide business intelligence. Figure 2 illustrates the three-layer IBM Tivoli software approach to system and applications infrastructure management. FIGURE 2 IBM TIVOLI SOFTWARE INFRASTRUCTURE MANAGEMENT APPROACH Business Service Management Align IT management with business objectives Assure customer service levels Turn data into actionable information Business-aligned dashboards Correlation and Automation Continuous availability of the business runtime environment Root cause determination Automatic application of event-driven and scheduled activities Autodiscovery of resource and business relationships Monitoring Identify problem signatures and symptoms Notify operations and higher layers Cure problems at their source Transaction performance analysis Configuration and Change Autodiscovery of resource inventory Autodiscovery of resource relationships Manage business system configurations Manage infrastructure changes IT Resources Business Runtime Environment Application Transactions Business Processes 2003 IDC #3737 3
IBM TIVOLI MONITORING FOR WEB INFRASTRUCTURE IBM Tivoli software provides products to manage business-critical applications. These products include the base IBM Tivoli Monitoring product, IBM Tivoli Enterprise Console, IBM Tivoli Configuration Manager, IBM Tivoli Business Systems Manager, IBM Tivoli Data Warehouse, and a family of IBM Tivoli Monitoring products to address specific components of the business applications infrastructure. IBM Tivoli Monitoring for Web Infrastructure (ITM for WI) is monitoring software used to help ensure the performance and availability of Web servers and J2EE-based application servers. In terms of the multilayer approach to applications management, ITM for WI addresses the specific monitoring layer requirements needed to manage J2EE-based applications servers, such as WebSphere Application Server and WebLogic, as well as standard Web servers, such as Apache, IIS, and iplanet. IBM Tivoli Monitoring for Web Infrastructure monitors Web servers and application servers and provides real-time status on performance and availability. ITM for WI provides a single point of control, through the common IBM Tivoli Health Console, that enables IT organizations to understand the health of the key elements of a Webbased environment. It allows administrators to identify problems, alert appropriate personnel as required, and offer a means for automated problem response and correction. The data collected by ITM for WI can be fed to the common IBM Tivoli Data Warehouse for extended analysis of performance and availability, such as historical reporting and trending. Data in the warehouse can also be monitored in conjunction with data from other sources for higher-level business service analysis. Figure 3 shows an overview of ITM for WI. FIGURE 3 IBM TIVOLI MONITORING FOR WEB INFRASTRUCTURE OVERVIEW Business Service Management Event Correlation and Automation Monitor Web Servers Apache HTTP Server IBM HTTP Server iplanet Web Server Internet Information Server Monitor Application Servers WebSphere Application Server WebLogic Server IBM Tivoli Monitoring for Web Infrastructure Monitor Key Resources Web Servers Instances, Processes, Hosts Application Servers Servlets, JSPs, EJBs, DB Pools 4 #3737 2003 IDC
IBM TIVOLI MONITORING STRATEGIES Traditionally, infrastructure monitoring has been implemented using a single-metric approach. An infrastructure component (or IBM Tivoli software managed endpoint) is monitored, often using a software agent installed on the infrastructure component. The monitor collects various performance and availability measures at frequent time intervals. For measurements or metrics of interest, monitored values are tested against thresholds, and actions are taken such as raising alerts, sending alarms, or providing other forms of problem notification. Limited local corrective problem fixes are performed, where possible. In traditional monitoring approaches, thresholds and responses at managed endpoints are often based on single metrics, such as server utilization or available memory space. Typically, data collected by a monitoring agent is sent to a management server for analysis and more extensive corrective actions, such as running scripts to bypass or repair a problem detected from measurement data and threshold-based exceptions. Aggregation of measurement data at the management server level permits analysis of multiple metrics gathered from a single managed endpoint, as well as analysis of data from multiple endpoints, using such techniques as event correlation and automated action-taking. Given the limited capabilities of the single-metric monitoring approach, most complex analysis and event-action responses will typically be performed at the management server level. Figure 4 illustrates the single-metric monitoring approach. FIGURE 4 SINGLE-METRIC VERSUS RESOURCE MODEL MONITORING Traditional Monitoring Resource Models Managed Endpoint! Single-metric threshold and response! Multimetric analysis! Persistence checking! Local correlation! Corrective action Management Server! Aggregation, limited corrective action, and upper-layer data feeds! Aggregation and upperlayer data feeds Management Console! Possible false alarms! Manual intervention! Capability to reduce manual intervention and improve automation 2003 IDC #3737 5
Recently, IBM Tivoli software implemented a "next-generation" monitoring approach that uses platform-specific "resource models" that enable multimetric analysis and corrective actions to be pushed down to the level of individual managed endpoints. Resource models are in essence a structure to automate "best practices" such as identifying key metrics, predefining thresholds, checking for persistent conditions, showing key relationships, and specifying corrective actions to be taken at the managed endpoint. The resource model approach also includes support for local correlation at the managed endpoint. Some resource models have been predefined by IBM Tivoli software, while others can be custom developed using the IBM Tivoli Resource Model Builder. Overall, the resource model approach means that many exceptional conditions involving multiple metrics can be automatically handled at managed endpoints, thus reducing traffic and overhead related to communicating with higher-level management layers such as the management server. A comparison of the resource model and single-metric approaches is shown in Figure 4. WEB INFRASTRUCTURE MONITORING: PUTTING THE PIECES TOGETHER IBM Tivoli Monitoring for Web Infrastructure encompasses many components that can be used individually and also in conjunction with other IBM Tivoli software infrastructure management software. In keeping with the recent evolution in the IBM Tivoli software product architecture, monitoring components can be installed using a simple, menudriven process. The idea is to deliver some of the advantages of quick-to-install "point products" while providing integration with common IBM Tivoli software enterprise management software components. The following is a summary of the major facilities: " Monitoring $ IBM Tivoli software common agent monitoring of basic infrastructure performance and availability measures, such as component utilization and operational status $ In-depth monitoring of performance metrics for specific Web infrastructure components based on APIs, such as those enabled through the WebSphere Proactive Monitoring Component (PAC) " Reporting $ IBM Tivoli Web-based, Real-time Health Console: Displays resource models installed on managed endpoints; shows selected metrics and health status for each resource $ IBM Tivoli Performance Viewer: Ad-hoc graphical display of all available performance metrics, such as those for WebSphere Application Server " Actions $ Automated problem identification, notification, and cure as defined by resource models " Integration $ Notification of exceptions and alerts sent to IBM Tivoli Enterprise Console $ Measurement data sent to IBM Tivoli Data Warehouse for reporting, historical analysis, and business intelligence 6 #3737 2003 IDC
EXAMPLE: MONITORING A WEBSPHERE APPLICATION SERVER The following examples show some of the key IBM Tivoli Monitoring facilities that are available to manage WebSphere Application Servers. The process begins with identifying the key WebSphere metrics that will be the focal point of performance and availability monitoring. The particular value in these metrics is that they can be correlated to help identify the root cause of a problem. Linking related metrics can help identify actual problem signatures, thus going beyond basic alerting of problem symptoms. As shown in Figure 5, metrics include such objects as Enterprise Beans, Web Applications, Connection Pools, Java Virtual Machines, and Transactions. FIGURE 5 IDENTIFYING KEY WEBSPHERE METRICS Enterprise Beans Num Creates Num Removes Num Activates Num Passivates Num Instantiates Num Destroys Num Loads Num Stores Concurrent Actives Concurrent Lives Total Method Calls Avg Method Rt (ms) Avg CreateTime (ms) Avg Remove Time (ms) Active Methods Gets from Pool Gets Found Returns to Pool Returns Discarded Drains from Pool Avg Drain Size Pool Size Web Applications Num Loaded Servlets Num Reloads Total Reque sts Concurrent Requests Response Time (ms) Num Errors Thread Pools Thread Creates Thread Destroys Active Threads Pool Size Percent Maxed Connection Pools Creates Num Destroys Num Returns Num Allocates Pool Size Concurrent Waiters Avg Wait Time (ms) Fa ults Percents Used Percents Maxed PrepStmt Cache Discards Java Virtual Machine Total Memory (bytes) Free Memory (bytes) Used Memory (byte s) Num Calls Avg Time Between Calls Avg Duration Num W aiters Avg Wait Time Num Objects Alloc Num Objects Freed Num Objects Moved Num Threads Started Num Threads Dead Transactions Globa l Trans Begun Global Trans Involved Local Trans Begun Active Global Trans Active Local Trans Globa l Trans Duration Local Trans Duration Global Before Completion Duration Global Prepare Duration Global Commit Duration Local Before Completion Duration Local Commit Duration Num Optimizations Global Trans Committed Local Trans Committed Global Trans RolledBack Local Trans RolledBack Globa l Trans Timeout Local Trans Timeout HTTP Sessions Created Sessions Invalidated Sessions Finalized Sessions Session Lifetime Session Invalidate Time Active Sessions Live Sessions 2003 IDC #3737 7
These metrics are then used as variables in a resource model, which is used to control the actions of WebSphere monitoring. As shown in Figure 6, the resource model is used to identify key metrics, predefine thresholds, check for persistence, show key relationships, and specify corrective actions to be taken. FIGURE 6 WEBSPHERE RESOURCE MODEL EXAMPLE WebApp Web Applications Num Loaded Servlets Num Reloads Total Reque sts Concurrent Requests Re sponse Time (ms) Num Errors Identify key metrics. Total requests, concurrent requests, Response time, num errors Predefine thresholds. Num errors greater than zero Response time exceeds 750 ms Check for persistence. Response time 9 out of 10 times Show Key Relationships. Response time graphically displayed with throughput and load Take Corrective Action. No action taken here INTEGRATION WITH IBM TIVOLI REAL-TIME HEALTH CONSOLE A key integration point for Web infrastructure monitoring is the IBM Tivoli Real-time Health Console. Information regarding the status of the monitored endpoint, such as a WebSphere Application Server, is displayed on the IBM Tivoli Real-time Health Console based on the metrics and health values defined in the resource models installed on the managed endpoint. As shown in Figure 7, the Web-based real-time view can show the health status of all resource models with red-yellow-green color coding to reflect the severity of the status information. The IBM Tivoli Health Console enables the monitored resource model health information to be displayed in an ordered, problem-solving format. The health display works to prioritize status information by forcing resources with the poorest health to "bubble up" to the top of the display so that the least healthy resources are listed first to receive the highest priority and the most immediate attention. A drill-down display (shown on the bottom half of the health console screen in Figure 7) provides more detailed status information for all the indicators within a resource model to assist in problem solving. 8 #3737 2003 IDC
FIGURE 7 IBM TIVOLI REAL-TIME HEALTH CONSOLE; WEBSPHERE APPLICATION SERVER MONITORING 2003 IDC #3737 9
CHALLENGES The facilities available in IBM Tivoli Monitoring for Web Infrastructure provide a sound basis for managing the performance and availability of J2EE-based applications servers together with associated Web servers. Incorporated in the IBM Tivoli software product architecture is the ability to share common components (such as the real-time health console) with other IBM Tivoli software monitors and also to integrate with "higher-level" IBM Tivoli software infrastructure management facilities such as crossdomain event correlation through the IBM Tivoli Event Console and business service management through post-processing of data in the IBM Tivoli Data Warehouse. IBM Tivoli software faces several challenges in positioning and marketing its products for Web infrastructure management. Of particular importance is the need to demonstrate that the "new product architecture" indeed brings such benefits as quickness and simplicity of installation, ease of operation, and user appeal similar to those of a "point product." Potential customers need to be made aware that IBM Tivoli software's well-known depth of product solution, ability to integrate, and automation of management actions and responses do not require the amount of up-front time investment by IT that has often been associated with older solutions. Such attributes as automatic discovery of infrastructure elements to be managed, automated configuration of the management software (including settings and thresholds), and automated deployment of management software components are highly desirable system management software features that can help customers to more quickly and widely deploy IBM Tivoli software monitoring solutions in operational settings. Certainly, IBM Tivoli software has taken steps in the right direction with such actions as providing preconfigured resource models to define common monitoring options and providing standard script files to help discover Web server resources. COPYRIGHT NOTICE External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2003 IDC. Reproduction without written permission is completely forbidden. 10 #3737 2003 IDC