PATROL Internet Server Manager Technical Brief
Contents Why Manage Web Applications?...1 What Is PATROL?...1 The PATROL Agent...2 PATROL Knowledge Modules...2 The PATROL Console...2 PATROL Internet Server Manager...3 Availability Monitoring...4 Performance Monitoring...5 Content Delivery Assurance...5 Advanced Monitoring and Administration...6 Web Server Log Monitoring...7 Integration with PATROL Site Monitor...7 Architecture of PATROL Internet Server Manager...8 Internet Client Library: XPC Server...8 Monitored Servers...9 Proxy Servers...9 DNS Servers: The Role of DNS Monitoring...9 Secured Servers...9 What Internet Server Administrators Need to Know...10 Are my Internet servers up and running right now?...10 What kind of response time are my servers providing? Is there a trend toward poorer response time that I should be concerned about?...11 How do increasing load rates affect my Web servers performance and availability?.. 11 What if I just don t have time to check for these errors on a regular basis?...12 I ve used tools in the past that tell me if my server process or service is up and running, but how can I tell me if my backend application servers and databases are doing what they are supposed to do?...12 If my mail servers get backlogged with messages and I don t clear out the queues quickly, my mail server goes down. How can I be proactive in this type of situation?.. 13 How can I monitor my Web server s content to find bad or broken links?...13 Helping you maintain advantage...13
Why Manage Web Applications? Companies deploy applications to the World Wide Web for two primary reasons. For many companies, the reason is strategic. Companies have transformed their businesses or built businesses around new models as a result of Internet technologies. These companies are using Web applications to drive new revenues. For others, Web applications are tactical in nature. Whether the purpose is to automate the customer-support function or to put expense reporting and the employee travel department online, the end goal is to lower costs through automation. In either case, if the Web applications are not functional or performing satisfactorily, the goal is not met. In the first case, the new business model fails if customers have a poor experience interacting with the company and choose to take their business elsewhere. In the latter case, the company does not achieve its Return of Investment (ROI) goals if employees or customers have a poor experience and choose not to use the Web application. Automation however, can cause companies to lose some of the richness of interactions with their customers. If they are not careful, they can lose touch with the quality of service they are providing. For example, in a grocery store, the manager can see when lines get too long and open another cash register to shorten the customer wait. Not monitoring the user experience of online applications would be analogous to that manager hiding in his office and counting receipts. He would have no understanding of the relationship between store revenues and the quality of service. To assure the performance and availability of Web applications, companies must deploy software to monitor and manage the servers from a software perspective, and also monitor the service they provide from a customer perspective. PATROL Internet Server Manager does this by monitoring key processes in the Internet infrastructure from DNS and mail to Microsoft Internet Information Server (IIS) and Apache. Also, PATROL Internet Server Manager monitors the quality of service these servers provide by connecting to them as users would and measuring response time and availability. What Is PATROL? PATROL is the recognized leader worldwide for automated administrative computing solutions. The PATROL Management Suite of products from BMC Software monitors and manages more applications, databases, middleware products, Internet applications and operating systems than any other solution on the market. PATROL shifts the administrative focus away from managing individual components, such as an application, database, or operating system, to an enterprise-wide view of the entire environment. PATROL operates using three main components: a PATROL Agent, PATROL Knowledge Module (KM), and a PATROL Console. Page 1
The PATROL Agent The PATROL Agent is a single process that is typically installed on all hosts that are managed by PATROL products. The PATROL Agent has the responsibility for managing all applications and resources that are known to it, and will do so without manual intervention. The PATROL Agent is both intelligent and autonomous. PATROL intelligence is the ability to have its knowledge updated while still resident in memory. That is, the agent does not need to be stopped and restarted to accept changes, additions, and deletions of information or knowledge. PATROL autonomy means that once instructed when, what, and how to do something, the PATROL Agent does not need to communicate outside of itself to get the task done. All configuration, parameters, thresholds, recovery actions, and events are stored locally in a binary file located in the PATROL directory tree. The PATROL Agent has an event management Application Program Interface (API) that is bidirectional. Events can be inserted into the agent and passed from it to other console environments, such as third-party alerting systems. Also, the PATROL Agent has the ability to communicate with any SNMP-based framework through its SNMP API. PATROL Knowledge Modules Knowledge Modules (KMs) are individual groups of knowledge libraries that contain information about a system or application that the PATROL Agent is managing. A KM is essentially a wrapper around some specific application or system resource that will be managed. Individual tasks within a KM are typically application routines, compiled programs, operating system commands, and PATROL Scripting Language (PSL) scripts. It is the PATROL Knowledge Modules that send the PATROL Agent its instructions and make it capable of managing diverse environments. As their name suggests, KMs are modular in nature, and the number and types of KMs that are loaded on an agent depend upon what services and applications you wants to manage on the system of interest. The loaded KMs provide the PATROL Agent the ability to automatically discover and manage all of the known applications, databases, and system components on the server. The agent continuously gathers information and statistics from the KMs and, when indicated, it executes a set of recovery actions. Recovery actions attempt to correct a problem, notify someone who can, or produce supporting reports that help operators to decide how to solve the problem. The PATROL Console The PATROL Console graphically depicts all applications, databases, middleware, and messaging systems, and the underlying resources that are monitored by PATROL. Page 2
The information and statistics gathered by the PATROL Agent are displayed on the PATROL Console in the form of alerts, graphs, and reports. The PATROL Agent can also accept requests from one or more consoles, initiate actions based on those requests, and it can do so without connecting to a console PATROL Console diagram The PATROL Console does not poll for events, but instead communicates with the PATROL Agent through the PATROL API. The PATROL Agent knows which connected consoles are registered for which events. PATROL Internet Server Manager PATROL Internet Server Manager provides Web server administrators and other systems administrators with a single tool that can monitor and manage all of the major types of Internet servers. When installed on an applicable system, the product discovers and monitors all Internet servers present on that system. Also, you can install PATROL Internet Server Manager onto a dedicated system in the network from which it can monitor a number of important Internet servers in the company from one central point. Out of the box, PATROL Internet Server Manager monitors all important performance metrics of discovered servers. As you progress in your management needs, you can easily configure the product to produce deeper usage statistics and perform automated recovery actions. PATROL Internet Server Manager supports all of the popular Unix operating systems on the market today, including Linux and Enterprise Linux for the IBM z/series mainframe. The product also supports Microsoft Windows NT and Windows 2000 servers. With PATROL Internet Server Manager you do not need different monitoring tools for the many Page 3
different servers in your heterogeneous environment leverage the BMC Software experience in operating systems and use this one product for all of your Internet management needs. Availability Monitoring PATROL Internet Server Manager monitors the availability of most major Internet server types. The Internet server protocols supported by PATROL Internet Server Manager are listed in the following table: Supported Internet servers and protocols Supported Server Type Web servers Secured Web servers Proxy servers FTP servers Domain Name Servers Mail servers Directory servers News and collaboration servers Chat and messaging servers Supported Protocol HTTP HTTPS HTTP and HTTPS FTP DNS SMTP, IMAP and POP3 LDAP NNTP IRC PATROL Internet Server Manager monitors the Internet servers and protocols by simulating a user accessing your site. The product connects to the monitored server using the server s native protocol, and then makes a request in the same manner as a standard client would. This ensures that the results of PATROL Internet Server Manager s monitoring always reflect actual availability of Internet servers as experienced by your users. For the supported servers listed in the preceding table, PATROL Internet Server Manager maintains several critical data parameters that allow you to quickly determine the current availability of your servers. When this product is used in combination with the PATROL Console or another similar tool, you can quickly analyze historical data on availability, which can indicate times of the day or week when failures occur or when service level agreements are not met. For each server that you monitor (locally or remotely) the product reports: Page 4
server status One stoplight-style icon provides current server availability information. server downtime This information tells you how long your server has been down so that you can track time to repair. server status code For most server types, the product reports the actual response codes returned from the monitored server, allowing you to customize recovery actions or automated notifications in a way that makes sense for your application. Performance Monitoring PATROL Internet Server Manager collects and reports the response time for all of the requests it sends to the local or remote monitored servers. The collected data is stored in the PATROL Agent history database, giving you the ability to examine how response time varies by the time of day or week and enabling you to determine whether you are meeting your service level agreements for a monitored server. Content Delivery Assurance PATROL Internet Server Manager offers an expanded depth of capabilities when monitoring your local or remote Web servers. In addition to monitoring the Web server response time and returned status code, you can configure the product to look for text patterns in the page and generate an alert if they are not present. As an alternative, you can configure the product to generate an alert if certain text is present within a returned page (such as ODBC Error ). To facilitate content delivery assurance with PATROL Internet Server Manager, you can create advanced queries that make up transactions to be tested against your local or remote Web server. You can accept and reuse cookies, post form data, and perform Web server authentication to simulate as closely as possible the actual paths that your customers take while they are working with your site. Because most Web sites consist of more than just a Web server there are usually application servers creating the content and middleware providing database connectivity, sometimes all the way back to the mainframe it is important to have a means of verifying the end-to-end availability of the interconnected components. The content checking capability of PATROL Internet Server Manager is a simple way to verify this end-to-end connectivity. By using the URL monitoring feature to make a query by posting data to a form on a Web page, you can verify the content of the returned page to ensure that the query worked as you expected. If the appropriate content is found, then you have just validated a roundtrip transaction all the way to the database. Not only did the Web server return a page in a Page 5
measured number of milliseconds, but it connected to the application server, back to the database, and it pulled the requested record, verifying end-to-end availability of the application. How many times have you encountered a SQL Error while working through a Web site on the Internet? You can configure PATROL Internet Server Manager to alert you or your database administrator (or both) when the Web server returns this particular content. Or suppose you are experiencing network issues and not all Web server data is making it to your clients. For example, the HTML pages end prematurely. You can configure the KM to look for a final footer segment that customers will see only when the complete page is delivered. If Internet Server Manager does not see this content, then you will be alerted to this failure. Advanced Monitoring and Administration PATROL Internet Server Manager offers additional monitoring and administration capabilities for a subset of the server types that it supports. Web Servers Enable PATROL Internet Server Manager s Web server log monitoring to learn detailed information about the traffic to your site, then correlate this information with response time and availability data to predict how your Web site will perform in the future. For more information, see Web Server Log Monitoring. Configure PATROL Internet Server Manager to monitor your SSL servers and to alert you before your certificates expire. Unix Sendmail PATROL Internet Server Manager can keep track of the message rates your mail server processes. Use this data with your response time and availability data to understand future capabilities of your server. Keep track of active and deferred messages in your mail queues; let PATROL Internet Server Manager alert you before these queues reach levels that lead to mail server outages. Track messages being relayed to other queuing servers. Domain Name Servers (DNS) PATROL Internet Server Manager can track of the request rates for five types of DNS requests, letting you plan future DNS server deployments. Note This applies only to BIND 4.x and 8.x on Unix. Configure the product to make user-defined DNS requests on a regular basis, then check the results for appropriateness of content and time-to-response. For more information see, DNS Servers: The Role of DNS Monitoring. Page 6
FTP Servers Configure PATROL Internet Server Manager to log on to your FTP server so that you can catch problems not seen by only checking for service availability or port scan response. Web Server Log Monitoring Sometimes knowing that your Web site has failed or is not responding rapidly is not sufficient; you want to know about the problems your customers are experiencing before they cause outages to your servers. Using PATROL Internet Server Manager to monitor your Web server s access logs, you can see when users encounter errors such as the unavoidable 500 Server Error or 404 Page Not Found. You can examine the results of Web server log monitoring in concert with availability and performance monitoring to identify actual causes of service failures, and you can often ascertain exactly which users were affected by these failures. PATROL Internet Server Manager automatically collects statistical data for common errors when you choose to activate the product s Web server log monitoring capabilities. You can also create your own expressions to detect other specific patterns in the access log, such as particular client IP addresses or certain types of URL requests. Each of the search patterns that you provide and each one that is offered out-of-the-box is called a statistic set. Each statistic set has a group of parameters that tabulate the rate (requests per second, minute, or hour), the peak rate, and the number of times a log file entry matched the search pattern. Using the agent history database provided with PATROL Agent, you can review this data at any time to analyze past performance. The product also keeps important real-time data on the top N occurrences of each statistic set. You can track up to 100 real-time instances. With top N reporting you can see the most popular pages users are accessing see the clients who are accessing your site the most track the Web browsers used to access your sites see what the current experience of 404 or 500 (or other user-defined) errors is on your site. You can use this feature to determine which pages are causing these failures and to take quick corrective action. Integration with PATROL Site Monitor Because you typically have more than one Web server that makes up your Internet site, PATROL Internet Server Manager offers you the capability to see management data from multiple servers together in a site view by using the PATROL Site Monitor component. PATROL Site Monitor takes data from the various PATROL Internet Server Manager installations in your environment and categorizes this data in a domain view. For example, Page 7
if your company s external Web site (www.companyname.com) is actually a farm of five Web servers, you can see composite domain-level data with PATROL Site Monitor. With this view, you can gain a quick understanding of how your site is performing and the availability status of your site as a whole, while being able to quickly identify individual problem spots. Architecture of PATROL Internet Server Manager PATROL Internet Server Manager acts as a normal Internet client while performing its monitoring. This ensures that the information provided to you by the product most closely reflects the actual experience of your users. PATROL Internet Server Manager uses an Internet client library called an XPC server. XPC (External Program Call) is the way the PATROL Agent communicates with external programs. Internet Client Library: XPC Server The following graphic illustrates the architecture of PATROL Internet Server Manager with respect to monitoring local and remote Internet servers. The XPC server is a process that runs persistently on your system waiting for requests from the PATROL Agent. These requests are initiated by PSL processes, which form the backbone of all PATROL monitoring. The PSL processes tell the XPC server to request an Internet resource and wait for the response. The XPC server makes an actual client connection to the monitored resource and returns the raw data from the monitored server. The PSL process then interprets this data and posts the results into the PATROL Agent namespace. You then use a PATROL Console or other event manager to access these results. Internet client library architecture Web Server KM Instance PSL Process Mail Server KM Instance DNS Server KM Instance PATROL Agent XPC Server Client.xpc NT or Unix Process HTTP(S) SMTP DNS Web Server Mail Server DNS Server PSL Process PSL Process Page 8
Monitored Servers Because PATROL Internet Server Manager acts just as a normal Internet client would, the product can monitor any type of server that adheres to the standard protocol for that server type. If all you need is response time and availability information from these servers, you can take advantage of PATROL Internet Server Manager s lightweight XPC server to monitor many different servers on your network from a single management node. This mode of operation can offer quicker time to value, because you have to install the product on only one system. Proxy Servers PATROL Internet Server Manager operates seamlessly in your existing environment, using proxy servers where necessary to give you the greatest possible flexibility in remote monitoring. When you configure the product to monitor a remote host, you have the option of routing the request through a proxy server, including adding a user name and password for authentication. PATROL Internet Server Manager also monitors proxy servers directly. You can install the product on the proxy server itself to take advantage of recovery actions such as automated restart, or you can monitor the proxy server remotely. DNS Servers: The Role of DNS Monitoring PATROL Internet Server Manager offers monitoring capabilities that enable DNS administrators to manage their DNS servers at a very granular level. Also, PATROL Internet Server Manager automatically monitors any DNS servers your Web server is configured to use. All Internet servers depend on Domain Name Service lookups for various tasks. For example, your Web server may contact database servers or Web application servers residing inside your firewall to service user requests. You will want to make sure these DNS lookups are responding appropriately and in a timely fashion. You can also create specific queries to perform against specific DNS servers, so you can tailor your monitoring to make sure the queries you need are responding correctly, with the proper content. Secured Servers PATROL Internet Server Manager can communicate with your secured Web servers by using the same SSL technology as your users Web browsers. The product supports the latest encryption levels, including 128-bit encryption, subject to export restrictions. This product also reports the number of valid days remaining on your SSL server s certificate, Page 9
warning you when the expiration date approaches. This will help you prevent the costly embarrassment of an expired SSL certificate when users attempt to access an application on your secure Web server. What Internet Server Administrators Need to Know The questions that follow are often asked by those responsible for Internet servers on a daily basis and these are questions that PATROL Internet Server Manager can help you answer. Some of the questions are followed by a list of parameters, in the product, that provide the needed information.with these parameters, you can view current, real-time status or you can use the PATROL Agent history to view past data. Use the PATROL Console to plot the data points from multiple data sources into a single graph, enabling you to gain a quick comprehensive understanding of the health of your servers. Are my Internet servers up and running right now? Use PATROL Internet Server Manager Status Parameters Relevant product parameters: httpstatus nntpstatus imapstatus smtpstatus pop3status ldapstatus dnsstatus ftpstatus ircstatus Think of these parameters as your one-stop shop when you need to understand whether specific Internet servers are up and running. You can examine this data at the server level in PATROL Internet Server Manager, or at the site level with PATROL Site Monitor. When viewed in the PATROL Console, these parameters appear as stoplights, giving you a quick visual understanding of your site s status. This data also triggers exception events that allow for lights-out management when used with PATROL Enterprise Manager or other event management consoles. Page 10
What kind of response time are my servers providing? Is there a trend toward poorer response time that I should be concerned about? Use PATROL Internet Server Manager Response-time Parameters Relevant product parameters: httpresponsetime nntpresponsetime imapresponsetime smtpresponsetime pop3responsetime ldapresponsetime dnsresponsetime ftpresponsetime ircresponsetime These parameters give you a highly granular view of response-time measurements for your monitored servers. Because PATROL Internet Server Manager monitors your servers by making actual connections to them in the same way as your customers do, the data reported here is of the utmost value to you in understanding customer experience. Since the data is measured and reported to you in milliseconds, you can detect minute differences in response time trends. You also will be able to compare response-times of individual servers that make up a server farm, adjusting the performance of those servers that need it until all servers are performing optimally. How do increasing load rates affect my Web servers performance and availability? Correlate the status and response-time parameters listed above with the data reported by the Web server log monitoring components. As a Web server administrator, your primary job is not to respond to and correct Web server failures, but to prevent Web server failures. Because PATROL Internet Server Manager collects performance and availability data for your servers both from the client perspective and by analyzing log files, you can match trends in server response time with trends in server load. For example, you can place the httpresponsetime parameter for one of your Web servers into the same graph as the requestratepermin (per-minute rate of requests processed by the Web server) parameter to see how increasing loads on your Web server affect overall customer experience in response-time changes. Using the PATROL Agent history, you can examine historical data to predict how increased load over time will affect server response. Use this data to plan new Web server deployments so that you will not be caught off-guard by unexpected performance problems. Page 11
What if I just don t have time to check for these errors on a regular basis? Activate PATROL Internet Server Manager s Web Server Error Log Reporting As a busy Web server administrator in charge of dozens of Web servers you do not have the time to check each one for error messages that can indicate problems that might occur in the future. Set up PATROL Internet Server Manager s Web server error log reporting to alert you when error conditions of your definition occur, immediately as they occur. This product is preconfigured to understand the typical error events that can occur with Netscape/iPlanet and Apache servers, so in most cases you will not need to configure the product to activate this real-time monitoring. In many customer environments you may be aware of specific non-standard error messages that eventually lead to outages at your site. With PATROL Internet Server Manager, you can easily configure the product to look for your choice of these errors and to alert you when they occur. You can view this information in the PATROL Console or use the product s event triggers to alert operations staff as these critical errors occur. I ve used tools in the past that tell me if my server process or service is up and running, but how can I tell me if my backend application servers and databases are doing what they are supposed to do? Use PATROL Internet Server Manager s Web Server Content Checking Features When it s not enough to know if your server is responding to basic client requests, configure PATROL Internet Server Manager to monitor those pages on your site that require dynamic content from an application server or database. Then configure the product to look for specific text or text patterns that must or must not be found in the final response. For example, you can tell the product to look for the phrase ODBC error and to alarm when this condition occurs. Without this type of monitoring, you may not be aware of these failures until one of your customers calls you to complain, and then it is too late the damage to your reputation is done. With PATROL Internet Server Manager, you can learn of these problems before they affect a wide number of your end users. Page 12
If my mail servers get backlogged with messages and I don t clear out the queues quickly, my mail server goes down. How can I be proactive in this type of situation? Monitor your mail server s queue depth PATROL Internet Server Manager can examine the message queue for your Sendmailbased mail servers and regularly report to you the number of active and deferred messages in the queue. You decide the acceptable level of queued messages for your environment and then let PATROL Internet Server Manager notify you when these levels are reached. In this way, you can take corrective action on your queues before your mail servers fail and users call you to complain. If your internal mail server receives messages for another mail server that is prone to failures in your site, you can configure PATROL Internet Server Manager to report the number of deferred messages bound for that troublesome queue on a regular basis. Using the PATROL Agent s history and reporting capabilities, you can gain a quick understanding of how many messages are processed for that remote mail server. How can I monitor my Web server s content to find bad or broken links? Use PATROL Internet Server Manager Web Server Link Checking Features You can configure the link checking capability of PATROL Internet Server Manager to traverse some or all links on the designated Web site and report which links are bad. Helping you maintain advantage BMC Software Professional Services helps your company maintain its competitive advantage through a comprehensive suite of services that includes service level management consulting, installation, implementation, configuration, and customization. Our professional services and education offerings are designed to ensure the ongoing availability of critical business applications, maximize product potential, reduce project risk, deliver IT value to your business, and improve your operations. For more information about BMC Software Professional Services, visit http://www.bmc.com/profserv. Page 13
About BMC Software BMC Software, Inc. [NYSE: BMC], is the leading provider of enterprise management solutions. The company focuses on Assuring Business Availability for its customers by helping them proactively improve service, reduce costs and increase value to their business. BMC Software solutions span enterprise systems, applications and databases. Founded in 1980, BMC Software has offices worldwide and is a member of the S&P 500, with fiscal year 2002 revenues of approximately $1.3 billion. Visit http://www.bmc.com to learn more. Page 14
For more information visit BMC Software on the Web at www.bmc.com BMC Software, the BMC Software logos and all other BMC Software product or service names are registered trademarks or trademarks of BMC Software, Inc. All other registered trademarks or trademarks belong to their respective companies. Copyright 2002, BMC Software, Inc. All rights reserved. August 30, 2002 19599