Data Center Real User Monitoring

Transcription

1 Data Center Real User Monitoring Web Application Monitoring User Guide Release 12.3

2 Please direct questions about Data Center Real User Monitoring or comments on this document to: Customer Support Copyright 2014 Compuware Corporation. All rights reserved. Unpublished rights reserved under the Copyright Laws of the United States. U.S. GOVERNMENT RIGHTS-Use, duplication, or disclosure by the U.S. Government is subject to restrictions as set forth in Compuware Corporation license agreement and as provided in DFARS (a) and (a) (1995), DFARS (c)(1)(ii) (OCT 1988), FAR (a) (1995), FAR , or FAR (ALT III), as applicable. Compuware Corporation. This product contains confidential information and trade secrets of Compuware Corporation. Disclosure is prohibited without the prior express written permission of Compuware Corporation. Use of this product is subject to the terms and conditions of the user's License Agreement with Compuware Corporation. Documentation may only be reproduced by Licensee for internal use. The content of this document may not be altered, modified or changed without the express written consent of Compuware Corporation. Compuware Corporation may change the content specified herein at any time, with or without notice. All current Compuware Corporation product documentation can be found at Compuware, FrontLine, Network Monitoring, Enterprise Synthetic, Server Monitoring, Dynatrace Network Analyzer, Dynatrace, VantageView, Dynatrace, Real-User Monitoring First Mile, and Dynatrace Performance Network are trademarks or registered trademarks of Compuware Corporation. Cisco is a trademark or registered trademark of Cisco Systems, Inc. Internet Explorer, Outlook, SQL Server, Windows, Windows Server, and Windows Vista are trademarks or registered trademarks of Microsoft Corporation. Firefox is a trademark or registered trademark of Mozilla Foundation. Red Hat and Red Hat Enterprise Linux are trademarks or registered trademarks of Red Hat, Inc. J2EE, Java, and JRE are trademarks or registered trademarks of Oracle Corporation. VMware is a trademark or registered trademark of VMware, Inc. SAP and SAP R/3 are trademarks or registered trademarks of SAP AG. Adobe Reader is a registered trademark of Adobe Systems Incorporated in the United States and/or other countries. All other company and product names are trademarks or registered trademarks of their respective owners. Local Build: December 8, 2014, 14:22

3 Contents Contents Introduction Who Should Read This Guide Organization of the Guide Related Publications Customer Support Information Reporting a Problem Documentation Conventions Chapter 1 Web Application Monitoring Configuration Process Overview Chapter 2 Adding Basic DC RUM Devices Adding an AMD to Devices List Adding a CAS to Devices List Adding ADS to Devices List Chapter 3 Verification of Traffic Monitoring Quality Sniffing Point Diagnostics Sniffing Point Diagnostics Reports Network Interface General Statistics Network and Transport Protocol Information Services Detected in the Traffic Session-Related Statistics SSL Diagnostics Application Overview Using RUM Console to Identify Problems Related to Network Hardware Operation... Chapter 4 Basic Monitoring Configuration Configuring General Data Collector Settings Configuring Operation-Related Global Settings Operation Time in Web Monitoring Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Defining Software Services

4 Contents Capturing Traffic Traces Manual Upload of Traffic Traces Application Traffic Categories Undecrypted SSL Traffic Selecting Services for Software Service Definition Configuring URLs for a Software Service Definition Using Wildcards in URLs User Name Recognition Configuration Overview of User Name Recognition Configuration in HTTP Mode Choosing Search Scope for User Identification Choosing Method of Searching for User Identification Overview of User Name Recognition Configuration in HTTP Legacy Mode Configuring User Recognition Method Based on HTTP POST Configuring User Recognition Method Based on HTTP GET Configuring User Recognition Method Based on Cookie Configuring User Recognition Method Based on Session Cookie Other Methods of Configuring User Name Recognition Using Regular Expressions to Extract User Identification Reviewing and Publishing a Software Service Definition Managing Software Service Definitions Defining Software Services Manually Chapter 6 Configuration Fine-Tuning Managing Devices Adding Devices in RUM Console Editing Device Connection Parameters Deleting a Device from the Devices List Integrating DC RUM with Dynatrace Application Monitoring Configuring Dynatrace Application Monitoring to Receive Performance Data from DC RUM Configuring the DPN Connection in RUM Console URL Auto-Learning Configuring URL Auto-Learning Details of the URL Auto-Learning Algorithm URL Auto-Learning Diagnostics Dimensions, Metrics and Attributes in HTTP Monitoring Excluding Elements from Orphaned Redirects Reporting Synthetic Agent and Browser Recognition Configuring Synthetic Agents, Browsers, Operating System and Hardware Recognition Synthetic Agent Recognition Based on Contents of HTTP Header Synthetic Agent Recognition Based on User Name or IP Address End-of-Page Components Automatic Page Name Recognition Reporting of URLs with Redirects Content Type URL Monitoring Monitoring of Non-HTML Objects Based on Content Type

5 Contents Logging Transactions, ADS Data and ADS Header Data Masking of Sensitive HTTP Information Character Encoding Support for HTTP Services Rule-based Character Encoding for HTTP Services Assigning HTTP Error Codes to Error Categories Managing SSL Alert Codes Defining SSL Error Names Excluding IP Ranges from AMD Client Analysis Importing and Managing User ID Mapping Information General Configuration Options for HTTP-Related Analyzers Choosing HTTP Analyzer Mode Global Settings for Recognition and Parsing of URLs Global Settings for Page and Session Recognition Based on Cookies Global Settings for Client IP Address Extraction Assembling Pages Multi-Frame Pages Calculating Server Time for Multi-Frame Pages Calculating Availability Configuring HTTP Availability HTTP Configuration Options for Selected Autodiscovered Software Services Additional Configuration Options for HTTP and SSL Software Services HTTP Express Analyzer General Configuration Options for HTTP Express Software Services Configuring HTTP Express Availability Configuring User-Defined Software Services Based on HTTP Express Analyzer... Configuring URL Monitoring for HTTP Express Analyzer Configuring Monitoring of URL Parameters for HTTP Express Analyzer Chapter 7 Monitoring Sequence Transactions Configuring the AMD to Monitor HTTP-Based Transactions Adding Transactions Adding Transactions for a Range of AMDs Filters and Transaction Inspector for HTTP Transactions Modifying, Deleting, and Cloning Transactions for a Single AMD Using Correlation ID to Monitor Asynchronous HTTP Transactions Chapter 8 Web Tiers Chapter 9 Web Application Traffic on CAS Reports Appendix A Diagnostics and Troubleshooting Guided Configuration Issues Troubleshooting SSL Monitoring Issues Report-Related Issues Modifying Connection Settings for Guided Configuration Connection Settings for CBA and CBA Agent Connection Settings for the CBA Agent and RUM Console Server SSL Settings for the CBA Agent and RUM Console Server Connection

6 Contents Appendix B Regular Expression Fundamentals Testing Regular Expressions Best Practices for Regular Expressions Appendix C Classification of Aborts Glossary Index

7 INTRODUCTION Who Should Read This Guide This manual is intended for users of DC RUM who want to configure monitoring of their web-based applications. Organization of the Guide This guide is organized as follows: Configuration Process Overview [p. 11] Guides you through the process of configuring monitoring of web-based applications, explains how to view monitoring results on the reports and how to troubleshoot issues related to HTTP or HTTPS monitoring. Adding Basic DC RUM Devices [p. 19] Describes how to add a monitoring device and a report server using RUM Console. Verification of Traffic Monitoring Quality [p. 25] Describes how to verify sniffing points traffic detection quality before the actual monitoring begins. Basic Monitoring Configuration [p. 37] Describes general settings related to the AMD, like the monitoring interval or the operation time. These settings apply to all the analyzers. Configuring AMD to Monitor HTTP and HTTPS Traffic [p. 47] Explains how to work with the Guided Configuration perspective in the RUM Console, how to record a traffic trace and how to define a new software service. Configuration Fine-Tuning [p. 81] Describes additional aspects of HTTP or HTTPS monitoring, for example URL auto-learning, synthetic agent recognition, operation attributes, custom metrics or SSL errors. Monitoring Sequence Transactions [p. 171] 7

8 Introduction Explains how to configure monitoring of sequence transactions. Web Tiers [p. 183] Lists the tiers showing the HTTP and HTTP traffic data. Web Application Traffic on CAS Reports [p. 185] Lists the DC RUM reports that show HTTP and HTTP traffic data. Diagnostics and Troubleshooting [p. 187] Lists most common configuration or reporting problems related to HTTP and HTTPS. Regular Expression Fundamentals [p. 211] Provides an overview of how to use regular expressions. Classification of Aborts [p. 217] Describes the 4 main categories of the transactions for which there was no HTTP server response detected or which were aborted after the HTTP server responded with an HTTP header. Related Publications Documentation for your product is distributed on the product media. For Data Center RUM, it is located in the \Documentation directory. It can also be accessed from the Media Browser. Go online ( for fast access to information about your Dynatrace products. You can download documentation and FAQs as well as browse, ask questions and get answers on user forums (requires subscription). The first time you access FrontLine, you are required to register and obtain a password. Registration is free. PDF files can be viewed with Adobe Reader version 7 or later. If you do not have the Reader application installed, you can download the setup file from the Adobe Web site at Customer Support Information Dynatrace Community For product information, go to and click Support. You can review frequently asked questions, access the training resources in the APM University, and post a question or comment to the product forums. You must register and log in to access the Community. Corporate Website To access the corporate website, go to The Dynatrace site provides a variety of product and support information. Reporting a Problem Use these guidelines when contacting APM Customer Support. 8

9 Introduction When submitting a problem, log on to the Dynatrace Support Portal at click the Open Ticket button and select Data Center Real User Monitoring from the Product list. Refer to the DC RUM FAQ article at to learn know how to provide accurate diagnostics data for your DC RUM components. Most of the required data can be retrieved using RUM Console. Documentation Conventions The following font conventions are used throughout documentation: This font Bold Citation Documentation Conventions [p. 9] Fixed width Fixed width bold Fixed width italic Indicates Terms, commands, and references to names of screen controls and user interface elements. Emphasized text, inline citations, titles of external books or articles. Links to Internet resources and linked references to titles in documentation. Cited contents of text files, inline examples of code, command line inputs or system outputs. Also file and path names. User input in console commands. Place holders for values of strings, for example as in the command: cd directory_name Menu Item Screen Menu items. Text screen shots. Code block Blocks of code or fragments of text files. 9

10 Introduction 10

11 CHAPTER 1 Web Application Monitoring Data Center Real User Monitoring enables you to monitor the performance of your web applications. It can be configured to monitor specific URLs or ranges of URLs, or to automatically recognize and analyze the URLs that appear most frequently. You can also configure many other aspects of web application monitoring such as user recognition, HTTP and SSL error groups, object and frame recognition, and synthetic agent recognition. After you install DC RUM, use the RUM Console to determine a configuration that best addresses your monitoring needs. There are two versions of the HTTP analyzer for monitoring traffic: HTTP Analyzer This is the standard version. It is capable of in-depth performance analysis and is recommended for monitoring application performance. HTTP Express This is a light version. With it, you can create a simple software service used to monitor URLs. It provides only basic HTTP analysis limited to hit identification and per-url monitoring, making it suitable for traffic categorization. It is not recommended for application performance monitoring. Configuration Process Overview Before monitoring web applications, you must configure devices, and create and modify monitoring rules. Before You Begin You should be familiar with DC RUM components and basic monitoring concepts. Refer to the Data Center Real User Monitoring Getting Started. You need to identify your monitoring goals. For more information, see Define and Prioritize Goals, Objectives, and Requirements in the Data Center Real User Monitoring Getting Started. You need to install the following DC RUM components: The latest version of AMD 11

12 Chapter 1 Web Application Monitoring Refer to the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. The latest version of RUM Console Refer to the Data Center Real User Monitoring RUM Console Installation Guide. The latest version of CAS Refer to the Data Center Real User Monitoring Central Analysis Server Installation Guide. Optionally: The latest version of ADS Refer to the Data Center Real User Monitoring Advanced Diagnostics Server Installation Guide. Make sure that default ports are available for communications between the individual DC RUM components. For more information, see Network Ports Opened for DC RUM in the Data Center Real User Monitoring Administration Guide. If you plan to monitor SSL traffic, first you need to configure SSL decryption. Refer to the Data Center Real User Monitoring SSL Monitoring Administration Guide. The following steps must be executed in order to begin monitoring the traffic using the DC RUM suite: Configuring Devices 1. Adding an AMD AMD is the main data source (Data Collector) for DC RUM; it collects and presents the monitored data to DC RUM report servers for analysis and reporting. You need to add at least one AMD to the list of devices in the RUM Console. For more information, see Adding an AMD to Devices List [p. 19]. 2. Adding a CAS The CAS is the main report server for DC RUM. It uses the data provided by the AMD and its monitoring and alerting mechanisms identify, track, and report on issues affecting the security, performance, and reliability of your services. Add at least one CAS to the device list and configure its connection with the AMD. Adding a report server to a list of devices is similar to adding an AMD. For more information, see Adding a CAS to Devices List [p. 21]. 3. Verify the traffic monitoring quality and completeness You can verify traffic quality and completeness before the actual monitoring begins. Sniffing point diagnostics allows you to perform pre-monitoring tasks without the need of accessing the AMD console and executing a series of Linux commands which usually serve the purpose of validating AMD physical installation and connection. For more information, see Verification of Traffic Monitoring Quality [p. 25]. Configuring Basic Monitoring 4. Configure general settings for your AMD 12

13 Chapter 1 Web Application Monitoring Before you proceed to detailed monitoring rules, you need to define the global settings that are applied to all software services monitored by a given AMD. These global settings include a monitoring interval and thresholds for the basic metrics. These settings can be overridden at a later time with more specific monitoring rules that you can define. For more information, see Configuring General Data Collector Settings [p. 37] and Configuring Operation-Related Global Settings [p. 41]. 5. Configure support for WAN optimization All AMDs that monitor network traffic using WAN optimization must be configured before DC RUM can automatically recognize the optimized WAN traffic. Otherwise, the measurements collected from such an environment will be incorrect. Refer to the Data Center Real User Monitoring WAN Optimization Getting Started. Customizing monitoring rules 6. Recording a traffic trace The RUM Console enables you to capture traffic and use the saved traffic data to search for servers, URLs, and users. For more information, see Capturing Traffic Traces [p. 50]. 7. Defining a new software service You can define a new software service using the application traffic statistics as filters, or define it manually, providing all the required values yourself. For more information, see Defining Software Services [p. 47] and Defining Software Services Manually [p. 79]. 8. Displaying the reports to review statistics for monitored traffic. Determining the best possible configuration for your needs may be an iterative process, where you adjust the configuration incrementally after viewing your report results. Fine-Tuning the Monitoring Configuration You can further fine-tune the monitoring configuration by specifying additional conditions or setting up specialized tools for analyzing and differentiating the traffic. For HTTP traffic, for example, you can limit monitoring to specific URLs or you can select URLs based on the frequency with which they appear in the monitored traffic. 9. Adding more Data Collectors in your configuration or editing the configuration of the existing ones In addition to the AMD, there are other Data Collectors that can be employed to gather information about the HTTP traffic: Real-User Monitoring First Mile, Enterprise Synthetic, and other CAS instances. Depending on the Data Collector type, the monitoring scope may be different. For more information, see Managing Devices [p. 81]. 10. Configuring a DPN connection in the RUM Console This step is required if you plan to have integrated monitoring of your web applications traffic with DC RUM and DPN. For more information, see Configuring the DPN Connection in RUM Console [p. 87]. 11. Configuring additional aspects of HTTP monitoring URL auto-learning 13

14 Chapter 1 Web Application Monitoring URL auto-learning enables you to define a set of URLs appearing in per-url reporting statistics, without the need to manually define each URL. Frequently found URLs are learned and reported. For more information, see URL Auto-Learning [p. 88]. Custom metrics You can define up to five custom metrics to cover certain types of measurable data that are specific to your network environment or software. Use this mechanism if you want to obtain non-standard measurements extracted from HTTP traffic. For more information, see Custom Metrics in HTTP Monitoring in the RUM Console Online Help. Extraction of operation attributes The AMD analyzes and categorizes the operation attributes that are text entities retrieved from both requests and responses of a web application operation, enabling DC RUM to diagnose and report on specific events or errors caused by end-user actions. For more information, see Operation Attributes in HTTP Monitoring in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Extraction of HTTP miscellaneous parameters Miscellaneous parameters are text strings available in the URL request or response body. You need to define recognizable text patterns conveying the miscellaneous parameters that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. Miscellaneous parameters, unlike parameters extracted for URLs with parameters, are not defined together with an accompanying URL. When extracting Miscellaneous parameters, only the initial hit triggering the web page load is taken into account. The extracted Miscellaneous parameters must not be longer than 1030 bytes. For more information, see Extracting Miscellaneous Parameters in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Extraction of HTTP grouping attributes Grouping attributes are text strings available in the URL request or response body that uniquely identify clients. You need to define recognizable text patterns conveying the grouping attributes that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. For more information, see Extracting Grouping Attributes in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Excluding elements from orphaned redirects reporting Configure this functionality if you need to prevent some elements from being reported as orphaned redirects. For more information, see Excluding Elements from Orphaned Redirects Reporting [p. 99]. Synthetic agent and browser recognition A synthetic agent is a simulator of user traffic to a given website, and synthetic agent traffic is recognized and treated differently than real user traffic. For more information, see Synthetic Agent and Browser Recognition [p. 100]. Character encoding support 14

15 Chapter 1 Web Application Monitoring Enabling internationalization for HTTP services makes it possible to recognize the character encoding of HTTP content. For more information, see Character Encoding Support for HTTP Services [p. 122]. End-of-page components For each software service, URL, or URL with parameters, you can define an end-of-page component identified by a URL, the loading of which indicates that the page is complete and no further elements are taken into account when calculating metrics for the operation. For more information, see End-of-Page Components [p. 104]. Automatic page name recognition URL strings appearing on reports can be very long and difficult to read. You can specify URL names to use instead. You can either add a static page name or configure the AMD to retrieve the page name automatically from the HTML body of the HTTP response page. For more information, see Automatic Page Name Recognition [p. 106]. Content type URL monitoring When defining URL monitoring, some definition criteria may cover content that is not interesting to you (for example, the binary content). To exclude it from URL-based software service monitoring, you can narrow the monitoring to selected content types only (for example, text/html or text/xml). For more information, see Content Type URL Monitoring [p. 115]. Monitoring of non-html objects based on content type AMD can be configured to treat types of objects as HTML pages to be monitored (for example, images, Flash objects, or objects that require third-party plug-ins to render). For more information, see Monitoring of Non-HTML Objects Based on Content Type [p. 116]. Assigning HTTP error codes to error categories You can configure the contents of the HTTP error categories on the AMDs that will feed your report server. The settings are global, which means that they apply to all analyzers reporting information on HTTP errors: HTTP, Oracle Forms, XML, and SOAP. For more information, see Assigning HTTP Error Codes to Error Categories [p. 124]. Data generation for transactions, ADS data and ADS header data If you are interested only in obtaining information based on aggregated monitoring data on your reports, and not in per-url data, you can globally disable data generation for transactions and ADS data and ADS header data in the RUM Console. For more information, see Logging Transactions, ADS Data and ADS Header Data [p. 118]. Monitoring HTTP Multi-Frame Pages It is possible to recognize framesets as single pages. The association between frames and their contents can be performed dynamically, by analyzing the HTML tags, or statically, by explicitly defining the framesets. For more information, see Multi-Frame Pages [p. 143]. HTTP general configuration options 15

16 Chapter 1 Web Application Monitoring Global settings are settings that affect monitoring of all of the services based on the HTTP or HTTPS analyzer for a given Data Collector. The default values provided should be sufficient for most purposes, so it may not be necessary to change them for your initial monitoring activities. You can review and modify them at this stage, or you can leave the default values and then adjust them after you have generated some reports and have identified the areas to be changed. For more information, see General Configuration Options for HTTP-Related Analyzers [p. 133], Global Settings for Page and Session Recognition Based on Cookies [p. 139], and Global Settings for Recognition and Parsing of URLs [p. 137]. Page assembly options Page assembly options relate to the methods of assigning individual hits to pages (assembling pages from a number hits). For more information, see Assembling Pages [p. 141]. Additional HTTP and SSL configuration options For more information, see Additional Configuration Options for HTTP and SSL Software Services [p. 154]. Fine-Tuning Reporting Configuration 12. Configure the sites, areas, and regions A site is a term for a group of users that are located in the same IP network or group of networks sharing similar routing properties. Sites can be grouped together into areas, which, in turn, can be grouped together into regions. The hierarchy of sites, areas, and regions provides an organized view of the monitored network on the reports. For more information, see Configuring Sites, Areas, and Regions in the Data Center Real User Monitoring Administration Guide. 13. Configure the transactions, applications, and reporting groups Transactions are sequences of information exchange that represent particular actions or functions performed by a human user or a client program. They are viewed as higher-level units of self-contained functionality and are tied to applications. For example, they may represent the procedure for an online purchase or ticket booking. AMD monitors traffic data and prepares it for transaction monitoring by an ADS and CAS. Some of the relevant configuration and processing is performed on the actual RUM Console and some is performed on the AMD. For more information, see Managing Business Units in the Data Center Real User Monitoring Administration Guide. 14. Configure the monitoring of sequence transactions DC RUM enables you to define and monitor transactions that are sequences of steps. For example, adding a product to a cart or selecting payment method could be one of the steps in the purchase transaction. For more information, see Monitoring Sequence Transactions [p. 171]. 15. Configuring tiers 16

17 Chapter 1 Web Application Monitoring A tier is a point where DC RUM collects performance data. It is a logical application layer, a representation of a fragment of your monitored environment. For more information, see Web Tiers [p. 183]. Troubleshooting 16. Troubleshoot problems You can review the answers to the most common questions and troubleshoot your setup and report configurations. For more information, see Diagnostics and Troubleshooting [p. 187]. 17

18 Chapter 1 Web Application Monitoring 18

19 CHAPTER 2 Adding Basic DC RUM Devices In a DC RUM configuration, there are two device types: data collectors and report servers. To start using the product, add and configure at least one AMD data collector and one CAS report server. You manage these devices using a configuration tool called the RUM Console. Adding an AMD to Devices List Before you can monitor traffic with DC RUM, you have to use the RUM Console to add and configure an AMD. Adding an AMD 1. Start and log on to RUM Console. 2. Select Devices and Connections Add device from the top menu. The Add Device pop-up window appears. 3. From the Device type list, select AMD. 4. In the Description box, type a description of the device. TIP It is recommended that you include the parent device name in the description of each device you add and to add these names consistently. This enables you to easily find your device in the list. Specifying the Connection Information 5. In the Device IP address box, type the device IP address. 6. In the Port number box, type the port number for communication with this device. The default port number for the communication with the AMD is Select Use secure connection if communication with this device should occur via a secure HTTP protocol. 19

20 Chapter 2 Adding Basic DC RUM Devices Providing the Authentication Details 8. Type the user name and password of the account to be used for managing this device. By default, the AMD user is set to compuware and the password is set to vantage. The credentials you enter here are used by the RUM Console to communicate with the device and are also passed to report servers so they can collect monitoring data for processing. The values used here for authentication are not equal to the values you use for logging in to the device via SSH or local console. Configuring Advanced Settings 9. Select the Advanced options tab. 10. Optional: Under Secondary device connection, provide an alternative IP address for this device. 11. Optional: Enable SNMP connection. Optionally, you can define the SNMP connection parameters so that you can obtain more detailed health information about the device. To define SNMP connection parameters: a. Select SNMP Connection check box. b. Type the read community name and port number. 12. Enable Guided Configuration. By default, the Guided Configuration connection is enabled when you add an AMD. However, for performance reasons, the number of AMDs with enabled Guided Configuration is limited to 50. Any additional AMDs do not feed data to the Guided Configuration perspective. This means that the monitoring data from the additional AMDs is not available for generating the web traffic statistics or defining the web software services with a wizard. By default, the port number for communication between the Console Basic Analyzer Agent and the RUM Console Server is set to 9094 and the secure connection is enabled. In most cases, it is not necessary to modify this setting. If the default port number is already in use by other services, however, type the new port number in the Port number box. In this case, you also have to manually change the port number setting on the Console Basic Analyzer Agent side. For more information, see Modifying Connection Settings for Guided Configuration [p. 206]. 13. Click Next to test your connection parameters. If your configuration fails the test, you can go back and adjust your settings. Note that if the device fails to respond correctly, it may take several seconds before the test times out. 14. Click Finish to save the configuration. As a result, your device appears on the Devices list. To view the list, go to Devices and Connections Manage Devices in the top menu of the RUM Console. The Devices screen presents a comprehensive view of all the devices that you add, including their IP Address, Port, Description, Type, Version, Connection, Hardware Health, and Configuration. 20

21 Chapter 2 Adding Basic DC RUM Devices Adding a CAS to Devices List To view reports based on the data from the AMD, use the RUM Console to add and configure a CAS report server. Adding a CAS 1. Start and log on to RUM Console. 2. Select Devices and Connections Add device from the top menu. The Add Device pop-up window appears. 3. From the Device type menu, select CAS. 4. In the Description box, type a description of the device. TIP It is recommended that you include the parent device name in the description of each device you add and to add these names consistently. This enables you to easily find your device in the list. Specifying the Connection Details 5. In the Device IP address box, type the device IP address. 6. In the Port box, type the port number for communicating with this device. The standard port number used by the CAS when communicating over HTTP is Select Use secure connection if you want to use HTTPS (secure HTTP) for communication between the console and the device you are adding. Providing the Authentication Details 8. Choose whether authentication should occur via CSS. 9. Type the user name and password of the account that will be used for managing this device. Configuring the Advanced Settings 10. Select the Advanced options tab. 11. Optional: Under Secondary device connection, provide an alternative IP address for this device. 12. Click Next to test your connection parameters. If your configuration fails the test, you can go back and adjust your settings. Note that if the device fails to respond correctly, it may take several seconds before the test times out. 13. Click Finish to save the configuration. Configuring the CAS-AMD Connection 14. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 15. Select a report server from the list of devices. Click the server once to display the detailed information for the device. 16. Select the Data Sources tab. 21

22 Chapter 2 Adding Basic DC RUM Devices 17. Click Add Data Source. 18. Select your AMD from the list and then click the button. 19. Click Finish to save the configuration. As a result, your device appears on the Devices list. To view the list, go to Devices and Connections Manage Devices in the top menu of the RUM Console. The Devices screen presents a comprehensive view of all the devices that you add, including their IP Address, Port, Description, Type, Version, Connection, Hardware Health, and Configuration. What to Do Next It is important to keep the devices synchronized to avoid improper data interpretation. For more information, see Synchronizing Time Using the NTP Server in the Data Center Real User Monitoring Smart Packet Capture User Guide and Time Synchronization Between AMD and Server in the Data Center Real User Monitoring Administration Guide. Adding ADS to Devices List To view reports based on data from the AMD, use the RUM Console to add and configure at least one CAS report server. In addition, you can add one or more ADS report servers in a farm configuration. Adding an ADS 1. Start and log on to RUM Console. 2. Select Devices and Connections Add device from the top menu. The Add Device pop-up window appears. 3. From the Device type menu, select ADS. 4. In the Description box, type a description of the device. TIP It is recommended that you include the parent device name in the description of each device you add and to add these names consistently. This enables you to easily find your device in the list. Specifying the Connection Details 5. In the Device IP address box, type the device IP address. 6. In the Port number box, type the port number for communication with this device. The standard port number used by ADS when communicating over HTTP is Optional: Select Use secure connection if you want to use HTTPS (secure HTTP) for communication between the console and the device you are adding. Providing the Authentication Details 8. Choose whether authentication should occur via CSS. 22

23 Chapter 2 Adding Basic DC RUM Devices 9. Type the user name and password of the account that will be used for managing this device. Configuring the Advanced Settings 10. Select the Advanced options tab. 11. Optional: Under Secondary device connection, provide an alternative IP address for this device. 12. Click Next to test your connection parameters. If your configuration fails the test, you can go back and adjust your settings. Note that if the device fails to respond correctly, it may take several seconds before the test times out. 13. Click Finish to save the configuration. Configuring the ADS-AMD Connection 14. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 15. Select the ADS from the list of devices. Click in the row corresponding with your server to display details for the device. 16. Switch to the Data Sources tab. 17. Click Add Data Source. 18. Select your AMD from the list and then click the button. 19. Click Finish to save the configuration. 20. Configure the ADS and CAS to work together. As a result, your device appears on the Devices list. To view the list, go to Devices and Connections Manage Devices in the top menu of the RUM Console. The Devices screen presents a comprehensive view of all the devices that you add, including their IP Address, Port, Description, Type, Version, Connection, Hardware Health, and Configuration. 23

24 Chapter 2 Adding Basic DC RUM Devices 24

25 CHAPTER 3 Verification of Traffic Monitoring Quality Use the RUM Console to verify the traffic monitoring quality using two tightly connected solutions: Sniffing Point Diagnostics and Application Overview. We highly recommend that you perform this step at the beginning of your DC RUM deployment to verify that your hardware is working properly and that the applications you intend to monitor are detected. You can verify the test results and repeat them as needed at any time and for any network conditions. IMPORTANT All verification is based on a traffic recording, either manual or automatic. The outcome may not be representative if the target traffic is low at the time of recording or if you are unable to capture a satisfactory number of complete sessions. Choose automatic or manual traffic recording to capture unfiltered or filtered traffic. Enable automatic recording only during the configuration process and then disable it. It can negatively affect the performance of the AMD during normal operations, especially if you are running a 32-bit AMD in a high-traffic environment or a 64-bit AMD with the native driver. For the most complete and reliable statistics, use the 64-bit customized driver on the AMD. The verification of traffic monitoring quality is possible only for AMD 11.7 or later. Sniffing Point Diagnostics Sniffing Point Diagnostics is a type of hardware state analysis that enables you to perform pre-monitoring tasks without the need to access the AMD terminal. You can use it to validate the operation of the sniffing points, instead of using a series of UNIX or rcon commands. This step can be performed at the DC RUM deployment stage or at any other time to determine if the AMD performance is affected by malfunctioning hardware or external networking conditions. The Sniffing Point Diagnostics analysis can detect issues, such as: No traffic detected on sniffing interfaces. Interface or link overload. 25

26 Chapter 3 Verification of Traffic Monitoring Quality Poor quality of traffic due to mirrored ports on switching hardware configuration. Dropped packets (indicates AMD overload). Network conditions when unidirectional traffic prevails. Rejected packets, invalid packets, wrong check sums for packets. Missing packets (either lost or dropped). Missing bytes (how much traffic is lost in general). Conditions affecting AMD calculations, such as: Duplicate traffic that cannot be handled by the AMD. Incorrect choice of packet deduplication method. Incorrect settings for packet deduplication buffer. Incorrect settings for maximum packet size or huge packet size. Conditions affecting AMD performance, such as: Duplicate traffic handled by the AMD. Large percentage of non-ip traffic (noise). Large percentage of non-tcp or non-udp traffic (noise). Reordered sessions. Miscellaneous SSL problems: Unsuccessful decryption (in general). Uninitialized SSL cards unable to decrypt traffic. The ratio of encrypted and successfully decrypted traffic to encrypted and non-decrypted traffic. Incorrect or missing private keys. No match between the key and server certificate. Dropped or corrupted packets preventing decryption. Unsupported cipher methods (for example, Diffie-Hellman based key infrastructure). Unsupported SSL versions or features. Prerequisites and Best Practices To diagnose application detection problems and sniffing point connection problems, ensure that: All cables are connected correctly. The AMD is properly installed and configured. This includes the post-installation steps, such as interface identification and network configuration. Traffic recording lasts long enough to capture a reasonable amount of traffic volume, for example, 20 to 30 minutes of traffic. 26

27 Chapter 3 Verification of Traffic Monitoring Quality Do not use specific capture profiles when recording traffic. Always use the All available option for capture profiles when you do manual recording. When you need to diagnose traffic or capture port problems, enable automatic trace recording. Trace recording provides access to regular and fresh snapshots of the traffic that is traveling on your network. Sniffing Point Diagnostics Reports Sniffing Point Diagnostics reports are organized into several sections, each representing a separate set of metrics related to either hardware or network traffic. This topic provides directions for viewing the reports, but you can follow each step or skip steps to view the only the information important to you. 1. Start either by looking at device health or from the reports section directly. If you enabled automatic trace recording, you can monitor the device state on the Device Status tab of the Devices screen. A separate set of statistics is provided for each AMD added to the console. If there are any alarm messages, go to Devices and Connections Verify quality of monitored traffic. Inspect network interfaces in detail for a selected AMD. Open the Overview report to verify that the proper type of network driver is being used (custom or native) and that traffic has been detected, and check the number of dropped packets and other performance related issues. You can also verify that the NIC drivers are operational. For more information, see Network Interface General Statistics [p. 27]. 2. Switch to the Protocols section to inspect protocols. See whether network protocols are detected (IPv4 or IPv6) and verify detection of transport protocols (TCP or UDP). For more information, see Network and Transport Protocol Information [p. 30]. 3. Switch to the Services section to see the most active services. For more information, see Services Detected in the Traffic [p. 30]. 4. Depending on your goals, switch to the Sessions section either by selecting a particular service on the Services report to see session details or by choosing the Sessions section to see general statistics for all sessions. For more information, see Session-Related Statistics [p. 30]. 5. If you use SSL decryption, you can inspect whether there are problems detected for the currently used SSL engine or keys. For more information, see SSL Diagnostics [p. 32]. Network Interface General Statistics The Overview section of the Sniffing Points Diagnostics reports enables you to verify the general state of capture ports on a selected AMD. The information in the Overview section is gathered directly from the NIC driver operating on the AMD. For the most reliable results, use the 64-bit customized drivers. 27

28 Chapter 3 Verification of Traffic Monitoring Quality Calculation of Analyzed Traffic The calculation of analyzed traffic is performed in several stages, gradually excluding the irrelevant statistics: 1. The overruns are excluded first. When the received packets are counted, the overruns are omitted. 2. The calculation of the received packets depends on the subtraction of errors and filtered-out packets. 3. The dropped packets are counted after the filtered packets are disregarded. 4. The number of analyzed packets is the count of packets remaining after all of the previous categories are subtracted. In default AMD installations, non-tcp/udp packets are not part of this process and are never counted when the number of analyzed packets is given. Non-TCP/UDP traffic increases the amount of analyzed traffic only if you enable the monitoring of the default software services. Figure 1. Graphical Explanation of Analyzed Traffic Calculation for an AMD with 64-bit Customized Network Interface Driver All network packets Overruns Packets not received Received packets Errors and non-conditional filtering Errors: length or bad checksum; filtered out: non-ip Load balancing If active, fraction of the traffic Configuration filtering Based on defined software services Sampling and dropped packets Packets not analyzed due to performance issues Non-TCP, non-udp If default software services enabled Analyzed packets 28

29 Chapter 3 Verification of Traffic Monitoring Quality Interface Operation-Related Metrics The statistics presented on this screen include: Overruns Overruns may indicate a link overload. The overload is typically caused by an exceptionally high traffic volume. This value may also indicate that the network interface or network interface driver cannot manage the amount of traffic received. Other hardware-related issues may also cause overruns. If a high overrun occurs, limit the traffic volume received by the card. Errors (length) Packets of erroneous length are reported when they are too big (such as jumbo frames) or are bigger than the maximum transmission unit (MTU). To avoid such problems, you can increase Maximum packet size in the Entire Configuration perspective. For more information, see Configuring General Data Collector Settings [p. 37]. Errors (bad checksum) Checksum-related errors are typically caused by insufficient signal strength on an optical link. In other cases, checksum errors may indicate Ethernet distortion, such as duplex problems, where the checksum errors may result, for example, when the duplex auto-negotiation process fails. Check the host switch and AMD duplex settings. Filtered out (non-ip) Non-IP packets, such as ARP traffic. Even large numbers of such packets are generally considered harmless. They are not analyzed by the AMD software and are regarded as noise. Preventing such traffic from reaching the AMD may reduce the possibility of performance degradation. Filtered out (load balancing) This setting is only applicable in deployments with multiple AMDs where each device only analyzes a certain part of the same traffic. Filtered out (configuration) Provides additional filtering based on software service definitions. In default installations, where monitoring of the default software services is turned off, the driver limits the number of processed packets to only those that are relevant to the IP addresses included in user-defined software service definitions. Dropped (sampling) Sampling here means dropping packets when the driver performance is degraded. Packets are dropped in a controlled manner, and always with care, to preserve complete and consistent sessions. The packet drops almost always mean that traffic is too heavy for a complete analysis and that, with packet drops, the precision of CAS reports is diminished. Sampling is only active with the customized 64-bit driver and diagnostics always use this sampling mechanism regardless of the settings used in the general AMD configuration. Dropped (driver performance) Drops are always a symptom of problems, especially when SSL analysis is deployed. Drops occur when AMD software is unable to analyze all of the packets it receives from the driver. If you use 32-bit or native drivers, you may experience uncontrolled packet dropping. If you use the 64-bit customized driver, packet dropping may occur, but in a software-controlled manner with care for monitored data contingency. 29

30 Chapter 3 Verification of Traffic Monitoring Quality To avoid packet dropping, decrease the traffic volume that your AMD analyzes or reduce the number of monitored software services. Non TCP/UDP Whether these statistics are classified as analyzed or not depends on the default software service monitoring. The numbers in this section are mostly relevant if you enabled monitoring of default software services. In this case, ICMP traffic is also analyzed. If monitoring of the default software services is disabled and you still see a large percentage of non-tcp and non-udp traffic, it is possible that AMD performance will be affected. Network and Transport Protocol Information Use the Protocols report to check the ratio of supported transport or network protocols. Only supported protocols are shown. In general, this report enables you to check whether traffic that makes sense (from the DC RUM perspective) is present and is heavy enough to give meaningful results for report servers. NOTE To obtain the most reliable results, use 64-bit customized drivers. The limited approximation algorithms used by native and 32-bit customized network interface drivers may lead to differences between the packet count in this and the Overview sections. Problem Detection Low traffic for the IPv4 or IPv6 network protocols may indicate further monitoring problems. The presence solely of multicast or broadcast traffic is an indication that port mirroring is not enabled or inactive. Services Detected in the Traffic This overview report enables you to identify the most active services on your network. You can see what their load is and what protocols they use, and filter the results to display all data, monitored services, or unmonitored services. You can also use filters to display statistics for all, monitored, or unmonitored services with additional protocol filtering. For each service, you can open the Sessions report to verify session-level statistics. NOTE To obtain the most reliable results, use 64-bit customized drivers. The limited approximation algorithms used by native and 32-bit customized network interface drivers may lead to differences between the packet count in this and the Overview sections. Session-Related Statistics The Sessions section enables you to view detailed information about traffic quality. The statistics presented on this screen include: 30

31 Chapter 3 Verification of Traffic Monitoring Quality Duplicates, Unhandled duplicates The value presented on the Sessions screen depends on the currently selected deduplication method in your AMD configuration. Packet duplicates may indicate incorrect configuration of mirroring ports. While this may be a sign of a problem, values of 10 to 20 percent typically are no reason for concern. The AMD is capable of packet deduplication. Higher numbers of duplicate packets will degrade the AMD performance and may negatively influence the monitoring results. The diagnostics mechanism for duplicate detection and counting for this report works with different settings than the network monitoring processes on the AMD. Duplicate detection is performed using both methods of duplicate detection and with different settings (buffer and delay detection size). Based on these settings and calculations, Sniffing Point Diagnostics provides suggestions concerning duplicate handling, such as increasing buffer size or changing the deduplication mechanism. You should check whether there are unhandled duplicates detected, in which case it is suggested that you switch the detection method in the AMD general settings. For more information, see Configuring General Data Collector Settings [p. 37]. Unidirectional TCP sessions and UDP streams This may indicate a problem related to incorrectly configured mirroring ports. If the value of unidirectional traffic exceeds 90 percent, the RUM Console always marks it as an error. The numbers on the Sessions screen are the sums of many measurements; you are able to go deeper and analyze details for each server and check whether this is a problem related to a significant service or protocol. Insignificant traffic may be recorded and included in the general analysis, so always check the detailed reports when you see alarming numbers on the Sessions report. TCP sessions with missing packets Missing packets may result from interface or driver packet drops. If a session with missing packets is shown, the percentage value is counted with regard to all sessions. For example, if two percent of sessions have missing packets reported, this means that two out of a hundred sessions have missing packets. TCP sessions with missing packets and TCP bytes lost in missed packets may provide valuable insight into SSL decryption problems, especially in the case of long SSL sessions. TCP bytes lost in missing packets This is a complementary value to the TCP sessions with missing packets. Verify the number of lost bytes with regard to missing packets to see whether the problem is serious (if there are large sums of missing bytes). This is useful additional information in the case of long TCP sessions; because one lost packet is enough to classify a session as having missing packets, the number here gives insight into the actual loss rate. TCP sessions with reordered packets Reordered packets are typically found when there is a WAN link enabled. Devices transferring WAN packets may affect the packet order. The existence of reordered packets is not a problem in itself, because the AMD software can restore original packet order, but an excessive number of such packets may cause performance degradation. 31

32 Chapter 3 Verification of Traffic Monitoring Quality NOTE To obtain the most reliable results, use 64-bit customized drivers. The limited approximation algorithms used by native and 32-bit customized network interface drivers may lead to differences between the packet count in this and the Overview sections. SSL Diagnostics The traffic for this report is dependent on capturing complete sessions. Incomplete sessions, missing packets, or missed handshakes cause a large number of errors and a large number of errors results in unreliable reports. Always be sure to record enough traffic for an adequate length of time to allow you to capture complete sessions. The Statistics for encrypted traffic, SSL card and keys report is only available after the traffic trace recording is finished. Partial statistics for SSL are not provided for unfinished sessions. General Statistics for Encrypted Traffic For a given time range, defined by the scope of the recorded traffic traces, you can see the recognized SSL engine (for example, OpenSSL or ncipher) and the number of keys exchanged in the traffic. The remaining sections of this diagnostic report show the detailed information about the keys, the overall summary of the captured SSL traffic, and whether there are errors. The servers section shows information for all SSL traffic captured during the traffic trace recording. All of the detected encrypted protocols are listed together with their matching keys, if they are seen in the traffic. You can see whether the key exchange was successful; the matched keys are indicated by the icon. Key and certificate matching enables you to verify that certificates were found and were valid. No matching may indicate that the certificates are out of date. SSL Server Status The Status column shows whether there are errors or whether erroneous sessions prevail. A traffic capture sometimes does not contain session beginnings, or it contains incomplete handshakes, or it has no master session; these sessions are marked as ignored, as indicated by the gray ( ) color bar. The sessions with errors are marked by a red ( ) color bar. The main causes of errors are missing packets or missing keys. Other causes of errors are listed in detail on the Detailed SSL Statistics for servers report. Detailed SSL Statistics for Servers Detailed SSL statistics for servers are accessed from the Server or Status columns. This report shows: The percentage of the sessions without error, with errors, or ignored. The counts of each problem, in detail, for the error or ignored sessions. The number of decrypted sessions if there are no problems. 32

33 Chapter 3 Verification of Traffic Monitoring Quality You can filter the results. Use Sessions finished to display the data for completed sessions. Use Sessions in progress to display the sessions that are still in progress (sessions that did not end before the traffic capture stopped; to see those session statistics). Figure 2. Example of Detailed SSL Statistics for Server, Errors Detected Due to Private Key Mismatch SSL Keys Because invalid or outdated keys are usually not removed from SSL cards, the list of keys for which an error status is indicated may be considerably long. In such cases, sort by the Status column to see keys correctly matched. Note that it may be necessary to format the SSL card storage area to refresh the key list. Application Overview The Application Overview screen enables you to answer several questions about your applications at the onset of your monitoring configuration. Are all my applications or servers detected? What applications or servers are detected? Can the detected applications or servers be successfully monitored? How heavy is the traffic for each application or server? What services are detected on each server? How heavy is the traffic for each detected service? Note that incomplete sessions are not analyzed. If no beginning is recorded for a session, that session is not analyzed. 33

34 Chapter 3 Verification of Traffic Monitoring Quality The Application Overview screen is an optional step towards defining new software services. To access it, select Software Services Add Software Service in the console top menu, then select By traffic lookup. Figure 3. Example of the Application Overview Screen Showing Detected Applications From this screen, you can configure software services either manually or by using the wizard. If it is possible to go through a step-by-step configuration, a wizard icon ( ) is displayed for the given protocol or service. Application Detection Mechanism Application detection is a three-stage process: 1. To provide the most accurate results, packet analysis for SSL, HTTP, HTTPS, SOAP, and related protocols is performed as a first step toward application type detection. Application recognition is based on the first matching pattern found. This means that some services may not be properly classified if multiple protocols are used in one session. For example, if your application uses HTTP and SOAP over HTTP protocols, and plain HTTP communication opens a session, the application is classified as HTTP. 2. Applications are also detected based on discovery of well-known ports. The default protocol definitions are stored on the AMD and can be exported from the RUM Console. For more information, see Exporting the AMD Configuration in the Data Center Real User Monitoring Administration Guide. At times applications may use ports commonly used for other purposes. The AMD is unaware of these circumstances and will report well-known protocol names. For example, if one of your web applications uses port 8080 and uses HTTP for communication, the AMD reports this as an HTTP proxy. 3. If none of the selected conditions matches, the application is labeled as Unknown TCP or Unknown UDP. 34

35 Chapter 3 Verification of Traffic Monitoring Quality Server recognition in application detection is based on heuristic session analysis; results may vary depending on the type of network interface driver used. Using RUM Console to Identify Problems Related to Network Hardware Operation Typical configuration errors related to port mirroring can, at times, severely affect the AMD software traffic analysis capabilities. Faulty hardware configuration may result in no data seen by the AMD, a large number of duplicate packets reaching the AMD, or only a limited portion of traffic visible to the monitoring software. Use the Application Overview and Sniffing Point Diagnostics sections as tools to solve issues related to the switching hardware configuration. The following list describes several common problems and some possible causes and solutions. No data seen by the AMD The cable is connected to the wrong physical port on the destination switch. This can be checked by physically tracing the cable directly to the switch and confirming the port ID. The port mirroring configuration (for example, SPAN on Cisco hardware) has been set or changed to mirror incorrect ports or an incorrect destination. This can be resolved by logging on to the source switch and checking the mirroring ports configuration relevant to the requirements (see the vendor-specific documentation for details). No data seen on Application Overview but non-tcp/udp traffic seen in interface statistics The port mirroring configuration (for example, SPAN on Cisco hardware) has been set or changed to mirror incorrect ports or an incorrect destination. This can be resolved by logging on to the source switch and checking the mirroring ports configuration relevant to the requirements (see the vendor-specific documentation for details). Application Overview does not show all expected data The port mirroring destination may be oversubscribed or dropping packets. Check this by logging on to the switch and checking the SPAN or mirror destination interface. If it is recording many drops, review the configuration of source ports to understand the ratio of source interface bandwidth to destination interface bandwidth. If the ratio is excessive (for example, greater than 4:1), consider reducing the number of source interfaces. If applicable, consider using device-specific filtering to reduce the load on the destination interface (for example, VACL, Rx-only, or Tx-only sources). By design, port mirroring does not forward faulty frames. Check the source device interface statistics to ascertain the nature of the drops (see the vendor-specific documentation for details). Check the interface-related metrics. If there is a high rate of Errors (bad checksum), consider hard-configuring one end of the AMD SPAN connection to prevent auto negotiation. Session-related report shows a high rate of packet duplicates A SPAN or mirror operates by copying frames from source interfaces and directing them to the destination interface. In effect, configurations often result in two copies of a packet. 35

36 Chapter 3 Verification of Traffic Monitoring Quality For example, if the source of a SPAN or mirror is set as a VLAN, any traffic that goes from one switch port to another switch port within the VLAN appears twice on the mirrored port. If the number of duplicates starts to affect AMD performance, consider reducing the number of source interfaces. If applicable, consider using a device-specific filtering control to reduce packet duplication (for example, VACL, receive-only, or transmit-only sources) or consider using tap technology as opposed to port mirroring to collect the data. Only unidirectional streams are seen on session-related overview If the AMD is connected via a SPAN or mirror, the configuration has been set incorrectly to send only one side of a receive or transmit stream to the destination. Log on to the local source switch to check the configuration (see the vendor-specific documentation for details). 36

37 CHAPTER 4 Basic Monitoring Configuration You can define many configuration settings globally for all software services for a given protocol and Data Collector, or locally for specific user-defined software services. If you specify both types of settings, the settings for a user-defined software service take precedence over the corresponding global settings. Use the RUM Console to perform basic monitoring configuration, including the global settings for Data Collectors, operations, and the analyzer, as well as configuring Dynatrace to recognize WAN-optimized traffic. Configuration and recognition of optimized WAN traffic in Dynatrace is optional and depends on whether WAN optimization is used in your network. Refer to the Data Center Real User Monitoring WAN Optimization Getting Started. NOTE If you make any significant changes in the configuration, such as removing defined software services or operations, your are advised to restart the AMD. This is to prevent persistent TCP sessions from blocking your changes. Configuring General Data Collector Settings For any given data collector device such as the AMD, you can set a variety of options, such as time thresholds. The general settings affect the monitoring of default and user-defined software services. Some of these settings can then be overridden by settings for a particular analyzer, software service, or URL. To define the general settings for an AMD: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Select Configuration Global General to access the list of general configuration settings. 37

38 Chapter 4 Basic Monitoring Configuration While some of the options control only general AMD behavior, some options in the Advanced group affect more specific configurations in application monitoring. For example, if Inherit from global settings is selected in your other configurations while configuring user-defined software services, the global setting takes precedence over the specific monitoring configuration. Configuration options include: Monitoring interval The monitoring interval in minutes. Increasing this value reduces the number of chunks of data that need to be transferred and processed. Default: 5 minutes. Verify that the monitoring interval is synchronized between the data collectors. Operation time threshold The number of seconds after which an operation is considered to be slow. The global threshold value depends on the analyzer. This threshold is used by the following analyzers: Cerner Cerner over MQ Epic Generic with transactions HTTP MS Exchange over HTTP MS Exchange over HTTPS Oracle Applications over HTTP Oracle Applications over HTTPS SAP GUI SAP RFC SAP GUI over HTTP SAP GUI over HTTPS SMTP SSL SSL Decrypted Server time threshold The Server time threshold relates to the server time portion of an overall operation time. Server times above the threshold limit are considered to be slow due to poor datacenter performance. This threshold is used by the following analyzers: HTTP SAP GUI over HTTP SAP GUI over HTTPS IP address of the server authorized to set AMD time The IP address of the report server that has the authority to synchronize the time with this AMD. 38

39 Chapter 4 Basic Monitoring Configuration In an environment with a number of servers sharing the same AMD, it is good practice to designate only one of these servers as a time synchronization server to make changes to AMD settings. Otherwise, the server used for time synchronization will change inadvertently every time you save an AMD configuration. Default analyzer The default setting for the TCP analyzer is Generic (with transactions). To change it, select another analyzer from the list. Client RST packet timeout to mark session as CLOSED If the time between the last ACK for data sent by the server and an RST packet sent by the client is greater than this value, the session is treated as closed instead of aborted. Huge packet size The upper size limit, in bytes, of an HTTP request to be processed successfully by the AMD. Maximum packet size The AMD is capable of processing packets of up to bytes, besides the Ethernet standard MTU (Maximum Transmission Unit) of 1536 bytes. Choose one of the predefined values (2048, 4096, 8192, or bytes) to enable the AMD to process non-standard MTU packets. When you have chosen the Maximum packet size value, make sure that you also set the Huge packet size to an applicable value. Enabling theamd to process nonstandard MTU packets without extending RAM on the machine and leaving Packet buffer size (64-bit AMDs only) and Data memory limit unchanged can cause an excessive packet loss. To avoid this, extend RAM and configure its usage as recommended in the tables below. For more information, see Setting Packet Buffer Size in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide and Setting Data Memory Limit in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. NOTE Do not enable the processing of large packets for a Small AMD. These devices are not designed to process larger packets. For more information, see Small AMD in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. Table 1. Recommended RAM Configuration for Maximum Packet Size Values for AMDs Maximum packet size 8192 B or less 8192 B B Recommended RAM size for 64-bit platforms 64 GB 96 GB 128 GB 39

40 Chapter 4 Basic Monitoring Configuration Sampling enabled Supported in 64-bit customized AMD drivers and all- native drivers. The sampling mechanism is beneficial when heavy traffic may negatively affect AMD performance and there is a risk of losing IP session consistency. When this option is enabled, the AMD tries to analyze the greatest possible portion of traffic. It drops packets in a controlled manner that preserves complete and consistent sessions. Note that statistics for dropped packets are not shown on the report server. If packets are dropped because of sampling, the CAS shows notification messages. For percentages between 75 and 99, a warning icon is displayed; for values below 75, the report server issues error messages. When this option is disabled and the network interface driver performance is degraded, random packets are dropped. Default: enabled. For more information, see Using Network Interfaces with Native Drivers in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide and Driver, Network, and Interface Configuration in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. NOTE When capturing packets on an AMD with sampling disabled, if the AMD experiences packet drop due to high traffic volume, the packet capture is not automatically canceled. If this occurs, select Tools Packet Data Mining Tasks on the CAS, find the task that was using the AMD in question, and click to cancel that task. Deduplication method You can choose one of four methods for eliminating duplicate packets: Based on TCP checksum and IP ID Using this method, duplicate packets are detected based on their TCP checksum and IP ID. Based on TCP checksum and IP ID (excluded SEQ and ACK numbers) Using this more complex, two-stage method, duplicate packets are detected based on a modified packet KCP checksum (SEQ and ACK numbers are excluded) and IP ID. This method is useful if the AMD captures packets on various interfaces of the router, rewriting SEQ and ACK numbers. A packet is considered a duplicate when the modified checksum, IP ID, and SEQ and ACK numbers are identical. First, a packet checksum with SEQ and ACK numbers is created and compared to the packets stored in the detection buffer. If the comparison indicates that the packet is not a duplicate, it is checked to determine whether it matches the current session. A packet matches the current session when its SEQ and ACK numbers are different from processed and cached numbers by the value defined in TCP duplicate window. If the difference exceeds the defined value, the AMD assumes the ACK and SEQ numbers were rewritten by the router and the packet is considered a duplicate. 40

41 Chapter 4 Basic Monitoring Configuration TCP checksum, IP ID and MAC address (excluded SEQ and ACK) Using this method, the deduplication process is similar to the one based on TCP checksum and IP ID (excluded SEQ and ACK numbers), but in addition to TCP checksum and IP ID, the source/destination MAC addresses are also taken into account for the calculation. TCP checksum, IP ID and MAC address Using this method, duplicate packets are identified based on their TCP checksum, IP ID and source/destination MAC addresses. TCP duplicate window This setting is useful only if Deduplication method is set to Based on TCP checksum with excluded SEQ and ACK numbers. It is used for determining whether a packet, based on its SEQ and ACK numbers, belongs in the session. If a packet's SEQ and ACK numbers differ from the current session's SEQ and ACK numbers by a value larger than TCP duplicate window, the packet is considered a duplicate. Default: Packet buffer size The number of packets to keep in the buffer for use as a basis for comparison in duplicate packet detection. Newly captured packets are sequentially compared to the packets in the buffer. A newly captured non-duplicate packet (all packets in the buffer are unique) is placed on the top of the stack and the oldest is removed. Range: 6 to 24 packets. Default: 16. Reset duplicate detection time threshold Time of inactivity (in seconds) after which the duplicate packets elimination mechanism is reset. If Deduplication method is set to Based on TCP checksum with excluded SEQ and ACK numbers or TCP checksum, IP ID and MAC address (excluded SEQ and ACK), and the Reset duplicate detection time threshold should be greater than every response generation time (server time). 5. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. 6. Close the AMD Configuration window. Configuring Operation-Related Global Settings The operation-related global settings enable you to define options that apply to all monitored operations. These settings take precedence over the options defined for individual operations. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 41

42 Chapter 4 Basic Monitoring Configuration 4. Select Configuration Global Operations to display the general configuration settings. The options are: Operation load time threshold The number of seconds after which an operation is considered slow. You can set this value with a precision of one ten-thousandth of a second. Default: seconds. The threshold is used by following analyzers: IBM over MQ Jolt MS Exchange Oracle Forms over HTTP Oracle Forms over HTTPS Oracle Forms over SSL Oracle Forms over TCP SOAP over HTTP SOAP over HTTPS XML XML over HTTP XML over HTTPS XML over MQ XML over SSL Max. operation duration The maximum number of seconds an operation can take. You can set this value with a precision of one ten-thousandth of a second. Default: 3600 seconds (1 hour). User abort threshold The minimum number of seconds between the beginning of a hit and TCP reset to count it as a user abort. Default: seconds. (You can set this value with a precision of one ten-thousandth of a second.) ADS data generation settings The options in the ADS data generation settings section can be used to handle various types of standalone hits, which are hits that cannot be automatically assigned to operations because the reference information, such as correlating response, defined or auto-learned URL, no authorization, or orphaned redirects, is missing. By default, most standalone hits are not taken into account when generating operations data. Report data without monitored URL Select this option to report data for hits without a URL that has been explicitly defined in user-defined services or recorded through auto-learning. Report standalone hits without monitored URL Select this option to report data for standalone hits that at the same time do not refer to a monitored URL, as in Report data without monitored URL. 42

43 Chapter 4 Basic Monitoring Configuration Standalone hits are hits without a response header, unauthorized hits, orphaned redirects, or other hits missing the reference context. Report hits without response header Select this option to report data for discarded hits (hits without a correlating response header). Report hits not added to any operation Select this option to report data for other standalone hits caused by factors not covered by other options of this section. Report unauthorized hits Select this option to report data for hits with rejected authentication. Report orphaned redirects Select this option to report data for redirects to sites that are not being monitored or are not visible and therefore appear as orphaned redirects. Report filtered data This is a diagnostics option. When configuring content type monitoring, you can filter out pages based on the content of the URL. For more information, see Monitoring of Non-HTML Objects Based on Content Type [p. 116]. If you select this option, the filtered out pages are not reported, but are saved in the AMD data files. Ignored clients A list of clients for which TCP setup time are ignored and all operations start from the request packet. Right-click the list to open a menu of command options: Add, Edit, or Delete. 5. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. 6. Close the AMD Configuration window. Operation Time in Web Monitoring In the case of web (HTTP) analysis, operation time may be referred to as page load time. The ADS reports operation time details on reports based on the Operation data data view. Operation Time is a compound metric whose value is based on an analysis of the operation being measured. The operation process consists of establishing an HTTP session with the web server and loading the page from the web server. A typical web page is composed of multiple objects, each of which is retrieved from the server through a single HTTP-level operation (a hit). The duration of each hit is of less importance than the complete operation time. To determine page load time, it is necessary to watch all individual HTTP operations (hits) that belong to the page. Redirect A redirect is when the server gets a request from the browser and it decides to redirect that browser to another server or another URL. For example, when the user types the URL the server may instruct the browser to fetch 43

44 Chapter 4 Basic Monitoring Configuration instead. A redirect operation is optional; it may or may not happen and may be more complex than described here. For example, the browser may be redirected to another server and another DNS Lookup and TCP Connection Setup may be required during that process. The time required to complete that whole process is represented by the Redirect bar on the page load diagram. Base page request After the TCP and optional SSL connections are in place, and after the optional redirect operation has completed, the browser requests the base page (an HTML document). Before the server responds, the HTTP stack has to process the request. That time is marked as HTTP Server Time on the diagram. Preparation of base page response In many cases, the page content will be dynamically created based on user visit history and preferences, or on some other criteria. This requires the HTTP server to involve a higher-level application to produce customized content, perhaps through a CGI script or perhaps through a separate application server or database server. The dynamic nature of some web pages is determined by the AMD looking at the server behavior when it sends the response to the client. If the AMD detects that the server is delaying response delivery despite the fact that client acknowledged the reception of all packets, the AMD marks this event as Server Think Time. Response download Now the content is transferred to the browser. After having parsed the document content (or during this process), the browser opens additional TCP connections to the target server. For example, the base page page.html on the page load diagram contains three objects, img1.jpg, img2.jpg, and img3.jpg. To load these objects, the browser opened one additional TCP connection. The number of these additional connections depends on the browser type and the number of embedded objects. The efficiency of HTTP depends on the ability to load page elements on concurrent, parallel connections, in which case, when the site is busy servicing a request for one object, the transfer of another object can use the available bandwidth between the browser and the web server. This transfer time is called Network Time (part 1). Preparation and download of content elements In some cases, the elements embedded in the base page are produced dynamically or are delivered from dedicated cache servers (in many cases, placed behind content switches). It therefore makes sense to measure the time spent in the data center to produce such content. The total time is represented by Image Server Response Time. After the page has been loaded, the AMD stores the complete Operation Time and associated metrics for further aggregation and analysis. 44

45 Lookup TCP SYN Chapter 4 Basic Monitoring Configuration CLIENT SERVER DNS lookup Response SYN SYN ACK time RTT ACK SSL handshake SSL connection setup time Network time (part 1) GET page.html HTTP redirect GET page.html Request time Redirect time Redirect time 1/2 Server ACK RTT ACK GET RESPONSE page.html HTTP server time Server time Operation time or 1/2 Client ACK RTT ACK Server think time Page load time SYN GET image1.jpg RTT ACK SYN ACK GET image3.jpg image1.jpg Image server response time (1) Network time (part 2) Image server response time (3) GET image2.jpg image3.jpg image2.jpg Image server response time (2) 45

46 Chapter 4 Basic Monitoring Configuration 46

47 CHAPTER 5 Configuring AMD to Monitor HTTP and HTTPS Traffic For a detailed analysis of web application traffic, you need to define software services to be monitored on specific IP addresses and ports: specify the service name and analyzer, designate the AMDs to monitor the service, define the software service rules, and check whether your configuration matches the existing traffic. To define one new software service: 1. Choose the traffic statistics to filter services available for the definition. For more information, see Capturing Traffic Traces [p. 50], Application Traffic Categories [p. 51], and Defining Software Services [p. 47]. 2. Select the services (server:ip address combinations) to monitor. For more information, see Selecting Services for Software Service Definition [p. 52]. 3. Configure URL monitoring. For more information, see Configuring URLs for a Software Service Definition [p. 54]. 4. Configure user identification. For more information, see User Name Recognition Configuration [p. 65]. 5. Verify and publish the configuration. For more information, see Reviewing and Publishing a Software Service Definition [p. 77]. Defining Software Services New DC RUM users can use the configuration wizard to define web software services and then use the traffic information to narrow the number of servers available for those software service definitions. Using the information from reports based on the recorded traffic, you can find and select the information related to your applications and use it as a starting point for your monitoring configuration. 47

48 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Before You Begin It is assumed that you have the latest version of the AMD added and configured in the RUM Console. For more information, see Adding an AMD to the Devices List in the Data Center Real User Monitoring Smart Packet Capture User Guide. It is assumed that you have the latest version of the CAS added and configured in the RUM Console and connected to your AMD. For more information, see Adding a CAS to Devices List [p. 21]. To use parameters from recorded traffic to configure application monitoring: 1. Start and log on to RUM Console. 2. In the top menu, select Software Services Add Software Service. The Add Software Service pop-up window appears, listing all ways of adding a new service. 3. Select By traffic lookup as a method of adding a new software service definition. It opens the Application Traffic Categories [p. 51] report that is based on a selected traffic trace. In case no traffic has been captured yet, you can record it using controls available on the report screen. For information on how to do it and how to manage traffic traces, go to Capturing Traffic Traces [p. 50]. 4. On the Application Traffic Overview, select a traffic trace from the list of traces. For more information, see Capturing Traffic Traces [p. 50]. 5. Display the statistics for HTTP or HTTPS traffic. Specify the part of the traffic that you want to browse by clicking the HTTP or HTTPS link. NOTE You can also view basic statistics for undecrypted SSL by clicking SSL, but they cannot be used to define software services. For more information, see Undecrypted SSL Traffic [p. 52]. 6. Choose the report listing the most frequently occurring parameters that you want to use for filtering servers. The preview tables list the top five parameters of one type. The detected parameter types include: Host URL Server IP address Client IP address POST parameter URL parameter Cookie Agent 48

49 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic The parameters selected from the top application statistics do not become a part of the software service definition you create. They are used only as filters. Note that server filtering is possible on just one parameter type at a time, which means you can, for example, search servers based on either URLs or client IP addresses, but not on both. If you want to search the servers based on a single parameter such as URL, click this parameter in the table listing top URLs. The Web/XML Software Services screen appears. To use more than one parameter for filtering: a. Click More at the bottom of a table to view the top 100 parameters of a given type. This extended list enables you to also see the top parameter values by hovering the mouse pointer over the icon in the row corresponding to a specific parameter. To switch to another parameter type, use the View table by list, which enables you to select different parameter types without going back to the Home screen. b. Use the check boxes to select parameters you want to filter on. You can also leave all parameters unselected, in which case no pre-filtering is applied to the list of servers. c. Click Next. For example, if you want to use several URLs as filters: a. In the top URLs preview table, click More results. A pop-up window lists the top 100 URLs observed in the traffic. b. Using the check boxes, select URLs that you want to add to the filter. c. Click Create software service. 7. Specify the software service name. When the configuration procedure is complete, you can find the new software service under this name on the list of software service definitions. 8. In the Rule description, provide a name that identifies this set of monitoring instructions. 9. Choose the AMDs to monitor the new software service. By default, the new definition is published on all available AMDs that are listed in the AMDs selected for software service pane. If you do not want a certain AMD to monitor a certain software service, clear the check box in the row corresponding to that device. 10. Proceed to the selection of monitored server and ports. At this stage of the configuration task, the new software service definition cannot yet be saved. To save it, you need to specify the IP addresses and ports of servers that you plan to monitor with DC RUM. For more information, see Selecting Services for Software Service Definition [p. 52]. 49

50 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Capturing Traffic Traces You can use the RUM Console to capture and save the data need to specify the traffic preview settings for the top web application statistics or, to search for servers, URLs, and users in the application monitoring wizard. The recording tool is available on traffic and application diagnostic reports: Sniffing Port Diagnostics In the console main menu, select Devices and Connections Verify quality of monitored traffic. Application Overview, Application Traffic Categories Select Software Services Add Software Service and select the By traffic lookup option. Application diagnostic reports are the first step in defining software services. Choose automatic or manual traffic recording to capture unfiltered or filtered traffic. Enable automatic recording only during the configuration process and then disable it. It can negatively affect the performance of the AMD during normal operations, especially if you are running a 32-bit AMD in a high-traffic environment or a 64-bit AMD with the native driver. There are four recording options under Traffic Recording: Record New Trace Use this option to record filtered traffic and perform the following tasks: Specify the total capture time Select one of the predefined traffic profiles Use a list of the available filtering criteria to specify on which entity you want to filter traffic Define one or more filter values for each capture definition in the Value column NOTE You can close the recording window and browse the reports while the trace recording is in progress. This does not interfere with the capture process. However, the recording will stop if you edit software services using the software service definition wizard. Manage Recorded Traffic Lists all manually and automatically recorded traffic traces. You can use this screen to browse the filter settings for each manually recorded trace or to delete the existing traces. To see the filter settings, click a specific trace. To remove a trace, select the check box next for the trace and click Delete. Use the links embedded in the list to re-record a manually-recorded trace with identical settings, or to reset a trace. If you re-record a trace, the new trace overwrites the older trace. Toggle Automatic Recording Turns on or off automatic unfiltered traffic. Automatic Recording Settings Enable this option to automatically record traffic and define the trace length. 50

51 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Enable automatic recording only during the configuration process, then disable it. Automatic recording can negatively impact the performance of the AMD. Manual Upload of Traffic Traces To upload a previously captured trace to the Guided Configuration recording tool instead of recording new traffic, first copy the trace file to the AMD. Before You Begin It is assumed that you have the latest version of the AMD added and configured in the RUM Console. For more information, see Adding an AMD to the Devices List in the Data Center Real User Monitoring Smart Packet Capture User Guide. The traffic trace that you want to upload must be a tcpdump file and have the.cap extension. Other file formats and extensions are not recognized by the AMD. A user must have a System Administrator's role to log into the RUM Console. 1. Use an SFTP client, such as WinSCP, to log in to the AMD as the root user. 2. Copy the trace file to the /var/spool/adlex/cba directory on the AMD. 3. Change the ownership of the copied file to user compuware in a group compuware. At the command prompt execute the following command: chown compuware.compuware [filename] Where [filename] is the copied traffic trace file. 4. Start and log on to RUM Console. 5. On a screen equipped with a recording utility, choose one of the following: Select Devices and Connections Verify quality of monitored traffic to access the traffic diagnostic report. Select Software Services Add software service and choose By traffic lookup. The Application Traffic Overview window appears. 6. Select the uploaded trace from the Traffic Trace list. 7. Click Reset to view the statistics based on the uploaded traffic trace. Application Traffic Categories The Application Traffic Categories report shows the most frequently occurring parameters in the monitored web and middleware application traffic. You can select the top parameters and use them to streamline the software service configuration process. For example, you can select a number of URLs from the list of the most frequently occurring URLs and then use the selected URLs to filter available servers. Use the Application Traffic Categories screen to define new software services. Select Software Services Add Software Service from the console top menu and then select By traffic lookup. This report shows the statistics from an unfiltered traffic trace that is automatically recorded when the connection to the AMD is configured and active. You can also select other traces, 51

52 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic including filtered traffic traces, from the Traffic trace list at the top of the statistics table. After you select a trace, the top application traffic statistics reload to reflect the new setting. NOTE For top statistics based on an automatically recorded trace, the database is cleaned every seven days. If the top statistics are based on an automatically recorded trace, and if the configuration of separators between URLs and their parameters changes on an AMD, click Reset to clean the database and start collecting the data that matches the updated separator configuration. If you are using a manually recorded trace and the separator configuration changes, reset the top statistics so that the trace can be re-read by the Guided Configuration and the new separator setting can be reflected in the top statistics. For more information, see Global Settings for Recognition and Parsing of URLs [p. 137]. For manually recorded traffic, click Reset to generate the top statistics based on a trace that has been captured with the tcpdump command on an AMD and placed in the /var/spool/adlex/cba directory. Undecrypted SSL Traffic The Application Traffic Categories report in the RUM Console shows a limited set of SSL traffic statistics even if the SSL traffic cannot be decrypted, a situation that may be caused by a lack of the SSL keys required for decryption. In these situations, you can see only the basic statistics: server and client IP addresses. To view the Application Traffic Categories report, define a new software service with the wizard by selecting the By traffic lookup option. For more information, see Defining Software Services [p. 47]. The preview of non-decrypted SSL statistics is only for diagnostic purposes. Unlike the HTTP or decrypted SSL traffic, you cannot use the undecrypted SSL parameters to create new software services with Guided Configuration. Selecting Services for Software Service Definition At this stage of the software service definition, you have to select the servers and the ports that you plan to monitor. This step is required before the software service definition can be saved. Before You Begin It is assumed that you have the latest version of the AMD added and configured in the RUM Console. For more information, see Adding an AMD to the Devices List in the Data Center Real User Monitoring Smart Packet Capture User Guide. It is assumed that you have the latest version of the CAS added and configured in the RUM Console and connected to your AMD. For more information, see Adding a CAS to Devices List [p. 21]. 52

53 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic The wizard screen for selecting services consists of the following: Basic information identifying the edited software service: the service name, rule name, analyzer, and the monitoring devices. At this stage, you can still edit all this information. The List of services table, showing the services to be monitored after the definition is saved. For each monitored server, this table shows the IP address and port number and indicates the server's status and whether it appeared in the top statistics and was observed in the traffic. If you have used top application traffic statistics to select parameters to filter the servers to be monitored by the software service, the matching servers are already listed in the List of services table; otherwise, the table is empty. A movable side bar with a preview of traffic statistics that vary, depending on the selected traffic trace and filters selected on the Home page of the Guided Configuration. The sidebar consists of four sections: Web application traffic Shows which trace is currently selected and enables you to select another one from the list. Parameter breakdown Lists the parameters of a given type together with the servers on which they were seen, based on the selected trace. It also enables you to search for a specific parameter in the captured traffic or change the parameter type. If you have selected parameters from the most active application traffic to filter servers, the chosen parameter type is already filled in and the section title is updated accordingly. Without parameter preselection, this section, by default, lists URLs. Service breakdown Lists the servers and ports on which parameters of a given type were observed in the selected trace. Traffic profiles Enables you to choose which part of the traffic observed by the AMDs is taken into account for configuration. Usually, the traffic is assigned to one profile at a time, but in some cases, you may want to add more profiles. For example, to have SOAP over HTTP traffic monitored with the HTTP analyzer, you need to select the SOAP profile in addition to the default HTTP profile. To manage the list of monitored services: 1. Find services based on the parameter preview. For each parameter in the Parameter breakdown (by default, URL breakdown) section, you can click the number of servers at which a given parameter was seen to view the server list and to add any of the servers to the monitoring list. You can add one by one or in groups. (If the check box for a specific server is disabled, this server cannot be monitored by your new software service because it is already in use by another service definition.) Similarly, you can select parameters observed at specific servers, which results in the servers being added to the monitored server list. 53

54 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic To change the preview settings, change the parameter type from the Category list. The section contents are updated accordingly. To search for a server where a specific parameter of a given type appeared, type the parameter name in the Filter box. To clear the filter settings, click. 2. Browse the services (server IP address and port number pairs) observed in the traffic. In the Service breakdown section, you can view the detected services together with the parameters of a given type and add the services to the monitoring list. When you see a server that you want to monitor, select the check box to add it to the List of services. You can add singly or in groups. If the check box for a specific server is disabled, this server cannot be monitored by your new software service because it is already in use by another service definition. To change the preview settings, change the parameter type using the Category list. The section contents are updated accordingly. Note that it also affects the preview setting in the Parameter breakdown section. To search for a specific service, type the server IP address and port number in the Service filter. To clear the filter settings, click. 3. Proceed to configuring the specific URLs or finish the software service configuration at this stage. To configure the URLs and, optionally, the pages under specific URLs, click Next. To finish creating the software service definition at this stage, click Finish. When the Review summary screen opens, verify the software service definition. For more information, see Reviewing and Publishing a Software Service Definition [p. 77]. What to Do Next You can also add a new server manually: type the server IP address and port number in the New service box at the bottom of the List of services and click Add. To delete a server from the list of servers included in the software service definition, click in the row corresponding to the server's IP address. Configuring URLs for a Software Service Definition After you have selected servers for a software service definition, you can define the URLs and, optionally, the parameter groups for the specific URLs. This task may be already partly done: if you have selected servers based on the most active URLs in the web application traffic, the filtering criteria has been copied to the URL configuration screen. The URL configuration screen consists of the following parts: Basic information identifying the edited software service: the service name and the analyzer. This information, as well as the name of the software service rule and the list of devices selected for monitoring the service, cannot be edited now. A list of URLs that will be monitored after the definition is saved. For each monitored URL, the table shows the URL type, the page name (if assigned), and URL parameters. 54

55 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic A movable side bar with the preview of traffic statistics that vary depending on the selected traffic trace and the filters selected on the Home page of the Guided Configuration. The sidebar has the following sections: Web application traffic Shows which trace is currently selected and enables you to select another one from the list. If you configure the software service in a sequence of steps, the setting in this section is the same as in the previous (server selection) wizard step. If you access the service configuration at a later time to edit it, this setting will be the one that was selected in the Guided Configuration when you last accessed the perspective. At any time, the setting can be changed. Hits against URLs This section is available only on the URL configuration screen. It lists the URLs detected in traffic, based on the selected trace, together with the number of hits for each listed URL. Parameters Available only on the URL parameter group configuration screen. It shows the parameters observed in the selected traffic trace. Search Available only on the URL parameter group configuration screen. To configure URL monitoring in the Guided Configuration perspective, perform the following steps: Configuring URL Monitoring 1. On the URL definition screen, specify URLs to monitor. To manually add a URL to the list of monitored URLs, type it in the New URL field and click Add. To select a URL from the traffic, select the check box next to the URL in Hits against URL(s) section of a sidebar. To search for a specific URL, type it in the Filter box. To clear the filter settings, click. 2. Specify the URL type. In the table listing URLs, click in the last table column to activate a specific table row. Then select one of the three available types from the URL type list. Virtual HTTP Server This option refers to monitoring a host where many websites reside under a single IP address. Using a virtual HTTP server causes all of the reported pages that have no separate definitions to be aggregated into one record and reported together. This does not apply to those pages from the IP address that are defined separately in a monitoring configuration. Such individual definitions do not require that you select this option. For example, a valid virtual HTTP server address is without a trailing slash. 55

56 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Static URL Part A fully qualified URL, containing the protocol to be used, the server to be contacted, and the file to be requested, such as This URL is added to the list of monitored URLs regardless of the limit of monitored URLs. URL as Regular Expression An extended POSIX regular expression describing a set of URLs. For more information, see Regular Expression Fundamentals [p. 211]. The syntax allows you to use parentheses () to select one or more sub-expressions (specific portions of the results). If this mechanism is used, only the specified portions are reported; if more than one portion is specified, the portions are concatenated. NOTE When using a regular expression to specify a set of URLs to monitor, you must: Explicitly include the string or in the expression. Thus, for example, you should not start the expression with.* and expect that the or strings will be assumed or resolved as a part of the regular expression. The parentheses you use to select the part of the URL to be extracted must include the above strings or and they must also include the name of the host. However, the name of the host does not have to be provided explicitly, but can be resolved by the regular expression. Thus, for example, ( is correct, and so is ( The regular expression must be constructed such that, after extracting the portions delimited by parentheses, the resulting string does not end with a slash character ( / ). This rule applies to all URLs except home pages (URLs consisting only of a protocol specification and a host name). Such URL specifications should end with a slash. For example ( is legal, but ( is not legal. Note also that a specification ending with (myreport/*) is not valid because it can be matched by a string ending with a slash, as the asterisk can match an empty string. You can test the patterns that will be used by the AMD using the Regular Expressions Test tool, which is activated after you click Test located next to the regular expression pattern field. For more information, see Testing Regular Expressions [p. 213]. Example 1. Example of Using a Regular Expression to Specify Monitored URLs The use of parentheses in a regular expression is demonstrated in the following example: ( 56

57 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic The above expression matches URLs such as: but only the bracketed portion ( ) will be reported. Example 2. Complex Example of Using a Regular Expression to Specify Monitored URLs The following is a more complex example that demonstrates concatenation of bracketed portions: A site contains URLs of the form: 04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLN4o39w0BSYGYRiGBpFoYsamaEIG8Y4IEW99X4_83FT9AP2C3NDQiHJHRQDwwo2X/ delta/base64xml/l3djdyevuud3qndnqsevnelvrs82xzjfnvvn?wcm_global_context=/ assurance/wcm/connect/my Life.fr/Aide/Accueil Aide&WT.tz=1&WT.bh=12&WT.ul=en-us&WT.cd=32&WT.sr= 1400x1050&WT.jo=Yes&WT.ti=AssuranceRetraitePERP&WT.js=Yes&WT.jv=1.3&WT.fi= Yes&WT.fv=3.0&WT.sp=@@SPLITVALUE@@ where only the part after...wcm/connect/, in this case My Life.fr/Aide/Accueil Aide, is relevant for differentiating this page from other pages of this site, the rest being session ID and various parameters. If you use the following regular expression to define monitored URLs: ( assurance/wcm/connect/([^&]*) the reported URL for this page is: life.fr/aide/accueil Aide NOTE A large number of URLs defined by using regular expressions can have an adverse effect on the performance of the AMD, because resolving regular expressions is processor-intensive. For the URLs selected from the most active web application traffic and used for filtering servers, the type is selected automatically for you. For more information, see Defining Software Services [p. 47]. 3. Optional: Provide a name for the URL. In the active table row, provide the name for the added URL in the Page name column. Using a descriptive name makes it easier for you to identify the page on the CAS reports. If you do not specify it, the reports display the URL itself. Configuring URL Parameter Monitoring 4. Optional: Define one or more parameter groups for a specific URL. a. In the Parameters column, click. On the screen that appears, define the parameter groups for the URL you have selected. 57

58 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic b. Select the Report only URL part when parameters do not match check box. As a result, all of the pages that do not meet the constraints are reported under the main URL. If the check box is not selected, these URLs are not reported at all. c. In the Group name box, type the name under which you want the parameters that match the definition to appear on CAS reports. Note that this name is applied to all URLs matching the criteria. d. Using the list in the Search scope column, select where you want to look for the URL parameters. In an HTTP header In a request URL In the POST body Using a combination of these check boxes, you can, for example, search for parameters in HTTP header, request URL, and POST body at the same time. e. In the Parameter column, specify the parameter constraints for a page. The constraint defines which pages are reported under a given URL. You can define up to four constraints per parameter group. Similarly to adding URLs, you can type the parameters or you can use the parameters listed in the traffic preview. f. Use the Match list to specify the parameter matching method. The following matching methods are supported: Exact Report the specified parameter or the parameter and value. Usage syntax 'name=value' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. So, the wildcard character * is taken literally. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. 58

59 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples You can specify 'john', to match though note that in this case will not be reported because the parameter value '=123' was not explicitly specified. To match it, you would need to specify 'john=123'. Start Report parameters that begin with a specified string; report only the matched pattern, truncate any remainder of the parameter. Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples 'fred=5' will match but it will be reported as The value 'fred' will match as well as and it will be reported as 59

60 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Start (expand) Report parameters which begin with a specified string; report the entire parameter, not only the matched pattern. End Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples 'fred=5' will match and it will be reported as The value 'fred' will match as well as and it will be reported as and respectively. Report parameters which end with a specified string; report the entire parameter, not only the matched pattern. Usage syntax 'name=value' or any final part of it this string, including string of the form '=value' or just 'value'. 60

61 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples For to be matched, you can specify the following ends: '0', '00', '100', '=100', 'n=100' and so on, up to 'john=100'. Thus is reported. Value RegEx Report parameters which begin with a specified string; optionally attempt to match the remainder of the parameter with a regular expression; report the start string and selected portions of the regular expression, if any. Usage syntax Parameter is entered as name=value or any initial part of it this string including string of the form name= or just name. A regular expression (regex) is entered as an extended POSIX regular expression. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the 61

62 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed on the Parameter part; the regex part is matched as a case-sensitive POSIX regular expression. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples parameter specification fred= and a regular expression AB(C?E) will match but it will be reported as because the AB portion of the regular expression was not included in round braces. Custom RegEx Report parameters that match the given regular expression; report those portions that have been selected within the regular expression. Usage syntax Enter an extended POSIX regular expression to match the desired string. Mark portions to be reported by using round braces ( and ). Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters The request URL, POST body, or HTTP header are not split into parameters prior to pattern matching. Instead, they are treated as single units of data and the regular expression is applied to their entire contents. Only the path part of the request URL is excluded from the matching process. Limitations The regular expression is entered according to POSIX syntax. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all 62

63 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples Regular expression fred=ab(c?e) will match but it will be reported as Regular expression (.*=)AB(C?E) will match as well as and it will be reported as and as respectively. The important thing to note is that you can choose between two different match method selection modes: Report parameter Report parameter with all values To switch between the modes, use the control in the Parameters pane. The Test matching button enables you to check whether any pages match the regular expressions you typed. 5. Optional: Click Add group to define and monitor another group of parameters if required. 6. Optional: Proceed to configuring user recognition or finish the software service configuration at this stage. To define use recognition methods, click Next. To finish creating the software service definition at this stage, click Finish. When the Review summary screen opens, verify the software service definition. For more information, see Reviewing and Publishing a Software Service Definition [p. 77]. What to Do Next To delete a URL from the list of added parameters, use the icon in the table row corresponding to a given URL. Using Wildcards in URLs You can use wildcards or regular expressions when defining URL patterns. The use of the wildcard character * in a URL is governed by the following rules: 63

64 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Placed it at any position within a URL string, * stands for any string of zero or more characters. A standalone wildcard character stands for any URL. An occurrence of a URL for which we allow repetitions does not increment the counter for transaction steps. An occurrence of a URL that is matched by a wildcard alone and for which we allow repetitions does not increment the counters for transaction steps or pages. Allowing repetitions usually requires that the URL has to appear at least once. However, in the case of a URL defined with a wildcard, it can repeat zero or more times. URL Pattern Exceptions When defining a URL pattern containing a character listed in the left column, change it to the corresponding phrase in the right column. Table 2. Transaction URL Pattern Exceptions Target URL Phrase & space, %xx (hexadecimally encoded character) > Transaction Definition URL Phrase && %20,, Decoded character > Using Regular Expressions in URLs A full regular expression can be specified for a URL definition. This is referred to as the regex mode and is configured by putting the string regex: in front of the URL definition you are creating. In this mode the space character is entered as %20 and the ampersand character is entered as && (a double ampersand). Example 3. Using Regular Expressions in URLs For example, if you want to match the following pages: use the syntax: regex: To match: y z y z 64

65 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic y z use: regex: User Name Recognition Configuration The methods of user name recognition vary depending on the HTTP analyzer mode and the version of AMD. You can use a new, enhanced mechanism of user name recognition only on AMDs of the release 12.0 and later, set to work in the HTTP mode of the HTTP analyzer. AMDs of releases prior to 12.0, and AMDs of release 12.0 and later set to work in the legacy mode of the HTTP analyzer, use an old method of extracting user names. Note that this introduces a restriction when copying a new enhanced configuration to AMDs of releases prior to 12.0 or devices set to work in the legacy mode. In such cases, a new user name recognition configuration is removed. When monitoring web applications in a Citrix environment, HTTP and Citrix users are discovered, but, because HTTP has higher priority, HTTP users are reported if both are available at the same time. Overview of User Name Recognition Configuration in HTTP Mode When creating a new software service, you can specify one of two ways in which users are monitored by the software service: either select the patterns automatically recognized in the traffic or specify the patterns of your choice based on the traffic preview. The user name recognition screen consists of the following parts: A list of user name recognition policies. The policies are extraction rules defining which users will be monitored after the software service definition is saved. A movable side bar with the preview of traffic statistics that vary depending on the selected traffic trace. The sidebar consists of the following sections: Web application traffic Shows which trace is currently selected and enables you to select another one from the list. If you configure the software service in a sequence of steps, the setting in this section is the same as in the previous wizard step: server selection. If you access the service configuration at a later time to edit it, the setting will be the one selected in the Guided Configuration when you last accessed the perspective. At any time, the setting can be changed. Statistics Lists statistics (based on the selected trace) that can be used to extract user identification information. Search Enables you to look for a specific string in the recorded traffic and copy the discovered parameters to the user name extraction rules. When adding detection rules to identify user names, follow one these user name recognition scenarios applies: 65

66 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Non-context User name recognition is performed individually per hit, so each hit must contain a user name. You only need to add user name rules. User session context User name recognition is performed in the context of a particular user session. All monitored hits must contain the session ID, but only a single login hit contains the user name. Besides user detection rules, you need to define session ID rules as well. User session context, acknowledge URL User name recognition is performed in the context of a particular user session and a login is validated by redirection to a special acknowledge URL. All monitored hits must contain the session ID, but the ACK hit is the first one to contain the user name. Besides user and session ID rules, you need to provide the acknowledge URL. 1. Configure a user recognition method. If you know which request part contains user login information, use the movable sidebar to filter or search the traffic for a specific POST/GET parameter, a cookie name, or a whole HTTP header. Use the provided controls to copy the selected entities to the user name rule. The new user recognition policy with a user name rule is created for you automatically. The controls are: Enables you to extract the user ID from the selected string. Enables you to specify a user login URL. If you know what a user login is, search for a specific string in the recorded traffic using the Search collapsible pane and copy the found parameters to the user name extraction rules. Type the searched phrase in the Find box. The search is case sensitive. Use the In list to specify where you want to search for the string, select the Follow user IP address check box if you want to see all requests for a given user IP address in the search results (not only requests matching the search criteria), and then click Search. Search results are grouped into several categories, depending on the specified search criteria. For each result, you can display the request details and copy the parameters to the user name extraction rules using the provided controls. The new user recognition policy is created for you automatically. Choose a location from the list to start creating a new rule. The new recognition policy with a user name rule that is set to the appropriate search scope is created for you automatically. 2. Adjust the user name rules. When in the user recognition policy window, click Add new rule in the User name rules tab. a. Choose a search scope. You can retrieve the user names and session identifiers from a number of entities, referred to as search scopes. For more information, see Choosing Search Scope for User Identification [p. 68]. 66

67 b. Choose a search method and create a search definition. Choose one of the available search methods to detect the user names in the selected search scope. For more information, see Choosing Method of Searching for User Identification [p. 68]. When using the Basic method, you can either type the parameter in or click the to select the one of your choice. c. Optional: Filter traffic. Enter the following to filter the traffic used for user name recognition. Host Server host name. icon Path The leading part of a URL, usually a user login URL. Only hits beginning with this string are matched.. You can either type the URL path in or click the icon to select the URL of your choice. If it is not specified, the path is assumed to be /* and it matches any requested URL. d. Optional: Transform search results. It is sometimes difficult to perform a successful match resulting in a legible string in one pass. In such situations, you can perform further transformations to your initial search result. Click Add transformations and select whether you want to decode the search result or extract parts of it. If your results are compressed or encoded (also URL encoded in the case of URL parameters), you can make the search results readable by using one of available decoders: url encoding, base64, base64 + gzip, or gzip. You can also extract parts of your initial search results by using the Text search or Regular expression search method. Choosing Method of Searching for User Identification [p. 68] e. Optional: Test the rule. Click Test to roughly verify the defined rule. All tests are taken against data from the statistics, search, and user input. It simulates AMD functionality and can help to perform an initial verification of the rule accuracy. Be aware that the results returned by the AMD may be different, because the test feature relates on a limited traffic trace that does not reflect the complexity of full traffic observed by AMD. f. Click OK to save the user rule. 3. Optional: Add session identifier rules. Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Session ID rules are optional but you need to define user name detection rules first. Session ID rules may be needed to help track users on servers where the user name is not passed with every request. Adding session ID rules is in most aspects identical to adding user name rules. The only exception is the events that are necessary for the AMD to recognize session termination. 67

68 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic These are provided as a list of cookie values indicating user logoff. Additionally, you cannot search for a session ID in the response body. 4. Optional: Add acknowledge URL rules. Acknowledge URL rules are optional but you also need to define user name and session ID detection rules first. Some web servers use special URLs to which the user is redirected after a successful login. The user recognition mechanism needs this information to successfully discover a user session. The Host field is optional. The Path field is a relative path of the acknowledge URL that you can type in or select using the icon. 5. Click Next to proceed to the software service definition summary. Choosing Search Scope for User Identification You can retrieve the user names and session identifiers from a number of entities, referred to as search scopes. When you create a user name or session ID rule definition, you have to apply it to only one search scope, which you have to choose as a first step in creating the definition. Identify the entities containing user or session identification, consider applicable scenarios, then choose one of the following scopes: Request or Response Headers Request or Response Body Request parameter Cookie Note that available search methods depend on the selected search scope. For more information, see Choosing Method of Searching for User Identification [p. 68]. When you create a number of definitions for one policy, the definitions are applied in the order in which they were entered in the rules table. The first successful match is used. Choosing Method of Searching for User Identification Choose one of the available search methods to detect the user names in the selected search scope. Depending on the selected search scope, choose one the methods of extracting user names. Each search method requires you to specify a different set of extraction rules. Add prefix Use this method if you expect the value to always be preceded by a specific prefix. To extract the value, provide the prefix expected to precede the value. Cookie name search Specify the cookie from which to extract the value. Provide the value of a specific cookie name confirming a successful login. The session ID, for mapping to the value, is extracted from this cookie. Successful logins are normally recognized by a SET COOKIE operation for the named cookie Basic Use this method if you expect the value to always be carried by a specific parameter. To extract the value, provide the name of the parameter. Depending on the selected search 68

69 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic scope, the term parameter refers to a specific entity, such as a cookie name when the search scope is set to cookie, or a header field when the search scope is set to request or response header. Enter the parameter name without delimiters. The search is not case sensitive and no wildcard characters are permitted in the string unless the wildcard character * is to be used literally. This search method is not available for the response body search scope. Decode / decompress If you expect to perform a search on a compressed or encoded data, or URL encoded in case of URL parameters, you can bring the search results to a human readable form by using one of available decoders, Base64, Base64 + Gzip, Gzip or URL encoding. You can also extract parts of your initial search results by using Text search or Regular expression search methods. Choosing Method of Searching for User Identification [p. 68] Mime encoded list filter Use this method if you expect to find a value in an MIME format. Including text in character sets other than ASCII, message bodies with multiple parts and in header information encoded in non-ascii character sets. NTLM search Use this method to search for a value in an NTLM authentication request header. Depending on your choice, the value can be composed of the following fields: workstation, domain, or user. Select the fields that compose an identified value and, if necessary, change the default character used to separate the selected components in the resulting value. Parameter name and value search Use this method if you expect the value to always be carried by the specific parameter. To extract the value, provide the parameter name. Depending on the selected search scope, the term parameter may refer to a specific entity, such as a cookie name (when the search scope is set to cookie), or a header field (when the search scope is set to request or response header). Parameter name prefix search Use this method if you expect the value to always be carried by a specific parameter with a specific prefix. To extract the value, provide the parameter name prefix and indicate what data should be reported. The results of the search can be presented as a parameter name and the value, just the parameter value or just a parameter prefix. Parameter value suffix search Use this method if you expect the user name to always be carried by the specific value of a parameter with a specific suffix. To extract the user name, provide the value for the suffix. Regex search You construct a regular expression that, when applied to a selected search scope, returns the value. The regular expression must contain at least one group enclosed in parentheses. If the regular expression returns a number of search groups, you can define the custom group order by entering a comma-separated list in the order of your choice (for example, 2,1,3). This method is not available for the cookie and response body search scopes. For more information, see Regular Expression Fundamentals [p. 211]. 69

70 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic You can test the patterns that will be used by the AMD using the Regular Expressions Test tool, which is activated after you click Test located next to the regular expression pattern field. For more information, see Testing Regular Expressions [p. 213]. Example 4. The following is an example of extracting the value of REMOTE_ADDR field from the HTTP header. An HTTP header might contain the following information: GET HTTP/1.1 Accept: */* Referer: Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: Connection: Keep-Alive Cookie: FPB=061j8hura11q56cv; CRZY9=t=1; REMOTE_ADDR: The following regular expression extracts the address from the REMOTE_ADDR field: %0d%0aREMOTE_ADDR:%20\([^%0d%0a]*\)%0d%0a The expression must contain a single sub-expression delimited by pairs of characters \( and \). The expression in this example states that the search string should start at the beginning of a header line and end at the end of the line (note the use of % to denote the hex values of the carriage return and line feed characters). The line should start with the string REMOTE_ADDR:. The sub-expression to extract is a string of characters different than ASCII CR or LF, and it should occur after the space following REMOTE_ADDR: Text phrase search Use this method if you expect the user name to always be found in the text. The provided value for the search parameter will be used to match the text phrases in the analyzed traffic. Text search Use this method if you expect to find a user name between the first occurrences of strings defined by Match start and Match end. Because it is not always possible to extract the user names directly, you can use this method as a first step in preparing content for search result transformations. You can set a Search limit in bytes to avoid lengthy search results. This method is not available for the cookie search scope. Overview of User Name Recognition Configuration in HTTP Legacy Mode When creating a new software service, you can specify one of two ways in which users are monitored by the software service: either select the patterns automatically recognized in the traffic or specify the patterns of your choice based on the traffic preview. The user name recognition screen consists of the following parts: Basic information identifying the edited software service: the service name and the analyzer. This information, as well as the name of the software service rule and the list of devices selected for monitoring of the service, cannot be edited any more. 70

71 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic A list of user name recognition policies. The policies are extraction rules defining which users will be monitored after the software service definition is saved. A movable side bar with the preview of traffic statistics that vary depending on the selected traffic trace. The sidebar consists of the following sections: Web application traffic Shows which trace is currently selected and enables you to select another one from the list. If you configure the software service in a sequence of steps, the setting in this section is the same as in the previous wizard step: server selection. If you access the service configuration at a later time to edit it, the setting will be the one selected in the Guided Configuration when you last accessed the perspective. At any time, the setting can be changed. Statistics Lists statistics (based on the selected trace) that can be used to extract user identification information. Search Enables you to look for a specific string in the recorded traffic and copy the discovered parameters to the user name extraction rules. The automatic user name recognition method is based on a list of patterns often observed in traffic. If such a pattern is detected in the traffic sample that you selected for configuring a software service, it is listed and available for selection. Currently, there is one user identification pattern that DC RUM can automatically recognize: POST Identification Pattern for SiteMinder Users SiteMinder users are identified with the following set of parameters: USER parameter found in POST body SMIDENTITY or SMSESSION cookie name If no patterns have been found, or, if you prefer to configure user recognition manually, proceed as follows: 1. Specify where to search for the user identification. Perform this step only if you know where to search for the user identification. Use the Policy list to select one of following sources: POST GET Cookie Session cookie HTTP authorization tags Request header If you do not know the source for the user identification information, go directly to Step 2 [p. 71]. 2. Configure a user recognition method. 71

72 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Each user name recognition method requires that you specify a different set of extraction rules. Specify these rules in several ways: Type the required information in the boxes. Filter and copy the traffic data from the Statistics collapsible pane to the user name extraction rule, using the provided controls. Extract the user ID from the selected string. Extract an element (usually a cookie) that identifies the session. Specify a user login URL. Search for a specific string in the recorded traffic using the Search collapsible pane and copy the found parameters to the user name extraction rules. Type the search phrase in the Find box. The search is case sensitive. Use the In list to specify where you want to search for the string, select the Follow user IP address check box if you want to see all requests for a given user IP address in the search results (not only requests matching the search criteria), and then click Search. The search results are grouped into several categories, depending on the specified search criteria. For each result, you can display the request details and copy the parameters to the user name extraction rules using provided controls. At any time, you can click the Reset button to clear the form, or click Add policy to define more user name extraction rules. See the following topics for a detailed description of parameters required for each user name recognition method: Configuring User Recognition Method Based on HTTP POST [p. 72] Configuring User Recognition Method Based on HTTP GET [p. 73] Configuring User Recognition Method Based on Cookie [p. 74] Configuring User Recognition Method Based on Session Cookie [p. 74] Other Methods of Configuring User Name Recognition [p. 75] 3. Click Next to proceed to the software service definition summary. Configuring User Recognition Method Based on HTTP POST To configure user recognition based on HTTP POST, you need to at least specify the parameter from which you want to extract the user ID and the parameter from which you want to extract the session ID. Specify the parameter from which you want to extract the user ID. Provide the value of a specific POST parameter in the Extract user ID from field, or click next to a POST parameter in the Statistics collapsible pane to select it for user ID extraction. Specify the parameter from which you want to extract the session ID. 72

73 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic In the provided box, specify the cookie from which you want to retrieve the session ID, or click next to a specific cookie in the Statistics collapsible pane to use it as a source for session ID extraction. Optional: Provide the Login URL. This is a relative URL login path that is taken relative to the URL path specified in the Path field, in the Advanced section. Both strings, if concatenated, should give the full path to the login page. Specify the URL manually or click pane to use it as a login URL. Optional: Specify the acknowledgment URL (ACK URL). next to a specific URL in the Statistics collapsible This entry gives the relative path to the URL confirming a successful login, if such a URL is used. Successful logins are normally recognized by a SET COOKIE operation for the named cookie, though sometimes a confirmation login is used instead, and a cookie of the given name is sent at the same time. Optional: Specify the host server name. Optional: Provide the path information. A path is the leading part of a URL. It is used for recognizing the login URL and for matching the subsequent hits with the user ID. Hits with the same session ID but different paths are not matched with the user. If the path is not specified, the path is assumed to be / and it matches any requested URL. For example, if Path is specified as /a and Login URL is given as /login, the login hit is for /a/login and subsequently all hits beginning with /a are assigned to this user, provided they match the session ID. However, hits beginning with, for example, /b, are not assigned to the same user, even if the session ID matches. In this example, for such hits to be matched, you need to specify Path as / and Login URL as a/login. Configuring User Recognition Method Based on HTTP GET Specify the parameter from which to extract the user ID. Provide the value of a specific GET parameter in the Extract user ID from field, or click next to a GET parameter in the Statistics collapsible pane to select it for user ID extraction. Specify the parameter from which you want to extract the session ID. In the provided box, specify the cookie from which you want to retrieve the session ID, or click next to a specific cookie in the Statistics collapsible pane to use it as a source for session ID extraction. Optional: Provide the Login URL. This is a relative URL login path that is taken relative to the URL path specified in the Path field, in the Advanced section. Both strings, if concatenated, should give the full path to the login page. Specify the URL manually or click pane to use it as a login URL. next to a specific URL in the Statistics collapsible 73

74 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Optional: Specify a host server name. Optional: Provide the path information. A path is the leading part of a URL. It is used for recognizing the login URL and for matching the subsequent hits with the user ID. Hits with the same session ID but different paths are not matched with the user. If the path is not specified, the path is assumed to be / and it matches any requested URL. For example, if Path is specified as /a and Login URL is given as /login, the login hit is for /a/login and subsequently all hits beginning with /a are assigned to this user, provided they match the session ID. However, hits beginning with, for example, /b, are not assigned to the same user, even if the session ID matches. In this example, for such hits to be matched, you need to specify Path as / and Login URL as a/login. Configuring User Recognition Method Based on Cookie Specify the cookie from which to extract the user ID. Provide the value of a specific cookie in the Extract user ID from field, or click a cookie in the Statistics collapsible pane to select it for user ID extraction. Optional: Specify the host server name. Optional: Provide the path information. beside A path is the leading part of a URL. It is used for recognizing the login URL and for matching the subsequent hits with the user ID. Hits with the same session ID but different paths are not matched with the user. If the path is not specified, the path is assumed to be / and it matches any requested URL. For example, if Path is specified as /a and Login URL is given as /login, the login hit is for /a/login and subsequently all hits beginning with /a are assigned to this user, provided they match the session ID. However, hits beginning with, for example, /b, are not assigned to the same user, even if the session ID matches. In this example, for such hits to be matched, you need to specify Path as / and Login URL as a/login. Optional: Use regular expressions to extract the user names. The regular expression is applied to the cookie values that are found in the traffic trace. If a regular expression is not specified, the whole cookie value is used as the user name. For more information, see Using Regular Expressions to Extract User Identification [p. 75]. To test regular expression patterns, click Test. For more information, see Regular Expression Fundamentals [p. 211]. Configuring User Recognition Method Based on Session Cookie Specify the session cookie from which to extract the user ID Provide the value of a specific cookie in the Extract user ID from box, or click next to a specific cookie in the Statistics collapsible pane to select it for user ID extraction. Specify a regular expression to extract the user name from the HTTP header. 74

75 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic This expression is applied to the entire header and therefore can, for example, combine the values extracted from a number of cookies or parameters. These are then combined according to the bracketing you specify in the regular expression. Optional: Specify grouping order for the regular expression. In addition to the straightforward combination of bracketed portions of the regular expression, you can specify a different order in which the indicated portions of the regular expression should be combined. Optional: Specify the host server name. Optional: Provide the path information. A path is the leading part of a URL. It is used for recognizing the login URL and for matching the subsequent hits with the user ID. Hits with the same session ID but different paths are not matched with the user. If the path is not specified, the path is assumed to be / and it matches any requested URL. For example, if Path is specified as /a and Login URL is given as /login, the login hit is for /a/login and subsequently all hits beginning with /a are assigned to this user, provided they match the session ID. However, hits beginning with, for example, /b, are not assigned to the same user, even if the session ID matches. In this example, for such hits to be matched, you need to specify Path as / and Login URL as a/login. Other Methods of Configuring User Name Recognition In addition to the user name recognition methods based on POST, GET, cookie, and session cookie, you can also extract user names from request headers or HTTP Authorization tags. To monitor users based on the HTTP Basic Authentication Scheme, select the HTTP Authentication policy. This user recognition method does not require additional configuration tasks. If the Authorization HTTP header field is not found among the most common HTTP header fields in the selected traffic trace, you can try looking for this field in the whole traffic trace using the search tool. To configure user recognition based on a request header: Type a regular expression to extract a user name from the request header. Click in the Statistics section to select the header field from which to extract a user name. Using Regular Expressions to Extract User Identification When simple extraction of the cookie values or parameter values is not sufficient for configuration purposes, you can search the value for a sub-pattern that you define as a regular expression. The regular expressions used here conform to the Basic POSIX syntax. A full description of the syntax is widely available in numerous online and printed publications. The example below demonstrates a simple use of such a regular expression to extract a substring from a cookie value. For more information, see Regular Expression Fundamentals [p. 211]. Regular expressions are symbolic patterns with which you can specify a range of text patterns. A single regular expression can match a wide range of very different text strings. For example, 75

76 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic the expression. is a wildcard that matches any single character and the expression.* is a wildcard that matches any number of occurrences of any character (in effect, it matches anything). Regular expressions can be used for finding particular text strings and then extracting certain parts of those text strings: the parts that match a sub-expression. The sub-expression is surrounded by parentheses ( ). For example, the expression a(b.) finds all strings composed of three characters, of which the first one is a, the second one is b, and the third one is any character. It will then extract the second and third character. Note, however, that the match has to be based on the regular expression part outside the parentheses and the part inside the parentheses. In other words, the character a has to be found in the string, even though it is not extracted. Some of the more common regular expression symbols: period. Matches any character. asterisk * Matches repetition of the previous character zero or more times. plus sign + Matches repetition of the previous character one or more times. NOTE In basic regular expressions, the plus sign must be preceded by a backslash to prevent it from being considered a normal character to match. caret symbol ^ This symbol has a number of meanings, depending on the context. If it appears at the beginning of the expression, it means the beginning of the line or search string. If it appears as the first character in square brackets (see below), it means a negation. For example, [^@] means any character that is NOT "@". In other cases, this character is considered a normal character and matches itself. dollar sign $ Matches the end of the line or search string. square brackets [...] Group together symbols denoting a class of characters that is symbols that are to match a single character. For example, [a-z] stands for any lowercase alphabetical character and [^@] means a character that is not symbol. hyphen - Combined with square brackets (see above), a hyphen is used to specify ranges of characters. parentheses\(...\) Select that part of the parsed string that we want to extract. Note: In basic regular expressions, parentheses must be preceded by backslashes to prevent them from being considered normal characters to match. Example 5. Walk-Through Example of How to Read and Interpret a Regular Expression If a cookie name is defined as Pag and the cookie header line is as follows: Cookie: Pag=cf68603b@TXP293@ @D1R1wLLsMrjhw; 76

77 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic the cookie value is: cf68603b@txp293@ @d1r1wllsmrjhw Assuming that the substring to extract is positioned between the first and character in the cookie value string, it is: TXP293 The regular expression for extracting that substring can then be defined as: ^[^@]*@\([^@]\+\)@ In this particular case, the above regular expression can be understood as follows: 1. ^ means to find the beginning of the line. 2. [^@]* means to skip zero or more occurrences of any character that is 3. \( means that the string to extract is described by that part of the expression that is contained within parentheses. 4. [^@] means that the first character (after we found above) must not 5. \+ means that we want to extract this character and any other characters that follow it and that are also 6. \) marks the end of the expression describing the string we want to extract. means that the string to be extracted has to be terminated but we do not include the character in the extracted string, because it is outside of the parentheses. If the regular expression compilation fails, the following message is written to /var/log/adlex/rtm.log: ERROR: user name cookie regular expression is invalid: error where error is replaced by a specific error from the regular expression library function. Reviewing and Publishing a Software Service Definition Before a new software service definition is saved, verify that there is traffic observed that matches the configuration. 1. Verify the information listed on the Review summary screen. The screen shows the following information: The list of the selected servers with their IP addresses and port numbers. The defined user name recognition methods. The list of configured URLs. For servers and URLs, the list shows the traffic [hits/s] for each configuration; for each defined user name recognition method, you can see the number of occurrences in the traffic. To modify the software service definition, click Previous to return to a previous step in the wizard and make the changes. 2. Click Apply to save the new configuration. In the Save configuration pop-up window, choose one of the following options: 77

78 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Save the configuration without publishing it on the devices. Immediately publish the configuration on monitoring devices and report servers. To publish the saved draft configuration at a later time: a. On the Guided Configuration Home page, click the link in the Configuration status section. The Web/XML Software Services screen appears. b. Click Publish Configuration. What to Do Next After publishing the software service definition, if you notice new services in the traffic and want to monitor those services, update the configuration by editing the existing definitions. For more information, see Managing Software Service Definitions [p. 78]. Managing Software Service Definitions Use the Software services screen in the RUM Console to view and edit the software service definitions and software service rules. These directions apply to all of the software services, whether they were created with the wizard or created manually in the RUM Console. Note, however, that you must manually change settings such as URL auto-learning, custom metrics, and character encoding. Editing a Software Service Definition To edit a software service definition: 1. Select Software Services Manage Software Services from the console top menu. It opens the list of all defined software services. 2. To provide a new name for a software service, select Actions Rename in the row corresponding to the selected software service. 3. To monitor a software service with another device, on the Deployment tab, select the service with a single mouse click and click Change assignment. Editing a Software Service Rule To edit a software service rule: 1. Select Software Services Manage Software Services from the console top menu. It opens the list of all defined software services. 2. Select a software service definition by clicking it once. The software service rules being appear on the Configuration tab. 3. To modify the configuration of service, URL, and user monitoring, on the Configuration tab, select Actions Edit with wizard or Actions Edit manually (depending on your needs). 78

79 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic Adding a Rule to Existing Software Service To add another rule to the existing software service: 1. Select Software Services Manage Software Services from the console top menu. It opens the list of all defined software services. 2. Select a software service definition by clicking it once. The software service rules being appear on the Configuration tab. 3. To add another rule for the existing software service, on the Configuration tab, click Add rule. Defining Software Services Manually If you are familiar with your network and monitoring environment, you can define a new software service manually. Before You Begin It is assumed that you have the latest version of the AMD added and configured in the RUM Console. For more information, see Adding an AMD to the Devices List in the Data Center Real User Monitoring Smart Packet Capture User Guide. It is assumed that you have the latest version of the CAS added and configured in the RUM Console and connected to your AMD. For more information, see Adding a CAS to Devices List [p. 21]. To create a new software service definition without using application traffic statistics, perform the following steps: 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Click Add Software Service. The Add Software Service pop-up window appears. 4. Choose Manually from the list of options. It opens the Add Software Service pop-up window. 5. Specify the software service name. When the configuration procedure is complete, you can find the new software service under this name on the list of software service definitions. 6. Right-click anywhere in the Rules table and select Add from the context menu. The Rule Configuration pop-up window opens. 7. In the Rule description, provide a name that identifies this set of monitoring instructions. 8. Choose the AMDs to monitor the new software service. By default, the new definition is published on all available AMDs that are listed in the AMDs selected for software service pane. If you do not want a certain AMD to monitor a certain software service, clear the check box in the row corresponding to that device. 79

80 Chapter 5 Configuring AMD to Monitor HTTP and HTTPS Traffic 9. Proceed to the selection of monitored server and ports. At this stage of the configuration task, the new software service definition cannot yet be saved. To save it, you need to specify the IP addresses and ports of servers that you plan to monitor with DC RUM. For more information, see Selecting Services for Software Service Definition [p. 52]. 80

81 CHAPTER 6 Configuration Fine-Tuning Managing Devices DC RUM has two device types: Report servers and data collectors. The RUM Console provides full configuration capabilities for the CAS and ADS report servers and the AMD data collector. RUM Console support for the configuration of other devices may be limited. Devices can be software or hardware. The main data source type for DC RUM is the AMD. However, as a reporting engine, the CAS accepts data from Enterprise Synthetic, other CAS instances, and other sources that provide data in applicable formats. Use the RUM Console to assign data sources to a CAS. The Devices screen in the RUM Console displays all of the devices and the IP Address, Port, Description, Type, Version, Connection, Hardware Health, and Configuration. NOTE It is important to keep the devices synchronized for proper data interpretation. For more information, see Synchronizing Time Using the NTP Server in the Data Center Real User Monitoring Smart Packet Capture User Guide and Time Synchronization Between AMD and Server in the Data Center Real User Monitoring Administration Guide. Adding Devices in RUM Console Use the RUM Console to add and configure the data sources and report servers before you monitor traffic with DC RUM. Adding a Device 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Click Add Device. 4. From the Device type list, select the device type that you are adding. 81

82 Chapter 6 Configuration Fine-Tuning 5. In the Description box, type a description of the device. TIP It is recommended that you include the parent device name in the description of each device you add and to add these names consistently. This enables you to easily find your device in the list. Specifying the Connection Details 6. In the Device IP address box, type the device IP address. When you add a CAS or ADS installed on the same machine as the RUM Console Server, provide the device network address and not the localhost address, such as (IPv4) or ::1 (IPv6). 7. In the Port number box, type the port number for communication with this device. HTTP The standard port numbers used by servers and data collectors when communicating over HTTP are: Advanced Diagnostics Server or Central Analysis Server. Note that you must not add or edit the slave members working in the server clusters. LAN or WAN Probe Flow Collector Enterprise Synthetic Agent Manager BSM, the Connection Manager for Business Service Management Dynatrace Application Monitoring Server HTTPS The standard port numbers used by servers and data collectors when communicating over HTTPS (secure HTTP) are: 443 EndaceProbe and AMD 8021 Dynatrace Application Monitoring Server 8. Select Use secure connection if you want to use HTTPS (secure HTTP) for communication between the console and the device you are adding. Providing the Authentication Details 9. For ADS or CAS devices only: Choose whether to authenticate using the CSS. 10. Type the user name and password of the account used to manage the device. By default, the AMD user is set to compuware and the password is set to vantage. The credentials you enter for an AMD are used by the RUM Console to communicate with the device. They are also passed to report servers to collect monitoring data for processing. 82

83 Chapter 6 Configuration Fine-Tuning The authentication values are not equal to the values you use for logging in to the device via SSH or local console. Configuring the Advanced Settings 11. Select the Advanced options tab. 12. Optional: Under Secondary device connection, provide an alternative IP address for this device. 13. Enable an SNMP connection. Optional. If you are adding an AMD, defining the SNMP connection parameters enable you to obtain more detailed health information about this device. If you are adding another device type, the SNMP connection screen is not available. To define SNMP connection parameters: a. Select the SNMP Connection check box. b. Type the read community name and port number. 14. Enable Guided Configuration (AMD only). These settings apply only to the AMD. If you are configuring another device type, proceed to the next step. By default, the Guided Configuration connection is enabled when you add an AMD. For performance reasons, the number of AMD instances with Guided Configuration enabled is limited to 50. Any additional AMD will not feed data to the Guided Configuration perspective. Therefore, you cannot monitor data from the additional AMDs to generate Web traffic statistics or to define Web software services with a wizard. By default, the port number for communication between the CBA Agent and the RUM Console Server is set to 9094 and a secure connection is enabled. Generally, you will not need to modify this setting. If other services are using the default port number, you must type your new port number in the Port number box. Additionally, you have to manually change the port number setting on the CBA Agent side. For more information, see Modifying Connection Settings for Guided Configuration [p. 206]. 15. Click Next to test your connection parameters. If your configuration fails the test, you can go back and adjust your settings. If the device fails to respond correctly, it may take several seconds before the test times out. 16. Click Finish to save the configuration. Editing Device Connection Parameters You can edit the connection parameters of devices listed in the Devices table in the RUM Console. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Edit Connection from the context menu for the device. 83

84 Chapter 6 Configuration Fine-Tuning 4. Edit the connection parameters. For more information, see Adding Devices in RUM Console [p. 81]. 5. Click Finish to update the connection settings. NOTE Changes to the connection settings require an update to all of the references to this device. If a message informing you about the impending reference update is displayed, click OK to update all of the references to this device or click Cancel to apply the changes only to the connection that you are currently editing. Deleting a Device from the Devices List You can delete a device from the Devices list using the right-click context menu. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Delete Device from the context menu for the device to be deleted. 4. Click Yes to confirm the deletion. NOTE Changes to the connection settings require an update to all of the references to this device. If a message informing you about the impending reference update is displayed, click OK to update all of the references to this device or click Cancel to apply the changes only to the connection that you are currently editing. Integrating DC RUM with Dynatrace Application Monitoring Integrate DC RUM with Dynatrace Application Monitoring by adding and assigning devices and enabling the integration on the Dynatrace Application Monitoring Client. Before You Begin You should be familiar with DC RUM components and basic monitoring concepts. Refer to the Data Center Real User Monitoring Getting Started. You need to identify your monitoring goals. For more information, see Define and Prioritize Goals, Objectives, and Requirements in the Data Center Real User Monitoring Getting Started. The integration requires the following working components: The latest version of AMD Refer to the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. 84

85 Chapter 6 Configuration Fine-Tuning The latest version of RUM Console Refer to the Data Center Real User Monitoring RUM Console Installation Guide. The latest version of CAS Refer to the Data Center Real User Monitoring Central Analysis Server Installation Guide. The latest version of ADS Refer to the Data Center Real User Monitoring Advanced Diagnostics Server Installation Guide. Dynatrace Application Monitoring Server and the access to Dynatrace Application Monitoring client. Make sure that default ports are available for communications between the individual DC RUM components. For more information, see Network Ports Opened for DC RUM in the Data Center Real User Monitoring Administration Guide. If you plan to monitor SSL traffic, first, configure SSL decryption. Refer to the Data Center Real User Monitoring SSL Monitoring Administration Guide. the DC RUM and Dynatrace Application Monitoring integration requires a typical DC RUM web application monitoring setup with the latest versions of the AMD, CAS, and ADS as required components. Configuring the Devices 1. Adding an AMD The AMD is the main data source (Data Collector) for DC RUM; it collects and presents the monitored data to DC RUM report servers for analysis and reporting. At least one AMD must be added to the list of devices in the RUM Console. For more information, see Adding an AMD to Devices List [p. 19]. 2. Adding a CAS The CAS is the main report server for DC RUM. The CAS the data provided by the AMD and its monitoring and alerting mechanisms identify, track, and report on issues affecting the security, performance, and reliability of your services. Add at least one CAS to the device list and configure its connection with the AMD. Adding a report server to a list of devices is similar to adding an AMD. For more information, see Adding a CAS to Devices List [p. 21]. 3. Adding an ADS The ADS analyzes key transactional application protocols and delivers definite answers to individual user problems regarding the performance and errors of business-critical front-end and back-end applications. Add an ADS to the device list and configure its connection with the AMD. Adding a report server to a list of devices is similar to adding the AMD. For more information, see Adding ADS to Devices List [p. 22]. 4. Adding an ADS to the CAS workflow 85

86 Chapter 6 Configuration Fine-Tuning You can access ADS reports seamlessly from the CAS, provided that the servers work in a farm. An ADS can exist with a CAS only in a server farm in which the CAS is the master server and all ADS servers are slaves. Click Add ADS to CAS reporting flow and use the Guided Configuration dialog or perform the configuration on the master CAS. 5. Assigning an AMD to a CAS and ADS Click Assign AMD to CAS and ADS to set the AMD as a data source for the report servers. 6. Adding the Dynatrace Application Monitoring server Click Add Dynatrace Application Monitoring Server to add it to the devices list. The default port for the Dynatrace Application Monitoring server is 8020 when communicating over HTTP or 8021 when communicating over HTTPS. 7. Assigning Dynatrace Application Monitoring to a CAS Set Dynatrace Application Monitoring Server as one of the data sources for the CAS. Click Assign Dynatrace Application Monitoring to CAS and follow the Guided Configuration provided dialog. 8. Enabling integration in Dynatrace Application Monitoring Client Log on to the Dynatrace Application Monitoring Client and enable integration on the Dynatrace Application Monitoring side. For more information, see Configuring Dynatrace Application Monitoring to Receive Performance Data from DC RUM [p. 86]. 9. Verify the traffic monitoring quality and completeness You can verify traffic quality and completeness before the actual monitoring begins. Sniffing point diagnostics allows you to perform pre-monitoring tasks without the need of accessing the AMD console and executing a series of Linux commands which usually serve the purpose of validating AMD physical installation and connection. For more information, see Verification of Traffic Monitoring Quality [p. 25]. Configuring Dynatrace Application Monitoring to Receive Performance Data from DC RUM To integrate Dynatrace Application Monitoring and DC RUM, you must configure your Dynatrace Application Monitoring installation to receive performance data from DC RUM. 1. Open the Dynatrace Application Monitoring Client. 2. Open the sliding panel on the left side of the screen and expand the System Profiles list. 3. Right-click the system profile for which to enable integration with DC RUM and select Edit System Profile from the context menu. 4. In the system profile panel, click Integration. a. Select the Enable DC-RUM check box. b. In the URL box, type the IP address of your AMD. c. In the User and Password boxes, provide user credentials. d. Optional: Click Test connection to see whether your configuration is successful. 86

87 Chapter 6 Configuration Fine-Tuning Figure 9. Enabling the Integration on the Dynatrace Application Monitoring Client 5. Confirm the changes. 6. Select Settings dynatrace Client. 7. Select Services. 8. Enable (if disabled) non-secure connections on port Optional: Configure the Dynatrace Application Monitoring User Experience Management if you intend to use the UEM data. See the APM Community article for more information. Configuring the DPN Connection in RUM Console To import Dynatrace Performance Network tests to DC RUM, use the RUM Console to configure the connection with the DPN service. Before You Begin DC RUM and DPN integration requires a typical Web Application Monitoring setup with the following DC RUM components installed and running: AMD. RUM Console. CAS. Optional: ADS. To add the DPN connection to CAS: 87

88 Chapter 6 Configuration Fine-Tuning Adding the DPN Account 1. Start and log on to RUM Console. 2. Select Devices and Connections DPN Accounts from the top menu. The DPN accounts screen appears. 3. Click Proxy Settings to specify the proxy if connecting to the Internet from behind the firewall. 4. Click Add. Specify the user authentication information (name and password) for your connection. The credentials you enter here will be used by the RUM Console to communicate with the DPN and will also be passed to report servers so that they can collect monitoring data for processing. 5. Click Next to test your connection parameters. If your configuration fails the test, you can go back and adjust your settings. 6. Click Finish. Assigning DPN to CAS 7. Enable the DPN data feed for selected CASes. After you add the DPN account, the CAS Assignments table appears in the bottom of the screen. The table lists all the CASes managed by your RUM Console. Select the box in the Data feed column to assign the selected DPN account to the CASes of your choice. 8. Publish configuration Click Publish configuration to complete the assignment. You can verify the assignment by checking the Data feed status column. After you add the DPN account to the CAS, you will see the Synthetic Overview item in the CAS reports menu and the Synthetic Backbone tan in the Synthetic Overview report. For more information, see Synthetic Backbone Overview in the Data Center Real User Monitoring Central Analysis Server User Guide. URL Auto-Learning URL auto-learning enables you to define the set of URLs appearing in per-url reporting statistics without the need to manually define each URL. URLs that are found frequently are learned and reported. Configuring URL Auto-Learning URL auto-learning can be configured globally for all HTTP software services or configured for an individual user-defined software service. You can also have a separate global configuration for default HTTP and HTTP Express analyzer based software services. 88

89 Chapter 6 Configuration Fine-Tuning Before You Begin It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. Configure Global Settings 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Select Global Front-End Monitoring Web HTTP URL Auto-Learning or to Global Front-End Monitoring Web HTTP Express URL Auto-Learning, depending on the analyzer you use to monitor the HTTP services. 6. Select the Enable URL auto-learning check box to enable auto-learning for all services based on this protocol. 7. Define the size of reported URLs pool. If necessary, you can change the default size of reported URLs pool. The pool is shared among all monitored servers. The auto-learning algorithm aggregates the loads of all URLs for all servers - the server IP, port or any other attribute of the server is not taken into account by the auto-learning algorithm. Any member of the pool will be reported for all servers, regardless the activity on individual servers. 8. Adjust the auto-learning algorithm. Click Advanced settings to show the properties so you can adjust the behavior of the auto-learning algorithm. For more information, see Details of the URL Auto-Learning Algorithm [p. 90]. 9. Specify whether URL auto-learning is to be limited to synthetic agents. Because the HTTP Express analyzer does not support synthetic agent recognition, this option is not available for HTTP Express. 10. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Optional: Configure the Settings at the Software Service Level 11. From the top menu, select Software Services Manage Software Services. 12. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 89

90 Chapter 6 Configuration Fine-Tuning 13. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 14. Click the URL Auto-Learning tab. 15. Enable or disable URL auto-learning. A user-defined software service has the following options: Off To turn URL-auto learning off for this service. Global Settings To use global settings for all services based on this protocol. Custom Settings To specify custom values for URL auto-learning settings for this software service. All To monitor all URLs for this software service. The option is not supported by the HTTP Express analyzer. 16. Optional: Adjust the auto-learning algorithm. This is only possible if you select Custom Settings in Step 15 [p. 90]. Click Advanced settings to show the properties so you can adjust the behavior of the auto-learning algorithm. 17. Click OK to save the configuration. Details of the URL Auto-Learning Algorithm The AMD uses a URL auto-learning algorithm to choose and report the most popular URLs observer in monitored traffic. The AMD uses a pool of auto discovered URLs. It consists of members and candidates. The auto-learning algorithm aggregates the loads of all URLs for all monitored servers. The server IP address, port or any other attribute of the server is not used by the auto-learning algorithm. If a URL becomes the member of the pool, it is reported for all servers, regardless the activity on individual servers. URLs are inserted into the list of candidates and the list of members and moved between the lists according to the configuration parameters set in the Advanced setting section of the URL Auto-learning screens in the RUM Console. The AMD removes the URLs from the member list every specified interval, as controlled by the Cleanup interval property. Use the Percentage of new URLs property to control the portion of the members pool to be freed at the beginning of each interval and reserved for new highly active URLs. A candidate URL becomes a pool member if it is more popular than the portion of members defined by the Page loads threshold property. If you want to report URLs from all servers in the farm, regardless of the individual host name, you can deselect Use host name to exclude host names from the URL auto-learning algorithm. 90

91 Chapter 6 Configuration Fine-Tuning Enable the Slow page weight property to ensure that slow operation URLs with high volume loads are included in the auto-learning algorithm. To avoid a situation when no URLs are reported for software services with very little traffic, you can set the rules at the any software service level. In this way, you create a software service level pool with lower limits, making sure its URLs are reported. You can create separate pools within a single software service based on a number of servers. This way, you ensure the URLs monitored on a server with a lower traffic do not have to compete with URLs from a much larger server in terms of volume. You achieve this by assigning servers to groups within a single software service which translates to separate pools. Use the Group name option in the Service Details dialog. For more information, see Configuring Rules for User-Defined Software Services in the Data Center Real User Monitoring Citrix/Windows Terminal Services Monitoring User Guide. You can also create separate monitoring pools for default software services based on HTTP and HTTP express analyzers, both globally, and the software service level, by enabling URL auto-learning for default software services. URL Auto-Learning Diagnostics To monitor the performance of the URL auto-learning engine, log on to the AMD using an SSH connection and input the rcon command. To assess how many URLs are monitored, counters are maintained and can be displayed with the rcon command: show status The following is an example of the relevant portion of the output produced by this command. URI discovery status: Global Http Pool: monitored=6/600 candidates=0/2400 candidatespromoted=0 candidatesremoved=0 contenders=0 contendersadded=0 contendersremoved=0 contenderspromoted=0 Default Apps Http Pool: monitored=0/700 candidates=0/2800 candidatespromoted=0 candidatesremoved=0 contenders=0 contendersadded=0 contendersremoved=0 contenderspromoted=0 Express Http Pool: monitored=0/500 candidates=0/2000 candidatespromoted=0 candidatesremoved=0 contenders=0 contendersadded=0 contendersremoved=0 contenderspromoted=0 Application pools: 1 monitored=6 monitoredmax=500 candidates=0 candidatesmax=2000 candidatespromoted=0 candidatesremoved=0 contenders=0 contendersadded=0 contendersremoved=0 contenderspromoted=0 The reported counters are displayed in the table below: 91

92 Chapter 6 Configuration Fine-Tuning Table 3. Status Counters Counter monitored candidates candidatespromoted candidatesremoved contenders contendersadded contendersremoved contenderspromoted Meaning The number of monitored URLs in pool and a maximum allowed value as set in Number of reported URLs. The number of additionally tracked URLs, candidates to become monitored URLs and the limit as set in Size of candidates pool. The number of URLs promoted to being monitored. The number of candidate URLs that were tracked by the AMD, but were removed due to URL auto-learning advanced configuration. The number of URLs, neither monitored, nor tracked, seen by AMD in current short interval. The total number of contenders added. The total number of idle contenders removed. The total number of contenders promoted to the pool of tracked URLs. To see the details of the current URL auto-learning pool, use the command show urls from the AMD console. The command can be executed without parameters, and it displays all of the information for all of the URLs. When the command is executed with parameters, use the following syntax: show urls [all] < group name > < software service name > < url pattern > all When specified, the output contains a list of contenders URLs. group name Quoted string, only the specified group is included in the result. application name Quoted string, only the matching software service will be included in the result. url pattern Quoted string, only URLs that match specified pattern are in the output. It is a simple substring match, for example Not all of the parameters have to be specified. The valid shorter versions of the command are: show urls show urls [all] group show urls [all] group application show urls [all] group application pattern If you do not want to specify a certain parameter, but it is required by the syntax, the value any can be used. For example, to display URLs for Example Software Service software service, you can use the following command: show urls any Example Software Service There are predefined values for the software service name parameter that can used to display the URLs monitored as a result of URL auto-learning configuration for default software services 92

93 Chapter 6 Configuration Fine-Tuning set for HTTP and HTTP Express analyzers at the general and software service level. GLOBAL_HTTP, DEFAPP_HTTP, GLOBAL_LIGHT_HTTP and DEFAPP_LIGHT_HTTP. Dimensions, Metrics and Attributes in HTTP Monitoring You can define custom metrics to monitor certain types of measurable data that is specific to your network environment or software. You can analyze and categorize operation attributes (text entities retrieved from requests and responses of a web application operation). You can extract miscellaneous parameters from the request or response body. You can also define grouping attributes by specifying the rules to be applied to the request URL or response body. A custom metric is a non-standard metric you can use to count values specific to your web application. Custom metrics can be, for example, the number of items sold or the total value of a transaction. The values are reported as user-defined metrics on the report server. Extracting an operation attribute, which is a text entity retrieved from the requests and responses of a web application operation, can help to diagnose and report specific events or errors caused by end-user actions. Miscellaneous parameters are text strings available in the URL request or response body. You need to define recognizable text patterns conveying the miscellaneous parameters that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. Miscellaneous parameters, unlike parameters extracted for URLs with parameters, are not defined together with an accompanying URL. When extracting Miscellaneous parameters, only the initial hit triggering the web page load is taken into account. The extracted Miscellaneous parameters must not be longer than 1030 bytes. Grouping attributes are text strings available in the URL request or response body that uniquely identify clients. You need to define recognizable text patterns conveying the grouping attributes that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. In order to define dimensions, metrics or attributes that you wish to extract and monitor, follow these configuration steps: 1. Open the Dimensions, Metrics, Attributes tab for the service. 2. Add or open a definition to be monitored and reported in a specific way. In the Dimensions, Metrics, Attributes table, right-click and choose Add to create a new definition, or Open to open an existing definition. The Dimensions, Metrics, Attributes window will open. 3. Choose how the value should be reported Custom metric You can define up to five custom metrics to monitor certain types of measurable data that is specific to your network environment or software. Use this mechanism if you want to obtain non-standard measurements extracted from the HTTP, XML, or SOAP traffic. A custom metric is a non-standard metric you can use to count values specific to your web application. Custom metrics can be, for example, the number of items sold or 93

94 Chapter 6 Configuration Fine-Tuning the total value of a transaction. The values are reported as user-defined metrics on the report server. Custom metrics can be defined on the level of: software service URL URL parameters The number of metrics for each level is globally limited to five. Choose the custom metric level that matches the characteristics of the traffic to monitor. For example, to monitor an HTTP software services in which URL monitoring is not deployed, define the custom metric for the software service custom metric level. However, to define URLs and URL parameters, use the appropriate custom metric levels for each. The level you choose should directly relate to where the information to monitor can be found Also consider performance issues when choosing a custom metric level. For example, if you define custom metrics for a software service globally, the rule will be applied to all URLs that the analyzer processes, and could possibly negatively impact the performance of the AMD. This may be unnecessary if you only want to extract this type of data only from two types of URLs, in which case you can define the rule at the URL or even URL parameter level. The custom metric values are collected during traffic monitoring and can be configured in the RUM Console for the HTTP and transactional software services (SOAP and XML). By default, the new metric names are derived from the field name in an HTTP or XML request. These names can be changed later on the report server for easier identification. The custom metric values are presented by the report server in dedicated columns that show the number of occurrences and the sum and average values. Grouping attribute Grouping attributes are text strings available in the URL request or response body that uniquely identify clients. You need to define recognizable text patterns conveying the grouping attributes that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. Note that the rules defined for URL or URL with parameters have a higher priority than those defined at the software service level. Miscellaneous parameter Miscellaneous parameters are text strings available in the URL request or response body. You need to define recognizable text patterns conveying the miscellaneous parameters that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. Miscellaneous parameters, unlike parameters extracted for URLs with parameters, are not defined together with an accompanying URL. When extracting Miscellaneous parameters, only the initial hit triggering the web page load is taken into account. The extracted Miscellaneous parameters must not be longer than 1030 bytes. 94

95 Chapter 6 Configuration Fine-Tuning When defining the rule, you can use one of six methods to search for the parameters in the request URL or you can also use one additional method to search for the dimension in the response body. Note that the rules defined for URL or URL with parameters have a higher priority than those defined at the software service level. Operation attribute Configuring extraction of request operation attributes is almost identical to the process for custom metrics, the main difference being that the custom metrics functionality is used to extract and report numeric values, while the request operation attributes relate to textual data. 4. Choose where the value should be reported The options available for this step depend on the selection made in the Choose how the value should be reported option. Custom metric (1, 2, 3, 4, 5) Select a metric category from 1 through 5, where each number corresponds to one of five custom metric categories. The values extracted will be reported by CAS in the same custom metrics category. Grouping attribute (1, 2, 3) You can define up to three grouping attributes. Each matching rule in each of the set of rules is taken into account regardless of its order, but the rules applied to the HTTP requests always takes precedence over the response rules and it is not advised to use both of them to extract one attribute. The extracted Grouping attributes should not be longer than 255 bytes. The order of entries is irrelevant because each matching hit is used to extract the grouping attributes. Miscellaneous parameter (1, 2, 3, 4, 5, 6, page name) You can define up to six parameters. The number of available Miscellaneous parameters, however, is limited by the number of defined URL with parameter definitions at the URL level. For example, if you use up all four parameter definitions available for a particular URL, you can define only two more Miscellaneous parameters. Operation attribute (1, 2, 3, 4, 5) Select a category from 1 through 5, where the selection from the dropdown list corresponds to one of five operation attribute categories. Values extracted will be reported by the CAS in the same category. 5. Choose where to search for the value You can retrieve the values from a number of entities: Request or Response Headers Request or Response Body Request parameter 6. Apply additional search and transformation rules. Choose one of the available search methods to detect the values: 95

96 Chapter 6 Configuration Fine-Tuning Add prefix Use this method if you expect the value to always be preceded by a specific prefix. To extract the value, provide the prefix expected to precede the value. Cookie name search Specify the cookie from which to extract the value. Provide the value of a specific cookie name confirming a successful login. The session ID, for mapping to the value, is extracted from this cookie. Successful logins are normally recognized by a SET COOKIE operation for the named cookie Decode / decompress If you expect to perform a search on a compressed or encoded data, or URL encoded in case of URL parameters, you can bring the search results to a human readable form by using one of available decoders, Base64, Base64 + Gzip, Gzip or URL encoding. You can also extract parts of your initial search results by using Text search or Regular expression search methods. Mime encoded list filter Use this method if you expect to find a value in an MIME format. Including text in character sets other than ASCII, message bodies with multiple parts and in header information encoded in non-ascii character sets. NTLM search Use this method to search for a value in an NTLM authentication request header. Depending on your choice, the value can be composed of the following fields: workstation, domain, or user. Select the fields that compose an identified value and, if necessary, change the default character used to separate the selected components in the resulting value. Parameter name and value search Use this method if you expect the value to always be carried by the specific parameter. To extract the value, provide the parameter name. Depending on the selected search scope, the term parameter may refer to a specific entity, such as a cookie name (when the search scope is set to cookie), or a header field (when the search scope is set to request or response header). Parameter name prefix search Use this method if you expect the value to always be carried by a specific parameter with a specific prefix. To extract the value, provide the parameter name prefix and indicate what data should be reported. The results of the search can be presented as a parameter name and the value, just the parameter value or just a parameter prefix. Parameter value suffix search Use this method if you expect the value to always be carried by the specific value of a parameter with a specific suffix. To extract the value, provide the value for the suffix. Regex search You construct a regular expression that, when applied to a selected search scope, returns the value. The regular expression must contain at least one group enclosed in parentheses. If the regular expression returns a number of search groups, you can define the custom group order by entering a comma-separated list in the order of your choice (for example, 2,1,3). This method is not available for the cookie and response 96

97 Chapter 6 Configuration Fine-Tuning body search scopes. For more information, see Regular Expression Fundamentals [p. 211]. You can test the patterns that will be used by the AMD using the Regular Expressions Test tool, which is activated after you click Test located next to the regular expression pattern field. For more information, see Testing Regular Expressions [p. 213]. Example 6. The following is an example of extracting the value of REMOTE_ADDR field from the HTTP header. An HTTP header might contain the following information: GET HTTP/1.1 Accept: */* Referer: Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: Connection: Keep-Alive Cookie: FPB=061j8hura11q56cv; CRZY9=t=1; REMOTE_ADDR: The following regular expression extracts the address from the REMOTE_ADDR field: %0d%0aREMOTE_ADDR:%20\([^%0d%0a]*\)%0d%0a The expression must contain a single sub-expression delimited by pairs of characters \( and \). The expression in this example states that the search string should start at the beginning of a header line and end at the end of the line (note the use of % to denote the hex values of the carriage return and line feed characters). The line should start with the string REMOTE_ADDR:. The sub-expression to extract is a string of characters different than ASCII CR or LF, and it should occur after the space following REMOTE_ADDR: Text phrase search Use this method if you expect the value to always be found in the text. The provided value for the search parameter will be used to match the text phrases in the analyzed traffic. Text search Use this method if you expect to find a value between the first occurrences of strings defined by Match start and Match end. Because it is not always possible to extract the value directly, you can use this method as a first step in preparing content for search result transformations. You can set a Search limit in bytes to avoid lengthy search results. This method is not available for the cookie search scope. 7. Advanced settings Define the matching based on pattern or absence of a pattern. a. Select a condition Pattern presence The response for a defined category is reported if a given pattern was detected, which is the default setting. 97

98 Chapter 6 Configuration Fine-Tuning Pattern absence The response for a defined category is reported if a given pattern was not detected. b. Enter a Host Pattern. Enter the pattern to match in the host field in HTTP requests. The pattern should consist of a case-sensitive string that is expected to be found in the host name and may also contain an optional wild-card character * to signify any number of any characters. If spaces are included in the pattern, the pattern must be enclosed in a pair of double or single quotation marks. Otherwise, quoting the pattern is optional. NOTE The eligible hosts are selected by limiting the size of the host group to the one defined by the narrowest condition. In other words, for a particular sample of monitored traffic data, if one host pattern defines a set of hosts that is included in the set of hosts defined by another pattern, a match will be attempted on the smaller group first. If the monitored traffic data does match the application response definition for the smaller group of hosts, there is no attempt to match the same traffic data to the application response definition for the larger set. For example, the pattern *.abc defines a larger set of hosts than the pattern *.myhost.abc. In this case, for any given sample of monitored traffic data, first, an attempt to match it to the response definition for *.myhost.abc and, if successful, there is no attempt to match it to the response definition for *.abc. Within a given host group, all path patterns that match are taken into consideration while searching for responses. To ensure meaningful results, no two different patterns defining host names should be matched by a single host, except for the pattern *. This means that the same patterns can repeat in the configuration file, but for any two different patterns, the search will not find a host that matches both of them. For example, if there are two patterns such as *t* and *u*, the host names Jupiter and Saturn both match both of the patterns because both of the names contain the letters u and t. So, if you are monitoring two such hosts, modify your pattern so that no host matches both of them. However, if there are no such hosts in the monitored data, the above pattern will cause no problems. Similar requirements apply to the patterns for paths and response pattern text. c. Enter a Path Pattern. Enter a pattern to match the path field in HTTP requests, after removing from it the leading portion, up to and including the host name. The pattern should consist of a case-sensitive string that is expected to be found in the path and it may also contain optional wild-card characters *, to signify any number of any characters. Note that if spaces are included in the pattern, the pattern must be enclosed in double or single quotation marks. Otherwise, quoting the pattern is optional. For example, if the path in the HTTP request is 98

99 Chapter 6 Configuration Fine-Tuning enter /sales/eg/index.jhtml. d. Match only when response has one of the following HTTP status code Match only when response has one of the indicated HTTP status codes. The HTTP status codes can be defined by providing the HTTP status code range. Use the official HTTP status codes to narrow down qualifying responses. 1xx - Informational 2xx - Successful 3xx - Redirection 4xx - Client error 5xx - Server error Excluding Elements from Orphaned Redirects Reporting For each software service, you can define elements to exclude from reporting as an orphaned redirect. Before You Begin It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. The feature is available only in HTTP analyzer mode. For more information, see Choosing HTTP Analyzer Mode [p. 137]. If you do not want particular redirects to be reported as orphaned, exclude them from orphaned redirects reporting by defining strings describing these elements. You can define the elements to be extracted from the Location field of the HTTP header or the request URL. 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Navigate to the Orphaned redirects tab. 6. Add or edit the elements to be extracted from the Location field of the HTTP header or the request URL. 99

100 Chapter 6 Configuration Fine-Tuning Right-click a row in the Location suffixes or Request URL tables and choose Add to create a new definition, or choose Open to open an existing definition. You can create a list of elements or you can also construct a regular expression to be applied to the URL to extract the element. Because the regular expression must return a value, make sure that it contains parentheses. When supplying elements to the Location suffixes or Request URL tables, you must provide the last URL element defining the file. For example, if an element that you want to exclude from reporting as an orphaned redirect is identified by the you need to add the footer.gif string. 7. Click OK to save the configuration. 8. Publish the draft configuration on the monitoring device. Synthetic Agent and Browser Recognition A synthetic agent is a simulator of user traffic to a given web site. Synthetic agents are usually distributed over a number of different geographical site and are designed to measure website availability and performance. Synthetic agent traffic is recognized and treated differently than real user browser traffic. Recognition of the following synthetic agents and browsers is supported: KTXN Envive Gomez Keynote Mercury SiteScope Firefox Internet Explorer Chrome Opera Safari Configuring Synthetic Agents, Browsers, Operating System and Hardware Recognition In addition to the pre-configured agents, browsers, and operating system, you can define further recognition based on the strings extracted from the User-Agent field of the HTTP request-header. To manage the list of supported synthetic agents, browsers, operating systems, and hardware: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 100

101 Chapter 6 Configuration Fine-Tuning 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Navigate to Global Front-End Monitoring Web HTTP Browser Recognition to manage the list of browsers and synthetic agents. 5. Right-click in the Browser Rules table and select: Add Browser to define a new synthetic agent or browser. Add Pattern to Browser to add an additional pattern to match an agent or browser. Open to modify an existing agent definition or browser. Delete to delete an existing agent definition. 6. Add or edit an agent or browser definition pattern. Right-click the Browser Recognition table. The Edit Browser dialog appears. a. Choose a user agent type. If you are adding a new user agent, select whether it is a browser or synthetic agent. b. Provide a user agent name to be used in the reports in case of a successful match. If you edit a synthetic agent, you have to provide its predefined identity (in the 50 to 150 range). c. Specify Hit Session Timeout. For each user agent on the list, you can specify a separate value for Hit Session Timeout, overriding the value configured globally for all software services. Maximum time delay allowing a hit to be linked to a page. The value is specified in seconds, with a resolution of one-tenth of a second, and is configured globally for all software services. Synthetic agents might require a higher Hit Session Timeout value because, unlike real users, they always load the full page contents and all of its elements. Real users, especially if they access frequently visited page, load many page items from the browser cache instead. d. Define the search pattern. In HTTP communication, the User-Agent request-header field contains information about the user agent originating the request. You can use either a simple pattern or a regular expression search. If you use a simple pattern, the string you enter is compared to the User-Agent field found in the HTTP header. A match occurs when the whole defined string has been found anywhere in the field. Because the User-Agent field may contain a longer string, you can choose to use a regular expression constructed to match a variety of strings falling into a category of your interest. 7. Navigate to Global Front-End Monitoring HTTP Operating System Recognition to manage the list of operating systems. 8. Add or edit an operating system definition pattern. Right-click the Operating System Recognition table to open the Operating System Rule dialog. In the Name field, provide an operating system name to be used in the reports in case of a successful match. When defining the search pattern, you can use either a simple pattern or a regular expression search. 101

102 Chapter 6 Configuration Fine-Tuning 9. Navigate to Global Front-End Monitoring HTTP Hardware Recognition to manage the list of hardware. 10. Add or edit a hardware definition pattern. Right-click the Hardware Recognition table to open the Hardware Rule dialog. In the Name field, provide a hardware name to be used in reports when there is a successful match. When defining the search pattern, you can use either a simple pattern or a regular expression search. 11. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. What to Do Next To disable synthetic agent and browser recognition, navigate to Global Front-End Monitoring Web HTTP Browser Recognition and deselect Enable User Agents Detections. Save and publish the changes. Synthetic Agent Recognition Based on Contents of HTTP Header For a new agent to be recognized automatically based on the contents of the HTTP header, configuration is necessary both on the AMD device that collects traffic data and on the report server. For more information, see Configuring Synthetic Agents, Browsers, Operating System and Hardware Recognition [p. 100]. New agent definitions on the report server are added in the Protocols/User Agents screen of the Diagnostic Console invoked by the following address : 102

103 Chapter 6 Configuration Fine-Tuning Figure 11. Example Protocols/User Agents Screen To define a new agent, fill the input fields in the bottom portion of the screen and click Add. You need to enter the following information: ID The agent ID number as defined on the AMD, but preceded with the minus sign. All synthetic agent IDs are entered as negative numbers. User Agent Name The name of your choice. This is the name under which the agent will appear on reports. Protocol This should always be TCP. Real/Synthetic This should always be Synthetic. Synthetic Agent Recognition Based on User Name or IP Address Recognition of synthetic agents can also be performed based on the IP address or user name. This may be used if no relevant information can be extracted from the HTTP header. To configure this type of synthetic agent recognition, select the menu item Settings Report Settings User-Protocol Mapping. 103

104 Chapter 6 Configuration Fine-Tuning This configuration screen enables you to set the PROTOCOL_MAPPING configuration property, which specifies mappings of user names or client IP addresses to special protocol IDs of synthetic agents. The syntax of the property value field is: username1=procotolid_1; username2=protocolid_2;...; ip:ipaddress1=procotolid_11; ip:ipaddress2=procotolid_12 The prefix "ip:" indicates that the client IP address is being mapped; otherwise, the user name is assumed. The wildcard character * can be used in user names or IP addresses. protocolid can be a number or a name of an agent. Names are not case sensitive. The screen lists pre-defined agent names and numbers. Example 7. Specifying synthetic agent mapping to user name or IP address topaz = -52 ; abcdef* = -53 ; ip: = Gomez ; ip: * = Keynote NOTE Every protocol ID used in User-Protocol Mapping should also be defined in the Protocols User Agents configuration screen, or protocol IDs may be reported incorrectly. End-of-Page Components For each HTTP (HTTPS) software service, URL, or URL with parameters, you can define an end-of-page component identified by its URL. Loading this component indicates that the page is complete; no further elements are taken into account when calculating metrics for the page. Before You Begin It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. You can define the end-of-page components at the level of software service, URL and URL with parameters. If you decide to define the end-of-page components at more than one level, the definitions are complementary to one another and not overriding definitions at other levels. For each level, you can either provide a single regular expression that will match the URL of a component or provide a static list of components. To define end-of-page components: Configure settings at the software service level The definitions at the software service level are complementary to the definitions at other levels and do not override them. 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. 104

105 Chapter 6 Configuration Fine-Tuning Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Switch to the End of Page Components tab. 6. Define end-of-page components. This can be done in two ways: Provide an Element regular expression to match the end-of-page component. You can define a single regular expression that will match the end-of-page component. For example, the expression will match all the footer.png files downloaded from any of the section of the website, for example: Provide Static page components definitions. You can also add a number of static definitions, listing all the necessary end-of-page components across the monitored operations. To add the end-of-page components, right-click a line in the Component column and choose Add. You must enter a fully qualified URL of the component starting from the or strings. 7. Click OK to save the configuration. 8. Publish the draft configuration on the monitoring device. Configure settings at the URL level The definitions at the URL level are complementary to the definitions at other levels and do not override them. 9. From the top menu, select Software Services Manage Software Services. 10. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 11. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 12. Switch to the URL Monitoring tab. 13. In the URL definitions table, right-click a specific URL and select Open from the context menu to open Monitored URL window. 105

106 Chapter 6 Configuration Fine-Tuning To quickly navigate to an entry in the URL definitions table, click in the table and then type some or all of the IP definition. Click the magnifying glass icon or press [Ctrl+F] to open a search box to limit the table view to only those rows that contain a match (in any column) to the search string. 14. Repeat Step 5 [p. 105] and Step 6 [p. 105]. 15. Click OK to save the configuration. 16. Publish the draft configuration on the monitoring device. Configure settings at the URL parameters level The definitions at the URL parameters level are complementary to the definitions at other levels and do not override them. 17. From the top menu, select Software Services Manage Software Services. 18. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 19. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 20. Switch to the URL Monitoring tab. 21. In the URL parameter group table, right-click a specific parameter group and select Open from the context menu. This will open the URL Parameter Group window. 22. Repeat Step 5 [p. 105] and Step 6 [p. 105]. 23. Click OK to save the configuration. 24. Publish the draft configuration on the monitoring device. Automatic Page Name Recognition URL strings appearing on reports can be very long and difficult to read, but you can specify URL names to use instead. You can either add a static page name, or you can configure the AMD to retrieve the page name automatically from the HTML body of the HTTP response page or from the request URL. Before You Begin It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. The feature is available only in HTTP analyzer mode. For more information, see Choosing HTTP Analyzer Mode [p. 137]. 106

107 Chapter 6 Configuration Fine-Tuning Automatic page name recognition can be configured at the software service level, at the URL level, or for a URL with parameters. However, this configuration will work only for URLs and parameter groups if explicitly assigned page names, as described in Configuring URLs for a Software Service Definition [p. 54], have not been set to have a higher priority. If a URL matches the criteria you have specified in the page name extraction definition and an assigned static page name is not set to have a higher priority, the AMD will attempt to retrieve the page name automatically from the HTML body of the HTTP response or the request URL. Because the same URL or URL with parameters may return various web page content, various page names may be assigned to one URL or URL with parameters as a result of automatic page name retrieval. Configure Settings at the Software Service Level 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Switch to the Page Name Recognition tab. 6. Choose the page name recognition priority. You can decide whether to give priority to static names or to names retrieved automatically using the response rules. Select Use static page names if defined if a static page name should always be used (if defined) or Use automatically retrieved page names from the HTML body of the response over static names if automatic page name recognition based on response rules should take precedence. 7. Add or edit a response recognition rule. Right-click the Response rules table and choose Add or Open. For software services based on SAP GUI over HTTP and SAP GUI over HTTPS analyzers, the table already contains a default definition enabling you to automatically extract the page name in SAP GUI over HTTP and SAP GUI over HTTPS environments. 8. Specify the strings to extract as the page name. The text between the first occurrences of strings defined by Begin tag and End tag found in the response body are reported as the page name. For example, suppose you want the web page title to be reported as the page name. The title is defined by the title element in the head section of the web page, like this: <head> <meta http-equiv="content-type" content="text/html; charset=iso "> <meta http-equiv="content-language" content="pl"> <title>page title</title> </head> 107

108 Chapter 6 Configuration Fine-Tuning To extract the Page title string as your page name, type <title> in the Begin tag field and </title> in the End tag field. Note that the definition is case sensitive. 9. Apply a regular expression to the extracted text. To trim the page name of redundant elements, you can add a regular expression in the Regex field to apply to the text extracted with Begin tag and End tag. 10. Decide whether you want to apply the definition immediately. Clear Disable this definition if you want to apply the definition immediately when you publish the configuration. Select Disable this definition if you want save the definition, but you do not want to use it now. 11. Add or edit a request recognition rule. The feature is available only in HTTP analyzer mode. For more information, see Choosing HTTP Analyzer Mode [p. 137]. Right-click the Request rules table and choose Add or Open. 12. Select a parameter matching method from the Parameter match list. The following matching methods are supported: Exact Report the specified parameter or the parameter and value. Usage syntax 'name=value' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. So, the wildcard character * is taken literally. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. 108

109 Chapter 6 Configuration Fine-Tuning Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples You can specify 'john', to match though note that in this case will not be reported because the parameter value '=123' was not explicitly specified. To match it, you would need to specify 'john=123'. Start Report parameters that begin with a specified string; report only the matched pattern, truncate any remainder of the parameter. Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples 'fred=5' will match but it will be reported as The value 'fred' will match as well as and it will be reported as 109

110 Chapter 6 Configuration Fine-Tuning Start (expand) Report parameters which begin with a specified string; report the entire parameter, not only the matched pattern. End Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples 'fred=5' will match and it will be reported as The value 'fred' will match as well as and it will be reported as and respectively. Report parameters which end with a specified string; report the entire parameter, not only the matched pattern. Usage syntax 'name=value' or any final part of it this string, including string of the form '=value' or just 'value'. 110

111 Chapter 6 Configuration Fine-Tuning Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples For to be matched, you can specify the following ends: '0', '00', '100', '=100', 'n=100' and so on, up to 'john=100'. Thus is reported. Value RegEx Report parameters which begin with a specified string; optionally attempt to match the remainder of the parameter with a regular expression; report the start string and selected portions of the regular expression, if any. Usage syntax Parameter is entered as name=value or any initial part of it this string including string of the form name= or just name. A regular expression (regex) is entered as an extended POSIX regular expression. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis 111

112 Chapter 6 Configuration Fine-Tuning of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed on the Parameter part; the regex part is matched as a case-sensitive POSIX regular expression. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples parameter specification fred= and a regular expression AB(C?E) will match but it will be reported as because the AB portion of the regular expression was not included in round braces. Custom RegEx Report parameters that match the given regular expression; report those portions that have been selected within the regular expression. Usage syntax Enter an extended POSIX regular expression to match the desired string. Mark portions to be reported by using round braces ( and ). Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters The request URL, POST body, or HTTP header are not split into parameters prior to pattern matching. Instead, they are treated as single units of data and the regular expression is applied to their entire contents. Only the path part of the request URL is excluded from the matching process. Limitations The regular expression is entered according to POSIX syntax. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches 112

113 Chapter 6 Configuration Fine-Tuning are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples Regular expression fred=ab(c?e) will match but it will be reported as Regular expression (.*=)AB(C?E) will match as well as and it will be reported as and as respectively. 13. Set the slow page thresholds for the particular page name string. You can set the page load time/operation time and server time thresholds for a particular page name string, either the name string set as a static page name, or the name string you expect to retrieve using automatic recognition rules. Right-click the Page name thresholds table and choose Add or Open. Type the page name, clear Inherit from rule setting, and set the value of your choice. If the page name string matches the name string reported, the thresholds you set take precedence over the thresholds set at other levels. 14. Click OK to save the configuration. 15. Publish the draft configuration on the monitoring device. Configure Settings at the URL Level Settings at the URL level take precedence over settings for a software service. 16. Repeat Step 1 [p. 107] through Step 4 [p. 107]. 17. Switch to the URL Monitoring tab. 18. In the URL definitions table, right-click a specific URL and select Open from the context menu to open Monitored URL window. To quickly navigate to an entry in the URL definitions table, click in the table and then type some or all of the IP definition. Click the magnifying glass icon or press [Ctrl+F] to open a search box to limit the table view to only those rows that contain a match (in any column) to the search string. 19. Switch to the Page Name tab. 20. Decide whether to add a static page name for the monitored URL. 113

114 Chapter 6 Configuration Fine-Tuning To define a static page name for all the URLs matching the definition criteria, you can type the preferred name in the Static name field. You can decide whether to give priority to static names or to the names retrieved automatically from response rules. You can use the priority defined at the software service level or you can clear Inherit setting from the rule and choose whether a static page name should always be used when available or whether automatic page name recognition based on response rules should take precedence. 21. Repeat Step 7 [p. 107] through Step 15 [p. 113]. Configure Settings at the URL Parameters Level Settings at the URL parameters level take precedence over settings for a URL and software service. 22. Repeat Step 1 [p. 107] through Step 4 [p. 107]. 23. Switch to the URL Monitoring tab. 24. In the URL parameter groups table, right-click a specific parameter group and select Open from the context menu. The URL Parameter Group window appears. 25. Switch to the Page Name tab. 26. Decide whether to add a static page name for the monitored URL with parameters. To define a static page name for all the URLs with parameters matching the definition criteria, you can type the preferred name in the Static name field. You can decide whether to give priority to static names or to the names retrieved automatically from response rules. You can use the priority defined at the URL level or you can clear Inherit setting from the URL and choose whether a static page name should always be used when available or whether automatic page name recognition based on response rules should take precedence. 27. Repeat Step 7 [p. 107] through Step 15 [p. 113]. Reporting of URLs with Redirects Several configuration settings have an effect on which URL and page name are reported if one or more redirects have been observed in traffic. Assume that you have created a software service definition and defined a URL that you want to monitor, and that a URL with a series of redirects was observed in the traffic. In this case, the AMD will only check whether the first redirect or the base URL (the last address after redirection) matches the configuration. It will skip the remaining redirects. Depending on the configuration, there are 3 basic scenarios: If either the first redirect or the base URL matches the configuration, the matching URL is reported; if you have defined a static page name for the URL in the configuration, the matching URL is reported under this name. If neither the first redirect nor the base URLs matches the configuration, the URLs detected in the traffic may only be reported as a result of auto-learning. For more information, see URL Auto-Learning [p. 88]. 114

115 Chapter 6 Configuration Fine-Tuning Which URL is added to auto-learning depends on the Report URL after redirect setting for the software service. By default, this setting matches the global setting. For more information, see General Configuration Options for HTTP-Related Analyzers [p. 133]. With reporting of a URL after redirect enabled, the learned URL is the base URL. With reporting of a URL after redirect disabled, the first redirect observed in the traffic is automatically learned by the AMD. Page names are reported according to the page name recognition settings defined for your software service. For more information, see Automatic Page Name Recognition [p. 106]. If both the first redirect and the base URL match the configuration, the Report URL after redirect setting for the software service determines which of the detected URLs is reported. By default, this setting matches the global setting. For more information, see General Configuration Options for HTTP-Related Analyzers [p. 133]. With reporting of a URL after redirect enabled, the reported URL will be the base URL. The page name reported will also be the one for the base URL (provided you defined a static name in the software service rule). With reporting of a URL after redirect disabled, the first redirect observed in the traffic will be reported. Also, the page name (if defined) will be the one of the reported redirect. If you do not define any static page names for the URL, page name reporting will always be based on page name recognition settings for the base URL. Content Type URL Monitoring When defining URL monitoring, some definition criteria may cover content that is not of your interest (such as the binary content). To exclude it from URL-based software service monitoring, you can narrow monitoring to selected content types such as text/html or text/xml. It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Switch to the URL Monitoring tab. 6. In the URL definitions table, right-click a specific URL and select Open from the context menu to open Monitored URL window. 115

116 Chapter 6 Configuration Fine-Tuning To quickly navigate to an entry in the URL definitions table, click in the table and then type some or all of the IP definition. Click the magnifying glass icon or press [Ctrl+F] to open a search box to limit the table view to only those rows that contain a match (in any column) to the search string. 7. Switch to the Content Types tab. 8. Choose the scope of content type monitoring. When defining content type URL monitoring, you can choose one of three options: Use active content types The default setting. If content type monitoring at the software service level is set to use the global values, the global settings for all software service are applied. Otherwise, the settings at the software service level will be used. For more information, see Monitoring of Non-HTML Objects Based on Content Type [p. 116]. This option is unavailable for AMDs prior to the 12.3 release. If you are defining software services for multiple AMDs and at least one is older than the 12.3 release, the Use global values option is unavailable. Report all content types If a URL matches the definition, it is always reported regardless of its content type. Report selected content types If you want to narrow URL monitoring to selected types, choose Report selected content types and select one or more of the preconfigured content types. You can also add other content types to appear in the table. When adding new content types, make sure you type them exactly as they appear in the Content-Type field of the HTTP header. If you select none of the content types from the list, the result is the same as if you set the Report all content types option. This setting overrides global content type configuration. 9. Click OK to save the new configuration and to go back to the main screen of the All Monitoring perspective. 10. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Monitoring of Non-HTML Objects Based on Content Type By default, an AMD recognizes only HTML objects as pages, but it can be configured to treat other types of objects as HTML pages to be monitored. Such objects may include, for example, images, embedded objects such as Flash objects, and objects that require third-party plug-ins to render. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 116

117 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Navigate to Global Front-End Monitoring Web HTTP Content Type Monitoring. 5. Add a content type to the table listing objects recognized as monitored pages. To have the AMD treat a certain content type as a monitored page, right-click the Objects recognized as pages table and select Add from the context menu (or click the add a new entry. For each entry, you can set the following options: icon) to Auto-Learning Enabled Enable URL auto-learning mechanism for pages of the selected content-type. For more information, see Details of the URL Auto-Learning Algorithm [p. 90]. Treat as HTML For asynchronous web applications, partial page updates are not declared as text/html, so the AMD does not handle such events using HTML-based monitoring features. Select Treat as HTML for update information content types (typically text/xml) to be able to recognize partial page updates as pages, report page elements, enable frame recognition, recognize the page name, apply response based rules, and report metrics and attributes. Otherwise, you will only be able to calculate basic performance metrics for partial page updates. The text/html content type is the default for pages and it cannot be removed from the list. Accordingly, the text/html pages are always treated as HTML and Treat as HTML is always set to true and cannot be modified. These entries must be compatible with those of the Content type field in the HTTP header. Many instances of the parameters are allowed, one for each content type to be recognized as a page. For example, image/jpg and image/gif are valid entries for an Objects recognized as pages table. 6. Configure page filtering based on the content of the URL. Filtering is governed by a configuration property defined in the Filtering out pages list. URLs to which the filtering criteria in the list apply are not reported in the performance data files. This can be useful if, for example, a client requests a page composed of HTML content and a number of images, but some of the requested images are missing. The web server would respond with an HTTP error code, but if it responds with an HTML page stating that an element is missing, this would be recorded as a legitimate page load and would misleadingly raise page volume reports and should be filtered out. In such cases, you could use the Filtering out pages list to prevent such pages from being recorded. The default list contains the following entries:.css.htc.gif.jpg Chapter 6 Configuration Fine-Tuning 117

118 Chapter 6 Configuration Fine-Tuning.jpeg 7. Publish the draft configuration on the monitoring device. What to Do Next In addition to monitoring based on content type derived from the HTTP header, you can specify objects to be included in auto-learning as described in URL Auto-Learning [p. 88]. Note that this feature refers to the content of a field in the HTTP header, and not to a string contained in a URL being loaded. Logging Transactions, ADS Data and ADS Header Data If you are interested in obtaining information based only on aggregated monitoring data on your reports, and not in per-url data, you can globally disable data generation for transactions and ADS data and ADS header data in the RUM Console. Before You Begin It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. Configuration options for logging of transactions and ADS data and ADS header data are defined globally for all services, though logging can then be disabled for individual user-defined services. To configure logging of transactions, ADS data and ADS header data on a specific AMD: Configure global settings 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Global Front-End Monitoring Web HTTP Sequenced Transactions and Header Data General. 6. Select to generate or not to generate monitoring data. Select or clear the option entitled Generate transactions and ADS data and ADS header data. NOTE This global option affects data generation for all HTTP-based services and takes precedence over them. Clearing this option here will cause no such data generated for any HTTP services, even if data generation is enabled for an individual user-defined service. 118

119 Chapter 6 Configuration Fine-Tuning If you do not require this type of data to be generated and have cleared the option, proceed to Step 14 [p. 120]. Otherwise, proceed to the next step. 7. Optional: Configure data generation for explicitly defined URLs only. If you require data to be generated only for URLs that have been explicitly defined in user-defined services or recorded through auto-learning, select the Generate data only for explicitly defined URLs check box. Monitoring this data for default services will then be turned off. 8. Optional: Select content type information to be saved. Select or clear the option entitled Save all content type in addition to HTTP hits to log or not to log hits other than HTTP. 9. Optional: Configure masking of sensitive information. Provide masks to avoid explicit logging of sensitive data. The masks should be specified by clicking the context menu in the Parameter masks section of the configuration screen. For more information, see Masking of Sensitive HTTP Information [p. 121]. 10. Optional: Specify the maximum size of POST data saved. Specify the number of bytes in the edit box of this name. 11. Optional: Specify the maximum size of cookie data saved. Specify the number of bytes in the edit box of this name. 12. Optional: Specify custom tags to be extracted. The custom tags functionality allows you to select a whole field in the HTTP header to write to the vdata files. Right-click the table entitled Custom tags and select Add from the pop-up menu. In the Custom Tags pop-up window, specify the following information: Name The name of the field to report in the vdata file. Syntax: letters, numbers, hyphens, and brackets are acceptable; other characters, including spaces, are forbidden. Pattern The field name to extract from the HTTP header. Syntax: the whole field name with the colon. For example: Cookie:, Host: or Content-type:. Extract from Defines the source of data: HTTP request, HTTP response, or HTTP request and response. Choosing None will result in no tags being extracted. Click OK to confirm the configuration. 13. Optional: Specify data filters. Data filters enable you to filter information by a client IP address or by a user name. The filtered data is reported by the ADS. To define data filters: a. Open the Global Front-End Monitoring Web HTTP Transactions and Header Data Data Filters screen. 119

120 Chapter 6 Configuration Fine-Tuning b. Enable filtering using the provided check box. c. Specify the conditions required for the data to be logged. Decide whether both IP address and user name have to be satisfied for information to be logged, or whether information should be logged even if only one filtering condition is defined. d. Define new data filters. To add filtering conditions, right-click the appropriate filter table and select Add from the context menu. To define a client IP address filter in the Client IP addresses table, choose one of the following options: A simple IP address A range of IP addresses An IP address with a mask To define a user name filter in the User names table, provide a simple user name or specify a regular expression to match a set of user names. You can test the patterns that will be used by the AMD using the Regular Expressions Test tool, which is activated after you click Test located next to the regular expression pattern field. For more information, see Testing Regular Expressions [p. 213]. 14. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Optional: Disable logging transactions, ADS data, and ADS header data for individual services If, in Step 6 [p. 118], you disabled data generation, you cannot enable it for an individual software service. If data generation is enabled in the global level, however, you can still disable it for a selected service. 15. From the top menu, select Software Services Manage Software Services. 16. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 17. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 18. Click the HTTP Options tab. 19. In the Data Generation section, clear the check boxes labeled Transactions and ADS Data or ADS Header Data as needed. By default, generation of transactions and ADS data is enabled. 20. Click OK to save the configuration. 120

121 Chapter 6 Configuration Fine-Tuning 21. Publish the draft configuration on the monitoring device. Masking of Sensitive HTTP Information Sensitive information can be masked out before HTTP monitoring data is stored in log files. The vdata and headerdata files on the AMD contain logs of transactions and ADS data and logs for ADS header data. These logs may contain sensitive information such as passwords embedded in parameter values, URLs, or cookies. To prevent such information from being logged, you can configure your AMD to mask specific portions of certain parameters. Masking is then performed using the asterisk character *. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Global Front-End Monitoring Web HTTP Transactions and Header Data General. 6. To add a new mask, right-click the Parameter Masks table and select Add. For example, the URL: would appear in the log files as: For vdata files, the parameters are masked out in the following HTTP fields: cs-uri-query cs(cookie) cs(referer) For headerdata files, the parameters are masked out in the following HTTP fields: chdr shdr reqparams postdata The parameter fragments to be masked out must be specified in the mask field. You indicate a particular parameter fragment to be masked out by specifying a string of characters either immediately following or immediately preceding the fragment. The search string is then left unchanged, while the adjacent fragment is masked out. Parameter boundaries are identified based on parameter separator characters. The following examples demonstrate masking out a parameter fragment both following and preceding a search string: 121

122 Chapter 6 Configuration Fine-Tuning The mask defined as region=* transforms the URL page=homepage&params.richmedia=yes&source=national&ord= &zip=33401&state=fl& dma=palm_beach&region=palmbeach3&yearrange=2&certified=n&tile=2&flip=a into page=homepage&params.richmedia=yes&source=national&ord= &zip=33401&state=fl& dma=palm_beach&region=**********&yearrange=2&certified=n&tile=2&flip=a The mask defined as *=FORD transforms the URL /html.ng/adsize=1x1&site=ntl&page=findacar::ispsearchform::srl& Params.richmedia=yes&source=ntl&ord= &zip=92821&state=CA&dma=LOS_ANGELES& region=losangeles7a&make=ford&model=escort&type=used&start_year=1981&end_year=2006& from_price=1.0&to_price= &yearrange=4&certified=n&tile=3&flip= into /html.ng/adsize=1x1&site=ntl&page=findacar::ispsearchform::srl& Params.richmedia=yes&source=ntl&ord= &zip=92821&state=CA&dma=LOS_ANGELES& region= LOSANGELES7A&****=FORD&model=ESCORT&type=used&start_year=1981&end_year=2006& from_price=1.0&to_price= &yearrange=4&certified=n&tile=3&flip= Multiple mask patterns can be defined by defining many mask entries. 7. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Character Encoding Support for HTTP Services Enabling the internationalization option for HTTP services makes it possible to recognize the character encoding of HTTP content. Before You Begin It is assumed that you have already created one or more user-defined software services for this protocol. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Go to Configuration Global Front-End Monitoring Web HTTP Character Encoding Support. 6. Enable character encoding support by selecting the Support Internationalization check box. By default, this option is disabled. 7. Select the required encoding from the Force a default encoding list. 122

123 Chapter 6 Configuration Fine-Tuning This makes it possible to apply a specific encoding regardless of the automatically detected one. 8. Select the Auto-Detection Algorithm to automatically detect monitored content encoding. This makes it possible to narrow down the choices of encoding where the algorithm is not able to identify a specific language. NOTE For Chinese encodings, there is no auto-detection; all encodings must always be specified manually. 9. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Rule-based Character Encoding for HTTP Services Global support for character encoding for monitoring HTTP-based traffic can be customized per software service. Before You Begin It is assumed that you have already created one or more user-defined software services for this protocol. You can define different character encoding for each of the HTTP components: URI encoding Parameter encoding (request) Header encoding (request) Response header encoding Response body encoding To customize character encoding: 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. Select the Character Encoding tab. 5. Right-click the Encodings table and then select Add to create a new setting, or select Open to modify an existing record. 6. Type the settings for a specified host. 123

124 Chapter 6 Configuration Fine-Tuning Note that if you leave the Host box empty, the settings will be applied to all entities that comply with the rule. Not all encoding settings have to be configured. The per-rule settings take precedence over global or implied settings. 7. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Assigning HTTP Error Codes to Error Categories You can configure the contents of HTTP error categories on AMDs that will feed your report server. The settings are global, which means that they apply to all analyzers reporting information on HTTP errors: HTTP, Oracle Forms, XML, and SOAP. The AMD is able to deliver information on seven HTTP error groups ( categories ), five of which have configurable contents: HTTP Authentication errors (default: 401 and 407) HTTP Not Found errors (default: 404) HTTP Custom Client errors 1 (4xx) HTTP Custom Server errors 1 (5xx) HTTP Custom Server errors 2 (5xx). The two remaining groups contain HTTP errors that do not fall into any of the above categories: HTTP Other Client errors (4xx) and HTTP Other Server errors (5xx). As the result of the assignment, the CAS reports the HTTP errors using the following metrics: HTTP errors The number of observed HTTP client errors (4xx) and server errors (5xx). HTTP client errors (4xx) The sum of all HTTP client errors (4xx). This includes 4 categories of errors (4xx), by default HTTP Unauthorized (401, 407) errors, HTTP Not Found (404) errors, custom client (4xx) errors and Other HTTP (4xx) errors. The contents of the first 3 categories can be configured by users. However, there are two types of the 4XX errors that are of particular importance: errors 401 related to server-level authentication, and errors 404 indicating requests for non-existent content. These two error types are reported separately, by specific metrics. 401 Unauthorized - Server reports this error when user's credentials supplied with request do not satisfy page access restrictions. The HTTP server layer, not the application layer, reports 401 errors. The AMD will report on "Unauthorized" errors only if server-level authentication has been configured. This is common practice for sites that are comfortable with very basic user access policies. Most commercial-grade applications do not rely on server-level authentication (e.g. most of online banking applications or online shopping), but rather authenticate users on the application layer. In such a case, even if authentication fails, the server will typically send 200 OK 124

125 Chapter 6 Configuration Fine-Tuning responses and authentication error information will be explained in page content. So this kind of error is not very common in commercial sites. 404 Not Found - Server reports "Not Found" errors when it cannot fulfill client request for a resource. Usually it happens due to malformed URL, which directs to a non-existing page or image. Such a URL request may result from a user, who misspelled the URL, trying to access a URL that the user stored in his "Favorites" folder a long time ago, or some other mistake. Malformed URLs may also exist in invalid or incorrectly designed Web pages so the error will be reported by browsers trying to load such a page. Significant and constant number of these errors usually indicates that some pages on the server have design-related or link validation issues. In some cases, 404 errors result from the server overload. It is good practice to check whether the percentage of errors is load-related. HTTP unauthorized errors 401, 407 (default name) The number of observed custom HTTP authentication related errors. These include "HTTP 401 Unauthorized" and "HTTP 407 Proxy authentication required" errors. HTTP servers generate errors "401 Unauthorized" in cases, when anonymous clients are not authorized to view the requested content and must provide authentication information in the WWW-Authenticate request header. The 401 errors are similar to "403 Forbidden" errors, however used when authentication is possible but it has failed or not yet been provided. The 407 error is basically similar to 401, but it indicates that the client should first authenticate with a proxy server. The AMD will report these errors only if the server-level authentication has been configured. Simple and basic user access policies are common in Web sites that do not store user-sensitive and/or business critical information. Most commercial-grade applications, based on HTTP, such as home banking applications or online shopping sites, rely on the application-level authentication rather than the server-level authentication. Such applications are designed in the way that even if the user authentication fails, the HTTP server usually sends the 200 OK response code and the authentication error message in the page content. Therefore, the 401 Unauthorized and 407 Proxy authentication required error codes are quite rare in commercial environments. HTTP not found errors 404 (default name) The number. These include the observed custom HTTP 404 Not found errors. HTTP client errors - category 3 (default name) The number of HTTP custom client errors (4xx). By default, there is no specific error type assigned here. HTTP other client errors (4xx) The number of HTTP other client errors (4xx). There are four categories of HTTP client errors (4xx), of which three can be configured by users. By default, the first category includes HTTP Unauthorized (401, 407) errors, the second category - HTTP Not Found (404) errors. The third category contains no default error types assigned, and can be configured by a user. Finally, a group of HTTP Other (4xx) errors contains all errors that do not fall into any other client errors category. 125

126 Chapter 6 Configuration Fine-Tuning The number is calculated based on the formula: [HTTP errors 4xx] - [HTTP Not Found errors 404] - [HTTP Not Authorized ( )] - [HTTP errors configured by user]. HTTP server errors (5xx) The number of observed HTTP server errors (5xx). The response status codes 5xx indicate cases, in which the Web server is aware that there was a server error or it is incapable of performing the request. Such error presence usually means that the Web server does not function as intended. The following 5xx errors are defined by the HTTP protocol standards: 500 Internal Server Error - The server encountered an unexpected condition, which prevented it from fulfilling the request. 501 Not Implemented - The server does not support the functionality required to fulfill the request. 502 Bad Gateway - The server received an invalid response from a back-end application server. 503 Service Unavailable - The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. 504 Gateway Timeout - The server did not receive response from a back-end application server. 505 HTTP Version Not Supported - The server does not support the HTTP protocol version that was used in the request message. HTTP server errors category 1 (default name) The number of custom HTTP server errors (5xx), category 1. By default, there are no specific error types assigned to this category. HTTP server errors category 2 (default name) The number of custom HTTP server errors (5xx), category 2. By default, there are no specific error types assigned to this category. HTTP other server errors (5xx) The number of HTTP server errors (5xx) that do not fall into categories 1 or 2 of custom HTTP server errors (5xx). Categories with no status codes assigned cannot have their names edited by a user of a report server. NOTE All AMDs connected to one CAS must have identical configurations for HTTP errors. If you have already configured HTTP errors on one of your AMDs, you can import or copy the configuration settings from this device to the other devices. The configuration is stored in the applications.xml file. For information on how to import the settings, see Importing the AMD Configuration in the Data Center Real User Monitoring Administration Guide. To learn how to copy the settings, see Propagating the AMD Configuration Using RUM Console in the Data Center Real User Monitoring Administration Guide. To configure HTTP errors: 126

127 Chapter 6 Configuration Fine-Tuning 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Global Front End Monitoring Web Errors. 6. Modify, add, or delete the HTTP status codes. To add a new error code, right-click the table corresponding to the error group to modify and select Add. This will automatically add a 400 or 500 code to the group of client or server errors, depending on which group you are editing. To modify the default status code number, click it to make the text editable, and then type the replacement number. To modify an error code, click the code number to make the text editable and type a new code. To delete an existing error code, right-click a code number and select Delete from the context menu. 7. Publish the draft configuration on the monitoring device. Managing SSL Alert Codes You can define new alert codes using the RUM Console, change predefined common SSL alert codes and decide which alert codes should be taken into account when calculating the failures (transport) metric.. By default, the most commonly used alert codes are already defined and divided into three groups: SSL Alerts A 10, 20, 21, 22, 30, 40, 49, 50, 51, 60, 70, 71, 110 This group is shown on Data Center Real User Monitoring reports as SSL Error 1, named SSL session fatal error by default.. SSL Alerts B 41, 42, 43, 44, 45, 46, 48, 111, This group is shown on Data Center Real User Monitoring reports as SSL Error 2., named SSL handshake fatal error by default. SSL Alerts N All alerts not mentioned above. This group is shown on Data Center Real User Monitoring reports as Other SSL Errors, named SSL warnings by default. The following table lists all SSL alerts that AMD can recognize: 127

128 Chapter 6 Configuration Fine-Tuning Table 4. SSL alert codes SSL alert name close_notify unexpected_message bad_record_mac decryption_failed record_overflow decompression_failure handshake_failure no_certificate_reserved bad_certificate unsupported_certificate certificate_revoked certificate_expired SSL alert code Description Notifies the recipient that the sender will not send any more messages on this connection. Received an inappropriate message This alert should never be observed in communication between proper implementations. This message is always fatal. Received a record with an incorrect MAC. This message is always fatal. Decryption of a TLSCiphertext record is decrypted in an invalid way: either it was not an even multiple of the block length or its padding values, when checked, were not correct. This message is always fatal. Received a TLSCiphertext record which had a length more than 2^ bytes, or a record decrypted to a TLSCompressed record with more than 2^ bytes. This message is always fatal. Received improper input, such as data that would expand to excessive length, from the decompression function. This message is always fatal. Indicates that the sender was unable to negotiate an acceptable set of security parameters given the options available. This is a fatal error. Send by a client to indicate that he does not have a proper certificate to fulfill a certificate request from the server. This alert description is no more used by TLS (now a client sets an empty certificate message if he does not have a proper certificate). There is a problem with the certificate, for example, a certificate is corrupt, or a certificate contains signatures that cannot be verified. Received an unsupported certificate type. Received a certificate that was revoked by its signer. Received a certificate has expired or is not currently valid. 128

129 Chapter 6 Configuration Fine-Tuning Table 4. SSL alert codes (continued) SSL alert name certificate_unknown illegal_parameter unknown_ca access_denied decode_error decrypt_error export_restriction protocol_version insufficient_security SSL alert code Description An unspecified issue took place while processing the certificate that made it unacceptable. Violated security parameters, such as a field in the handshake was out of range or inconsistent with other fields. This is always fatal. Received a valid certificate chain or partial chain, but the certificate was not accepted because the CA certificate could not be located or could not be matched with a known, trusted CA. This message is always fatal. Received a valid certificate, but when access control was applied, the sender did not proceed with negotiation. This message is always fatal. A message could not be decoded because some field was out of the specified range or the length of the message was incorrect. This message is always fatal. Failed handshake cryptographic operation, including being unable to correctly verify a signature, decrypt a key exchange, or validate a finished message. Detected a negotiation that was not in compliance with export restrictions; for example, attempting to transfer a 1024 bit ephemeral RSA key for the RSA_EXPORThandshake method. This message is always fatal. The protocol version the client attempted to negotiate is recognized, but not supported. For example, old protocol versions might be avoided for security reasons. This message is always fatal. Failed negotiation specifically because the server requires ciphers more secure than those supported by the client. Returned instead of handshake_failure. This message is always fatal. 129

130 Chapter 6 Configuration Fine-Tuning Table 4. SSL alert codes (continued) SSL alert name internal_error user_canceled no_renegotiation unsupported_extension certificate_unobtainable unrecognized_name bad_certificate_status_response SSL alert code Description An internal error unrelated to the peer or the correctness of the protocol makes it impossible to continue, such as a memory allocation failure. The error is not related to protocol. This message is always fatal. Cancelled handshake for a reason that is unrelated to a protocol failure. If the user cancels an operation after the handshake is complete, just closing the connection by sending a close_notify is more appropriate. This alert should be followed by a close_notify. This message is generally a warning. Sent by the client in response to a hello request or sent by the server in response to a client hello after initial handshaking. Either of these would normally lead to renegotiation; when that is not appropriate, the recipient should respond with this alert; at that point, the original requester can decide whether to proceed with the connection. One case where this would be appropriate would be where a server has spawned a process to satisfy a request; the process might receive security parameters (key length, authentication, and so on) at start-up and it might be difficult to communicate changes to these parameters after that point. This message is always a warning. Sent by the client if the ServerHello does contain an extension that the client did not requested in his ClientHello, fatal Sent by the server to indicate that he cannot obtain a certificate from the URL the client has sent within a ClientCertificateURL extension, maybe fatal Sent by the server if he does not recognize a server name included in the ServerNameList extension received from the client, maybe fatal Sent by the client if he gets an invalid certificate status response after having sent a CertificateStatusRequest extension, fatal. 130

131 Chapter 6 Configuration Fine-Tuning Table 4. SSL alert codes (continued) SSL alert name bad_certificate_hash_value unknown_psk_identity other SSL alert code ? Description Sent by the server if a certificate hash value does not match to the corresponding value received within a ClientCertificateURL extension message, Fatal Indicates that the server does not recognize the PSK identify sent by the client. Fatal other By default, the most commonly used alert codes are already defined, including the alert source: server, client or both. Use the SSL Alerts table to to indicate the codes that should be reported as failures (transport). For more information, see Calculating Availability [p. 146].. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate the Configuration tree to Global Advanced SSL Options. 6. Select the Report server name from SSL certificate check box to enable the AMD to extract the names from SSL certificates. These names are included with the monitored data along with the SSL setup time, protocol, and cipher. 7. Right-click and select Add or Delete to add or delete the SSL alert codes in the SSL Failures table. You can also choose the source of alert code to trigger an SSL failure: server, client or both. 8. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. What to Do Next If the AMD is connected to CAS, SSL errors can be given customized names on the report server side. For more information, see Defining SSL Error Names [p. 132]. 131

132 Chapter 6 Configuration Fine-Tuning Defining SSL Error Names SSL connection setup errors are aggregated into groups by the AMD according to the AMD configuration. The aggregated errors appear on reports as SSL error 1, SSL error 2, and Other SSL errors. Before You Begin Administrative privileges are required to access the Advanced Properties Editor. Under normal circumstances, use the Customized names configuration tool to configure the SSL error names, but if that is not possible, use the Advanced Properties Editor on the report server instead. To customize these default names, change the report server configuration in the Advanced Properties Editor in Diagnostic Console: 1. Open and log on to the report server. 2. Open the Diagnostic Console. In your web browser address field, enter: 3. In the Diagnostic Console, select Advanced Properties Editor. 4. Click the right arrow to page to the SSL error names section. 5. Type the new error names. Other SSL Errors name (SSL_ERR.3) SSL Error level 1 name (SSL_ERR.1) SSL Error level 2 name (SSL_ERR.2) 6. Click Save to save your changes. Excluding IP Ranges from AMD Client Analysis You can exclude particular client IP address ranges from AMD analysis. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Select Global Advanced Excluded Client ranges. 6. Provide the start and end IP addresses for each range to exclude from AMD analysis. Be sure not to filter everything out or there will be no data in your reports. 7. On the Devices screen, click Publish Configuration. 132

133 Importing and Managing User ID Mapping Information Additional mappings of user IDs to session IDs can be preloaded from an external configuration file. The file should be named usermap.config and should reside in usr/adlex/config. Importing mapping information from a file facilitates recognition of those users who may have already logged into or registered at a particular web service. The external configuration file is read by the AMD on startup, the information is imported and stored internally in the AMD configuration, and the file is renamed to prevent it from being imported again. The file contains text lines, with each line consisting of a cookie value, a semicolon, and then a user ID. Whenever one of the listed cookie values appears in the monitored traffic, the corresponding user ID is used for user identification. Example 8. An example of the usermap.config file content # mapping session IDs to user IDs TjrUUwN6b5DjAV8pMk515phzVISv3Wts6pKyxKUb5fwgDn2IVAKA0Dmkc4J6vla; user00000@thecompany.com JfkxFm5HBfmbvMM1ttHlQBVfddnpP28ituBlPb5uLc9oVfEDf3qbEJ7Ycs0U70E; user00001@thecompany.com Note that the hash character can be used to denote comment lines. Related console commands The following AMD console commands are provided for managing user ID mappings: SHOW HTTP USERMAP STATUS SHOW HTTP USERMAP SAVE HTTP USERMAP CLEAR HTTP USERMAP Chapter 6 Configuration Fine-Tuning General Configuration Options for HTTP-Related Analyzers HTTP general configuration options are options related to a variety of settings such as timeout values, redirection handling, session recognition, multi-frame page handling, and cookie handling. They can be set globally for the AMD or individually for particular software services. Before You Begin It is assumed for this task that you have already created one or more user-defined software services for this protocol and know how to access and modify global settings for an AMD and settings for a specific service. To configure general options for monitoring this protocol: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 133

134 Chapter 6 Configuration Fine-Tuning 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Front-End-Monitoring Web HTTP General. NOTE Configuration options related to general settings for an AMD for this protocol analyzer are also under the HTTP Options tab for individual user-defined services. 6. Configure options available in the General section. The list of configuration options includes: Redirect timeout This timeout period, expressed in seconds, is configured globally for all software services. HTTP redirects are stored until either a matching target URL is seen or a timeout expires. If the redirect target page has not been seen by the time the redirect timeout expires, the AMD reports the URL with all transactional metrics equal to zero and the redirect is referred to as an orphaned redirect. The URL reported is taken from the orphaned redirect. Cascaded unauthorized hits timeout Cascaded unauthorized hits older or equal to this timeout (in seconds) are treated as unauthorized. In case of a mixed cascade, redirects and unauthorized hits, the head of cascade determines which timeout should be used, Redirect timeout or Cascaded unauthorized hits timeout. Last packet HTTP session timeout If the time since the last packet for an HTTP session is longer than this value (in seconds), the hit is considered finished and closed. This timeout period is configured globally for all software services. Hit session timeout Maximum time delay allowing a hit to be linked to a page. The value is specified in seconds, with a resolution of one-tenth of a second, and is configured globally for all software services. User agent timeout Time in minutes after which a user agent will be erased from the cache. The value is configured globally for all software services. Maximum header length Maximum size in bytes of the buffer that the HTTP header can be assembled into before considering it incomplete and proceeding with its processing. This option is available only in HTTP mode of the HTTP analyzer. The value is configured globally for all software services. Maximum request body length Maximum size in bytes of the buffer that HTTP request body can be assembled into before considering it incomplete and proceeding with its processing. This option is 134

135 Chapter 6 Configuration Fine-Tuning available only in HTTP mode of the HTTP analyzer. The value is configured globally for all software services. Report URL after redirect This option causes addresses after the last redirection to be reported for redirected pages. By default, redirections are reported as addresses of the originating page, before redirection takes place. The final target page will be reported regardless of how many redirects are detected in between. The option can be set globally for all software services or configured for a specific user-defined software service. Specific settings take precedence over global settings. It is not supported by HTTP Express analyzer. Report URL prefixed with analyzed HTTP method If this option is selected, the string POST or GET is prefixed to the reported URL. This option can be set globally for all software services or configured for a specific user-defined software service. Specific settings take precedence over global settings. The All methods option allows for processing all detected HTTP methods including the WebDAV HTTP extesion. The extended WebDAV methods automatically identified include: PROPFIND Retrieves properties and a directory hierarchy of a remote system. PROPPATCH Changes and deletes multiple properties is a single operation. MKCOL Creates directories or collections. COPY Copies a resource from one URI to another. MOVE Moves a resource from one URI to another. LOCK Puts a lock on a resource. UNLOCK NOTE Removes a lock from a resource. Monitoring WebDAV software services requires a specific configuration options. In order to properly report a hit as a separate operation, you must define a URL with regex matching all URLs ( and content types. Treat a client RST packet sent by the session as closing session If this option is selected, the protocol analyzer will treat a client RST packet sent by the session as closing the session instead of aborting it if there was no content length header. It is configured globally for all software services. 135

136 Chapter 6 Configuration Fine-Tuning 7. Configure page and session recognition based on cookies You can use cookies to distinguish between separate HTTP pages and sessions. This is useful when, for example, a number of distinct users are hidden behind a shared load balancer or a proxy server.for more information, see Global Settings for Page and Session Recognition Based on Cookies [p. 139]. 8. Select client IP address extraction method. To turn off automatic client IP address extraction, select Off. To extract the client IP address from a header tag, select Header tag and type the name of the HTTP header field containing the real client IP information. The string extracted by the regular expression becomes the real client IP address reported to the report server. Example 9. The following is an example of extracting the value of REMOTE_ADDR field from the HTTP header. An HTTP header might contain the following information: GET HTTP/1.1 Accept: */* Referer: Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: Connection: Keep-Alive Cookie: FPB=061j8hura11q56cv; CRZY9=t=1; REMOTE_ADDR: The following regular expression extracts the address from the REMOTE_ADDR field: %0d%0aREMOTE_ADDR:%20\([^%0d%0a]*\)%0d%0a The expression must contain a single sub-expression delimited by pairs of characters \( and \). The expression in this example states that the search string should start at the beginning of a header line and end at the end of the line (note the use of % to denote the hex values of the carriage return and line feed characters). The line should start with the string REMOTE_ADDR:. The sub-expression to extract is a string of characters different than ASCII CR or LF, and it should occur after the space following REMOTE_ADDR: For details on how expressions are used, see Using Regular Expressions to Extract User Identification [p. 75]. To use the real client IP address as both the user ID and the user IP address, select Try to convert user name to IP address. 9. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. 136

137 Choosing HTTP Analyzer Mode You can choose between two HTTP analyzer modes: HTTP and HTTP legacy. The HTTP legacy mode is similar to the analyzer available in previous releases. If you want to take advantage of the new, enhanced features of the HTTP analyzer, you should use the default enabled HTTP mode. The following features are supported only by the HTTP analyzer: Miscellaneous parameters Miscellaneous parameters are text strings available in the URL request or response body. You need to define recognizable text patterns conveying the miscellaneous parameters that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. Miscellaneous parameters, unlike parameters extracted for URLs with parameters, are not defined together with an accompanying URL. When extracting Miscellaneous parameters, only the initial hit triggering the web page load is taken into account. The extracted Miscellaneous parameters must not be longer than 1030 bytes. For more information, see Extracting Miscellaneous Parameters in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Grouping attributes Grouping attributes are text strings available in the URL request or response body that uniquely identify clients. You need to define recognizable text patterns conveying the grouping attributes that then can be used in DC RUM reports as dimensions and enable additional ways of grouping data under specific categories of your interest. For more information, see Extracting Grouping Attributes in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Excluding Elements from Orphaned Redirects Reporting If you do not want particular redirects to be reported as orphaned, exclude them from orphaned redirects reporting by defining strings describing these elements. You can define the elements to be extracted from the Location field of the HTTP header or the request URL. For more information, see Excluding Elements from Orphaned Redirects Reporting [p. 99]. Extracting page names from the HTTP requests For more information, see Automatic Page Name Recognition [p. 106]. Advanced user recognition For more information, see Overview of User Name Recognition Configuration in HTTP Mode [p. 65]. When choosing the HTTP mode, ensure your system satisfies the enhanced requirements. For more information, see General Hardware Requirements in the Data Center Real User Monitoring Hardware Recommendations. Global Settings for Recognition and Parsing of URLs Chapter 6 Configuration Fine-Tuning The global configuration settings for recognition and parsing of URLs are inherited by all user-defined software services for HTTP. Global settings can be overridden by specific settings for a particular user-defined software service. 137

138 Chapter 6 Configuration Fine-Tuning NOTE The default values for these settings should be sufficient for most purposes and care should be taken when modifying them. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Open the global settings section for recognition and parsing of URLs. Navigate to Global Front End Monitoring Web Recognition and Parsing of URLs. 6. Select the method of truncating URLs. In the field Method of Truncating URLs, select the method of truncating URLs when monitoring HTML page loads: No cut URLs are not truncated. Cut after last slash URLs are truncated after the last slash ( / ) character. Cut after first separator URLs are truncated after the first separator (see below for separator definitions). If cutting according to separators is selected and if the set of defined separators is empty, the URL is not cut, which is equivalent to specifying No cut. 7. Specify characters to be recognized as separators. a. In the First parameter separators field, type characters to be recognized as separators between URLs and their parameters. b. In the Parameter Separators in URL field, type characters to be recognized as separators between consecutive parameters in URL and POST body. c. In the Parameter Separators in HTTP Header field, enter characters to be recognized as separators between consecutive parameters in the HTTP header. NOTE If monitored pages may include the question mark (? ) character as the value of a parameter, it is necessary to remove it from the list of previously defined separators. 8. Define the order of searching for parameters. When defining a specific user-defined service to be monitored, you can indicate whether you want parameters extracted from the URL or from the URL request, or from any combination of these.for more information, see Configuring Monitoring of URL Parameters in the RUM Console Online Help. 138

139 Chapter 6 Configuration Fine-Tuning However, the order in which these components of the HTTP packet are searched is determined here for all HTTP services, and the first parameter that matches the search criteria is accepted. In the Search for Parameters First section, select In POST Body to cause the POST body to be searched before the URL. Selecting In URL Request will cause the URL to be searched before the POST body. The HTTP header is always searched last. 9. Publish the draft configuration on the monitoring device. Global Settings for Page and Session Recognition Based on Cookies You can use cookies to distinguish between separate HTTP pages and sessions. This is useful when, for example, a number of distinct users are hidden behind a shared load balancer or a proxy server. Before You Begin It is assumed that you have already created one or more user-defined software services for this protocol. To configure page or session recognition using cookies: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Select Configuration Global Front-End-Monitoring Web HTTP General. 6. Select or clear the check box to specify whether to use cookies to distinguish separate sessions and pages. 7. Optional: Enter the cookie name. If you provide a cookie name, only cookies with the specified name will be used for session and page recognition. If no cookie name is specified, all cookies are used. Thus, if a name is entered, hits with this particular cookie name will be linked, provided the cookie value is the same for all the hits. If no name is entered, all cookies in a hit are looked at and their value is extracted. Matching hits must have the same cookie names and cookie values. 8. Specify the cookie time-to-live resolution. If a given cookie or cookie set with particular cookie values does not appear in the analyzed traffic for this length of time, the cookie or cookie set is discarded and the corresponding session is considered closed. Future occurrences of this set will be treated as belonging to a new session. The value is specified in seconds. 9. Save or publish the configuration. Click Save to save your changes and continue with configuration. 139

140 Chapter 6 Configuration Fine-Tuning Click Save and Publish to immediately update the devices configuration. Global Settings for Client IP Address Extraction You can choose from three methods to extract the real client IP address from the HTTP header. Before You Begin It is assumed for this task that you have already created one or more user-defined software services for this protocol, and that you know how to access and modify global settings for an AMD and settings for a specific service. Configuration options related to general settings for an AMD for this protocol analyzer are found under Front-End-Monitoring Web HTTP General for Global settings, and also under the HTTP Options tab for individual user-defined services. To configure general options for client IP address extraction, modify the following settings in either global or service-specific settings, as applicable: 1. Specify one of the following settings for extracting the client IP address. To turn off automatic client IP address extraction, select Off. To extract the client IP address from a header tag, select Header Tag and type the name of the HTTP header field containing the real client IP information. To extract the client IP address by applying a regex to the HTTP header, select Header Regex and type a regular expression matching the real client IP information in the HTTP header. The string extracted by the regular expression becomes the real client IP address reported to the report server. Example 10. The following is an example of extracting the value of REMOTE_ADDR field from the HTTP header. An HTTP header might contain the following information: GET HTTP/1.1 Accept: */* Referer: Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: Connection: Keep-Alive Cookie: FPB=061j8hura11q56cv; CRZY9=t=1; REMOTE_ADDR: The following regular expression extracts the address from the REMOTE_ADDR field: %0d%0aREMOTE_ADDR:%20\([^%0d%0a]*\)%0d%0a The expression must contain a single sub-expression delimited by pairs of characters \( and \). The expression in this example states that the search string should start at the beginning of a header line and end at the end of the line (note the use of % to 140

141 Chapter 6 Configuration Fine-Tuning denote the hex values of the carriage return and line feed characters). The line should start with the string REMOTE_ADDR:. The sub-expression to extract is a string of characters different than ASCII CR or LF, and it should occur after the space following REMOTE_ADDR: For details on how expressions are used, see Using Regular Expressions to Extract User Identification [p. 75]. To use the real client IP address as both the user ID and the user IP address, select Try to Convert User Name to IP address. 2. Save or publish your changes. For more information, see Configuring General Data Collector Settings [p. 37]. Assembling Pages Page assembly options concern the methods of assigning individual hits to pages (assembling pages from a number hits). Before You Begin It is assumed for this task that you have already created one or more user-defined software services for this protocol, and that you know how to access and modify global settings for an AMD and settings for a specific service. To configure general options for monitoring this protocol: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Front-End-Monitoring Web HTTP Page Assembly. Regular hits Configure options available in the Regular hits section. The options relate to various flavors of cross-site hit assignment (assembling pages from hits requested or loaded from a number of hosts). 6. Decide whether to enable cross-site hit assignment. Select Enable cross-site hit assignment to enable page components loaded from different hosts to be recognized as belonging to the same page loaded as a result of a single transaction. 7. Decide whether to enable multi-client cross-site hit assignment. Select Enable multi-client cross-site hit assignment to enable HTML page components that belong to one page, but that are requested by a number of parallel proxy servers on 141

142 Chapter 6 Configuration Fine-Tuning behalf of a single client, to be recognized as belonging to the same page loaded as a result of a single transaction. Such a situation may occur if client traffic is directed through a number of proxy servers for each TCP/IP session. 8. Decide whether to enable multi-client real IP cross-site hits assignment. Select Enable multi-client real IP cross-site hit assignment in deployments with the AMD monitoring traffic from a number of load balancers. This way, the cross-site hit assignment is based on the real IP address of the client extracted from the X-forwarded-for HTTP header field. 9. Decide whether to enable hit assignment for different combinations of operating systems, browsers, and hardware. Select Enable hit assignment for different OS/Browser/Hardware combinations to enable HTML page components that belong to one page, but that are requested by hosts with a different operating system, browser, and hardware combination than the one detected in the first hit, to be recognized as belonging to the same page loaded as a result of a single transaction. 10. Decide whether to assign a hit to a page when no referrer is found. Select Assign hit to page when no referer found to assign a hit to a single page load if the analyzer has only one page load in progress and a new hit occurs that has no Referer field. This option is configured globally for all software services for this protocol. Redirects Configure options available in the Redirects section. These options control the way pages are assembled in the case of redirects. 11. Choose keys used to assemble pages in the case of redirects. You can enable redirects identified by the same client IP, real client IP, user name, software service name, or OS/browser/hardware combination to be recognized as belonging to the same page loaded as a result of a single transaction. The real client IP is useful when monitoring the traffic from a number of load balancers. This way, you can use the real IP address of the client extracted from the X-forwarded-for HTTP header field as a key for redirects. 12. Decide whether to report redirections from HTTPS to HTTP. In the redirect from HTTPS to HTTP, the referrer is not set. Select Report redirections from HTTPS to HTTP to report this kind of redirect. 13. Decide whether to report a redirect as a page. You can configure page redirects as single regular pages and report them separately. This makes it possible to combine the redirects with the originating or target page (depending on the setting in Report URL after redirect). In this way, redirects can become operations and you can create transactions consisting of more than one step. Redirects are commonly used with login procedures. For example, if your web service requires login and four redirects occur, you can select Report redirect as page for the 142

143 Chapter 6 Configuration Fine-Tuning second URL and obtain a two-step transaction. This makes it possible to observe in greater detail where the problem occurs after it has been reported. Reporting of redirects as pages can be configured per server, per URL, and per URL parameter. If you do not select this option per URL, it inherits the value from the related per-server configuration. Default: not selected. Multi-frame pages It is possible to recognize framesets as single pages. This can be performed dynamically, by analyzing HTML tags, or statically, by explicitly defining framesets. For more information, see Multi-Frame Pages [p. 143]. 14. Decide whether to enable multi-frame page recognition. Select Enable multi-frame page recognition to enable the entire mechanism of frame recognition automatic frame recognition and static frame recognition at all levels: global, service, and URL. 15. Decide whether to enable automatic multi-frame page recognition. Select Enable automatic multi-frame page recognition to enable frame recognition based on analyzing HTML FRAMESET and IFRAME tags. 16. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Multi-Frame Pages It is possible to recognize framesets as single pages. This can be performed dynamically, by analyzing HTML tags, or statically, by explicitly defining framesets. Before You Begin It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. Configure global settings 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Front-End-Monitoring Web HTTP Page Assembly. 143

144 Chapter 6 Configuration Fine-Tuning 6. In the Multi-frame pages section, enable multi-frame page recognition. Here you enable or disable the entire mechanism of frame recognition automatic frame recognition and static frame recognition at all levels: global, service, and URL. NOTE You must turn this option on, if you want to monitor frame sets at all: Even if you define static frames to monitor on per-service level, the mechanism will not function, unless this global option here is turned on. Note also that this feature is not supported by HTTP Express analyzer. 7. Optional: Enable or disable automatic multi-frame page recognition. Automatic frame recognition is based on analyzing the HTML FRAMESET and IFRAME tags. 8. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Optional: Configure settings related to individual software-services Service-specific settings take precedence over global settings. At this level, you can only configure automatic multi-frame page monitoring. 9. From the top menu, select Software Services Manage Software Services. 10. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 11. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 12. Click the HTTP Options tab. 13. Modify options related to automatic multi-frame page monitoring. By default, the configuration is inherited from global settings, which you can change using the provided check-boxes: Inherit from global setting and Enable automatic multi-frame page monitoring. 14. Publish the draft configuration on the monitoring device. Optional: Configure settings related to individual URLs URL settings take precedence over service rule settings and global settings. 15. Repeat Step 9 [p. 144] through Step 11 [p. 144]. 16. Click the URL Monitoring tab. This will open the Edit Rule pop-up window. 17. In the URL definitions table, right-click a URL for which you want to modify the configuration and select Open from the context menu. The Configure Monitored URL pop-up window will appear. 144

145 Chapter 6 Configuration Fine-Tuning 18. On the URL tab, in the Options section, modify settings related to automatic multi-frame page monitoring. By default, the configuration is inherited from global settings, which you can change using the provided check-boxes: Inherit from global setting and Enable automatic multi-frame page monitoring. 19. Define subframe URI matching patterns for static frame recognition. Subframe URI strict matching patterns For the static frame recognition method, this option enables you to define subframe URIs. For a subframe to be recognized, you need to enter the entire URI. If you have set URL cut method to nocut in global HTTP settings, you also need to specify the parameters to match. Otherwise, enter the URI up to the cut point. For information on setting the cut method, see Global Settings for Recognition and Parsing of URLs [p. 137]. Subframe URI regex patterns For the static frame recognition method, this option enables you to define subframe URIs as regular expressions. This method of defining URIs is very powerful and flexible, but it consumes more processing power than subframe URI strict matching. Also note that the cut method specified in global HTTP settings does not apply here, so the regular expression you enter should take into account any URI parameters expected in the URI you want to match. For more information, see Regular Expression Fundamentals [p. 211]. 20. Click OK to save the configuration. 21. Publish the draft configuration on the monitoring device. What to Do Next For individual user-defined software services for which multi-frame pages can occur, you can configure the method of calculating server time for such pages. For more information, see Calculating Server Time for Multi-Frame Pages [p. 145]. Calculating Server Time for Multi-Frame Pages You can configure how to calculate server time for HTTP-based protocols for which multi-frame pages can occur. This configuration is performed for individual user-defined software services. Before You Begin It is assumed that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. This configuration option is provided because for multi-frame pages the server time for the first HTTP object can be insignificant compared to the server times of subsequent objects. You can therefore specify a method of calculating the server time based on either the server time for the first object or based on a formula incorporating the server times of the subsequent objects. 145

146 Chapter 6 Configuration Fine-Tuning 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. In the Rules table, right-click a rule for which you want to configure calculating server time for multi-frame pages and select Open from the context menu. The Rule Configuration pop-up window is displayed. 5. Click the HTTP Options tab. 6. Choose one of the following options for Server time method for multi-frame pages: Server time for the first HTML object Use the server time for the frame holder (for example, an HTML frameset document or a document with an iframe tag). The longest server time of all HTML objects Use the single longest server time of all HTML objects in the page. Total server time for all HTML objects Use the sum of all server times for all HTML objects in the page. This is the default value. The longest server time of all objects Use the single longest server time of all server time calculations for all page objects (HTML objects or other objects). 7. Click OK to save the configuration. 8. Publish the draft configuration on the monitoring device. Calculating Availability Availability is measured and presented as the percentage of successful attempts (operations) compared to all attempts. The availability metric is calculated as the percentage of successful attempts that is: Availability = 100% * (All Attempts All failures) / All Attempts where: All attempts = all failures + all successful operations + all standalone hits not classified as a failure + all aborts not classified as a failure All failures = all failures (transport) + all failures (TCP) + all failures (application) Each attempt is classified as one of: operation, standalone hit, abort, failure (TCP), failure (transport) or failure (application) and the classification depends on the configuration. An attempt may fall to a number of categories, for example encounter both the TCP and application failure, however it is classified in only one category using the following priority: 1 - Failure (TCP), 2 - Failure (transport), 3 - Failure (application), 4 - Abort, 5 - Operation. Availability is calculated for all the analyzers capable of reporting operations, see the list below. Refer to the configuration documentation for the scope of failure reporting for each analyzer. 146

147 Chapter 6 Configuration Fine-Tuning HTTP HTTP Express Oracle Forms MS Exchange SAP GUI SAP RFC XML SOAP Jolt (Tuxedo) IBM MQ XML MQ Database analyzers DNS LDAP SMTP Simple Parser Operation Starting with release 12.2, an operation counts only if it is a successful operation. The operation count does not include failures, aborts and standalone hits. Abort An operation manually aborted by a user, for example by clicking the browser's Stop button. For more information, see Classification of Aborts [p. 217]. Note that you may classify an abort as a failure (transport) using the availability configuration. In such case, it is longer part of the Aborts count. Standalone hit An incomplete response, a hit that is not included in any reported operation. Note that an incomplete response classified as a failure (transport) is not included in the standalone hits count. Failures (TCP) An operation that failed due to one the TCP errors.. Failures (TCP) have the highest priority. Failures (transport) The failures (transport) relate to problems occurring in the transport layer of a protocol monitored by the AMD: the errors in the transport layer, 147

148 Chapter 6 Configuration Fine-Tuning SSL alerts classified as a failure, abort classified as a failure in the configuration, incomplete response classified as a failure in the configuration SSL errors are also treated as transport failures. You can specify which SSL alert codes should be classified as availability problems, separately per alerts sent by server and client. For more information, see Managing SSL Alert Codes [p. 127]. The priority of transport failures is lower then TCP failures, which means that the failures (transport) metric will not take into account any operation which was reported as transport failure even if an error in transport occurred. The configuration enables you to decide which type of error, incomplete response or abort should be taken into account when calculating availability. Additionally, you can limit the failure reporting to specific conditions. The set of error types available for failure reporting depends on an analyzer. Failures (application) The failures (application) relate to problems occurring in the application layer. Using the configuration, you can select which operation attributes should be included as an application problem. The priority of application failures is lower then transport failures, which means that the failures (application) metric will not take into account any operation which was reported as application failure even if the application error occurred. Some analyzers are preconfigured to detect typical application problems. The Failures (application) are available only for analyzers capable of detecting operation attributes. Reporting availability The key Availability metric is used both on the Data Center Analysis Reports, EUE Overview and Software Services reports. It is accompanied by a breakdown into transport, TCP and application context. 148

149 Chapter 6 Configuration Fine-Tuning Note that introduction of the new availability affects most data reported by the CAS and ADS. Unlike in previous releases, the operations count does include failures. Consequently, all the metrics calculated using the operations counter may report different values. This includes Operation Time, as failures are not included in this calculation. Operation time in environments recording many failures may increase after upgrade. Errors metric is replaced by "Application Responses" and on reports Failures metric is used instead to show only these errors that are critical. For further analysis of a failure reason and related errors or responses, drill down to the detailed error reports, including the new Application Responses report, either directly from Failures (total) column or the drill down menu. The availability is reported by means of the following metrics available in the DMI Data Views. Availability (total) The percentage of successful attempts, calculated using the following formula: Availability (total) = 100% * (All Attempts All failures) / All Attempts where All attempts = all failures + all successful operations + all standalone hits not classified as a failure + all aborts not classified as a failure All failures = all failures (transport) + all failures (TCP) + all failures (application). Availability (application) Availability limited to the application context, calculated using the following formula: Availability (application) = 100% * (All Attempts Failures (Application) / All Attempts where All attempts = all failures + all successful operations + all standalone hits not classified as a failure + all aborts not classified as a failure. Availability (TCP) Availability limited to the network context, calculated using the following formula: Availability (application) = 100% * (All Attempts Failures (TCP) / All Attempts where 149

150 Chapter 6 Configuration Fine-Tuning All attempts = all failures + all successful operations + all standalone hits not classified as a failure + all aborts not classified as a failure. Availability (transport) Availability limited to the transport context, calculated using the following formula: Availability (application) = 100% * (All Attempts Failures (Transport) / All Attempts where All attempts = all failures + all successful operations + all standalone hits not classified as a failure + all aborts not classified as a failure. Failures (total) The total number of failures, that is all Failures (transport) + all Failures (TCP) + all Failures (application) Failures (application) The number of operation attributes of all types set to be reported as an application failure. Failures (TCP) The total number of operations that failed due to Connection refused or Connection establishment timeout errors. Failures (transport) The number of operations that failed due to the problems in the transport layer. These include protocol errors, SSL alerts classified as a failure, incomplete responses selected be classified as failures. Health Index Metric that includes aspects of performance and availability. Calculated as percentage of fast (and successful) operations to all attempts. Configuring HTTP Availability You can configure HTTP availability globally or at the software service level, user-defined URL level and at the URL with parameters level. overriding the global settings. For global configuration, open the AMD configuration and go to Global Web HTTP Availability. For the software service level, select the Availability tab in the Edit Rule window. For the URL level, select the availability tab in the URL Monitoring screen and for the URL with parameters select the Availability tab in the Parameters Monitoring screen. In HTTP availability reporting, for each failure category, you can control the error classification by using following options available in the list next to each of the available transport and application errors: Any component Error is classified as failure if occurred in any component (hit) of the operation. Subset Error is classified as a failure only if occurred in components (hits) matching the regular expression provided in the accompanying field. Base component Error is classified as a failure only if it occurred in the base component (main hit) of the operation. 150

151 Chapter 6 Configuration Fine-Tuning Any component (monitored URL) Similar to Any component, but failure reporting is narrowed to monitored URLs, that is user defined URLs and reported auto-learned URLs. Subset (monitored URL) Similar to Subset, but failure reporting is narrowed to monitored URLs, that is user defined URLs and reported auto-learned URLs. Base component (monitored URL) Similar to Base component, but failure reporting is narrowed to monitored URLs, that is user defined URLs and reported auto-learned URLs. Disabled Error is not classified as a failure. Failures (transport) Incomplete responses You can determine whether the following types of incomplete responses should be classified as failures (transport). For more information, see Classification of Aborts [p. 217]. Partial response (standalone hit) An incomplete response observed for a hit without an operation context, classified as a Dead hit. This pertains to situations when server started the response but never finished due to a timeout or other problems. Aborted response (standalone hit) An incomplete response observed for a hit without an operation context, classified as a Break. This pertains to situations when server started the response but aborted it before completion with TCP reset. No response A request hit with no response from a server. This pertains to situations when server did not respond at all or responded in unrecognizable way. Partial response An incomplete response with a Dead hit status. This pertains to situations when server started the response but never finished due to a timeout or other problems. Aborted response An incomplete response with a Break status. This pertains to situations when server started the response but aborted it before completion with the TCP reset. HTTP errors The AMD is able to deliver information on seven HTTP error groups ( categories ). HTTP client errors (4xx) HTTP server errors (5xx) HTTP unauthorized errors HTTP Not Found errors HTTP client errors (category 3) HTTP server errors (category 1) HTTP server errors (category 2) 151

152 Chapter 6 Configuration Fine-Tuning You can decide whether each of these should be taken into account when calculating (failures transport). Note that HTTP client errors (4xx), HTTP server errors (5xx), HTTP unauthorized errors, HTTP Not Found errors, and HTTP server errors (category 1) have configurable contents. For more information, see Assigning HTTP Error Codes to Error Categories [p. 124]. Failures (application) You can decide whether each of five operation attributes should be reported as failures (application). For more information, see Operation Attributes in HTTP Monitoring in the Data Center Real User Monitoring SAP Application Monitoring User Guide. HTTP Configuration Options for Selected Autodiscovered Software Services HTTP options for software services are options related to a variety of settings such as redirection handling, session recognition, multi-frame page handling, and client IP address extraction. They can be set globally for the AMD or individually for particular software services. To modify the configuration options related to service-specific settings for an individual HTTP software service: 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Switch to the HTTP Options tab. 6. In the Multi-frame pages section, enable multi-frame page recognition. Here you enable or disable the entire mechanism of frame recognition automatic frame recognition and static frame recognition at all levels: global, service, and URL. NOTE You must turn this option on, if you want to monitor frame sets at all: Even if you define static frames to monitor on per-service level, the mechanism will not function, unless this global option here is turned on. Note also that this feature is not supported by HTTP Express analyzer. 7. Configure report URL after redirect. This option causes addresses after the last redirection to be reported for redirected pages. By default, redirections are reported as addresses of the originating page, before redirection 152

153 Chapter 6 Configuration Fine-Tuning takes place. The final target page will be reported regardless of how many redirects are detected in between. The option can be set globally for all software services or configured for a specific user-defined software service. Specific settings take precedence over global settings. It is not supported by HTTP Express analyzer. 8. Configure report URL prefixed with analyzed HTTP method. If this option is selected, the string POST or GET is prefixed to the reported URL. This option can be set globally for all software services or configured for a specific user-defined software service. Specific settings take precedence over global settings. 9. Configure the methods of assembling pages. For more information, see Assembling Pages [p. 141]. 10. Decide whether the port number should be ignored in the HTTP host field. To overcome inconsistency in adding the port number to the host field by a browser, you can configure the HTTP analyzer to ignore the port in the HTTP host field. 11. Select client IP address extraction method. To turn off automatic client IP address extraction, select Off. To extract the client IP address from a header tag, select Header tag and type the name of the HTTP header field containing the real client IP information. The string extracted by the regular expression becomes the real client IP address reported to the report server. Example 11. The following is an example of extracting the value of REMOTE_ADDR field from the HTTP header. An HTTP header might contain the following information: GET HTTP/1.1 Accept: */* Referer: Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) Host: Connection: Keep-Alive Cookie: FPB=061j8hura11q56cv; CRZY9=t=1; REMOTE_ADDR: The following regular expression extracts the address from the REMOTE_ADDR field: %0d%0aREMOTE_ADDR:%20\([^%0d%0a]*\)%0d%0a The expression must contain a single sub-expression delimited by pairs of characters \( and \). The expression in this example states that the search string should start at the beginning of a header line and end at the end of the line (note the use of % to denote the hex values of the carriage return and line feed characters). The line should start with the string REMOTE_ADDR:. The sub-expression to extract is a string of characters different than ASCII CR or LF, and it should occur after the space following REMOTE_ADDR: 153

154 Chapter 6 Configuration Fine-Tuning For details on how expressions are used, see Using Regular Expressions to Extract User Identification [p. 75]. To use the real client IP address as both the user ID and the user IP address, select Try to convert user name to IP address. 12. Select analyzed HTTP methods. Choose between Only POST and GET and All Methods. This option is configured individually for user-defined software services. 13. Specify data generation options. This controls the scope of data generated by the AMD that is used in CAS and ADS reporting. CAS Data If you select Disabled, the AMD will stop saving data used in most CAS reports. In normal circumstances, you should not disable CAS data generation. ADS Data When controlling ADS data generation, you can either disable it completely or decide on the depth of available data. ADS data only The AMD will generate data enabling you to access essential operation-level information. ADS data and hit details The AMD will generate data enabling you to access a deep drilldown report that represents an HTTP page hit broken down into specific HTTP elements. ADS data, hit and header details The AMD will generate data enabling you to access even deeper drilldown information retrieved from related request and response headers for the hit. 14. Click OK to save the configuration. 15. On the Software Services screen, click Publish Configuration. Additional Configuration Options for HTTP and SSL Software Services For each user-defined software service based on HTTP or SSL, you can define additional configuration options. To modify the configuration for an individual software service rule: 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 154

155 Chapter 6 Configuration Fine-Tuning 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Switch to the Options tab. 6. Modify the configuration settings. Enable monitoring of persistent TCP sessions When this is selected, TCP sessions that do not start with SYN packets are monitored. By default, this is selected. Persistent TCP sessions are TCP sessions for which the start was not recorded. They are also referred to as non-syn sessions. These sessions can be included in the TCP statistics, based on the configuration properties you enable in RUM Console. The inclusion of these sessions may render the statistics somewhat inaccurate and must be undertaken with care. Page load time threshold An operation that takes more than this many seconds is considered slow. When Inherit from global setting is selected, the global setting is used. To edit the global setting, open the AMD configuration, go to Global General and set the Operation time threshold. Server time threshold Server time threshold relates to the server time portion of an overall operation time. Server times above the threshold limit are considered to be slow due to the poor datacenter performance. 7. Click OK to save the configuration. 8. On the Software Services screen, click Publish Configuration. HTTP Express Analyzer The HTTP Express analyzer is a simplified version of the HTTP analyzer. Use this analyzer for network performance monitoring when you know that HTTP traffic is present and you require basic HTTP information about servers and URLs but not in-depth transactional or payload analysis. The HTTP Express analyzer supports the basic HTTP monitoring features, enabling you to create a simple software service used to monitor URLs. It provides basic HTTP analysis limited to the hit identification and per-url monitoring. 155

156 Chapter 6 Configuration Fine-Tuning Table 5. Comparison of the HTTP and HTTP Express Analyzers URL and URL with parameters monitoring. For more information, see Configuring URL Monitoring in the Data Center Real User Monitoring SAP Application Monitoring User Guide and Configuring URL Monitoring for HTTP Express Analyzer [p. 160]. HTTP Yes HTTP Express Yes, with limitations 1 URL auto-learning. For more information, see URL Auto-Learning [p. 88]. Recognition and parsing of URLs. For more information, see Global Settings for Recognition and Parsing of URLs [p. 137]. Yes Yes Yes Yes Character encoding support. For more information, see Character Encoding Support for HTTP Services [p. 122]. Yes Yes Content type monitoring. For more information, see Content Type URL Monitoring [p. 115] and Monitoring of Non-HTML Objects Based on Content Type [p. 116]. Yes No Extracting additional dimensions. For more information, see Extracting Grouping Attributes in the Data Center Real User Monitoring SAP Application Monitoring User Guide and Extracting Miscellaneous Parameters in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Operation attributes reporting. For more information, see Operation Attributes in HTTP Monitoring in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Yes, in HTTP mode. Yes No No Custom metrics reporting. For more information, see Custom Metrics in HTTP Monitoring in the RUM Console Online Help. Page name recognition. For more information, see Automatic Page Name Recognition [p. 106]. Defining end-of-page components. Yes Yes. From responses in HTTP and HTTP legacy mode. From requests only in HTTP mode. Yes No No No 1 Many of the DC RUM monitoring features are available at the level of a software service, URL, and URL with parameters. All monitoring features not supported by the HTTP Express analyzer are naturally not available for URL and URL with parameters monitoring. 156

157 Chapter 6 Configuration Fine-Tuning Table 5. Comparison of the HTTP and HTTP Express Analyzers (continued) HTTP HTTP Express For more information, see End-of-Page Components [p. 104]. Excluding elements from orphaned redirect reporting. For more information, see Excluding Elements from Orphaned Redirects Reporting [p. 99]. Yes, in HTTP mode. No User identification. For more information, see User Name Recognition Configuration [p. 65]. Yes No Transaction reporting, including asynchronous HTTP transactions. For more information, see Logging Transactions, ADS Data and ADS Header Data [p. 118] and Using Correlation ID to Monitor Asynchronous HTTP Transactions [p. 180]. Yes No Browser, operating system, and hardware recognition. For more information, see Configuring Synthetic Agents, Browsers, Operating System and Hardware Recognition [p. 100]. Assembling pages. Assembling Pages [p. 141] Multi-frame pages reporting. For more information, see Multi-Frame Pages [p. 143]. SSL monitoring. Yes Yes Yes Yes No No No No Within the features supported, the HTTP Express analyzer configuration are similar to the standard HTTP analyzer. General Configuration Options for HTTP Express Software Services HTTP general configuration options for the HTTP Express analyzer are limited to the options related to the HTTP sessions and session timeouts. They can be set globally for the AMD or individually for particular software services. Before You Begin It is assumed for this task that you have already created one or more user-defined software services for this protocol and that you know how to access and modify global settings for an AMD and settings for a service. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. 157

158 Chapter 6 Configuration Fine-Tuning The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Front-End-Monitoring Web HTTP Express General. NOTE Configuration options related to general settings for an AMD for this protocol analyzer are also under the HTTP Options tab for individual user-defined services. 6. Configure options available in the General section. The list of configuration options includes: Last packet HTTP session timeout If the time since the last packet for an HTTP session is longer than this value (in seconds), the hit is considered finished and closed. This timeout period is configured globally for all software services. Report URL prefixed with analyzed HTTP method If this option is selected, the string POST or GET is prefixed to the reported URL. This option can be set globally for all software services or configured for a specific user-defined software service. Service-specific settings take precedence over global settings. Treat a client RST packet sent by the session as closing session If this option is selected, the protocol analyzer treats a client RST packet sent by the session as closing the session instead of aborting it if there was no content length header. It is configured globally for all software services. 7. Optional: Configure URL Auto-Learning. For more information, see Configuring URL Auto-Learning [p. 88]. 8. Optional: Configure Character Encoding Support. For more information, see Character Encoding Support for HTTP Services [p. 122]. 9. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. Configuring HTTP Express Availability By configuring the availability, you can determine which attempt failures are included in the availability metric calculation. You can configure HTTP Express availability globally or at the software service level. For global configuration, open the AMD configuration and go to Global Web HTTP Express Availability. For the software service level, select the Availability tab in the Edit Rule window. 158

159 Chapter 6 Configuration Fine-Tuning For HTTP Express, you can determine whether the following HTTP errors, all disabled by default, should be included in the calculation of Failures (transport) metric. HTTP client errors (4xx) HTTP server errors (5xx) HTTP unauthorized errors HTTP not found errors For more information, see Assigning HTTP Error Codes to Error Categories [p. 124]. Configuring User-Defined Software Services Based on HTTP Express Analyzer HTTP options for software services based on the HTTP Express analyzer can be set globally for the AMD or individually for particular software services. To modify the configuration options related to service specific settings for an individual HTTP software service based on the HTTP Express analyzer: 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Configure URL monitoring. For more information, see Configuring URL Monitoring for HTTP Express Analyzer [p. 160]. 6. Optional: Configure URL parameters monitoring. For more information, see Configuring Monitoring of URL Parameters for HTTP Express Analyzer [p. 164]. 7. Optional: Configure URL Auto-Learning. For more information, see Configuring URL Auto-Learning [p. 88]. 8. Optional: Configure Character Encoding Support. For more information, see Rule-based Character Encoding for HTTP Services [p. 123]. 9. Optional: Switch to the HTTP Options tab. 10. Configure report URL prefixed with analyzed HTTP method. If this option is selected, the string POST or GET is prefixed to the reported URL. This option can be set globally for all software services or configured for a specific user-defined software service. Specific settings take precedence over global settings. 11. Optional: Select analyzed HTTP methods. 159

160 Chapter 6 Configuration Fine-Tuning Choose between Only POST and GET and All Methods. This option is configured individually for user-defined software services. 12. Optional: Switch to the Options tab. If you select Enable monitoring of persistent TCP sessions, TCP sessions not starting with SYN packets are monitored. 13. Publish the draft configuration on the monitoring device. Configuring URL Monitoring for HTTP Express Analyzer You can create named URL definitions to monitor specific URLs and you can specify URLs to be excluded from monitoring. You can also specify a virtual HTTP server to handle scenarios in which many web sites reside under a single IP address. Before You Begin It is assumed for this task that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. For more information, see Configuring User-Defined Software Services Based on HTTP Express Analyzer [p. 159]. To specify definitions for the monitoring of URLs for a user-defined HTTP software service, create or edit one or more URL definitions: 1. Open the URL Monitoring screen for the service. In the Rules table for the service, select the URL Monitoring tab. 2. Add or open a definition for a URL to be monitored or to be excluded from monitoring. In the URL Definitions table, right-click and choose Add Monitored URL to create a new definition for monitoring URLs, Add Excluded URL to create a new definition for URLs excluded from monitoring, or Open to open an existing definition. The Configure Monitored URL or Configure Excluded URL window will open. The order in which you arrange URLs is important. When adding several URLs of the same type, make sure that you arrange the definitions from the most specific to the most general, because the URLs are processed from top to bottom. In particular, if you add a specific excluded URL, make sure that you place it before a more general monitored URL, or the exclusion will be ignored. 3. Select a URL type. The option you select here determines the type of URL information that you will need to enter further down in the URL Definition section: Virtual HTTP Server Static URL Part URL as Regular Expression. 4. Enter a URL definition string. 160

161 Chapter 6 Configuration Fine-Tuning The information you enter here depends on the URL Type selection you made in the previous step. Virtual HTTP Server This option refers to monitoring a host where many web sites reside under a single IP address. Using a virtual HTTP server causes all reported pages that have no separate definitions to be aggregated to one record and reported together. This does not apply to those pages from the IP address that are defined separately in a monitoring configuration. Such individual definitions do not require that you select this option. A valid virtual HTTP server address to enter would be, for example, without a trailing slash. Static URL Part A fully qualified URL (one containing the protocol to be used, the server to be contacted, and the file to be requested) such as This URL will be added to the list of monitored URLs regardless of the limit of monitored URLs. URL as Regular Expression An extended POSIX regular expression describing a set of URLs. For more information, see Regular Expression Fundamentals [p. 211]. The syntax allows you to use parentheses () to select one or more sub-expressions (specific portions of the results). If this mechanism is used, only the specified portions are reported; if more than one portion is specified, the portions are concatenated. NOTE When using a regular expression to specify a set of URLs to monitor: Explicitly include the string in the expression. You can not, for example, start the expression with.* and expect that the string will be assumed or resolved as a part of the regular expression. The parentheses you use to select the part of the URL to be extracted must include and the name of the host. However, the name of the host does not have to be provided explicitly, but can be resolved by the regular expression. Thus, for example, ( is correct, and so is ( The regular expression must be constructed such that, after extracting the portions delimited by parentheses, the resulting string does not end with a slash character ( / ). This rule applies to all URLs except home pages (URLs consisting only of a protocol specification and a host name). Such URL specifications should end with a slash. For example ( is valid, but ( is not valid. Note also that a specification ending with (myreport/*) is not valid because it can be matched by a string ending with a slash, as the asterisk can match an empty string. 161

162 Chapter 6 Configuration Fine-Tuning You can click the Test button located beside the regular expression pattern field to use the Regular Expressions Test tool to test patterns that will be used by the AMD. For more information, see Testing Regular Expressions [p. 213]. Example 12. A simple example of using a regular expression to specify monitored URLs The use of parentheses in a regular expression is demonstrated in the following example: ( The above expression will match URLs such as but only the bracketed portion ( ) will be reported. Example 13. A more complex example of using a regular expression to specify monitored URLs The following is a more complex example that demonstrates concatenation of bracketed portions: A site contains URLs of the form: 04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLN4o39w0BSYGYRiGBpFoYsamaEIG8Y4IEW99X4_83FT9AP2C3NDQiHJHRQDwwo2X/ delta/base64xml/l3djdyevuud3qndnqsevnelvrs82xzjfnvvn?wcm_global_context=/ assurance/wcm/connect/my Life.fr/Aide/Accueil Aide&WT.tz=1&WT.bh=12&WT.ul=en-us&WT.cd=32&WT.sr= 1400x1050&WT.jo=Yes&WT.ti=AssuranceRetraitePERP&WT.js=Yes&WT.jv=1.3&WT.fi= Yes&WT.fv=3.0&WT.sp=@@SPLITVALUE@@ where only the part coming after...wcm/connect/, in this case My Life.fr/Aide/Accueil Aide, is relevant for differentiating this page from other pages of this site, the rest being session ID and various parameters. If you use ( assurance/wcm/connect/([^&]*) to define monitored URLs, the reported URL for this page will be: life.fr/aide/accueil Aide NOTE Because resolving regular expressions is processor-intensive, defining a large number of URLs with regular expressions can have an adverse effect on the performance of the AMD. If you are configuring excluded URLs, this step completes this particular definition. If you are defining monitored URLs, proceed to the next step. 5. Optional: Select additional options. 162

163 Chapter 6 Configuration Fine-Tuning In the Options section, select or clear the desired options as required: Report URL Prefixed with Analyzed HTTP Method All methods of passing HTTP parameters can be distinguished if this option is selected. To use the value defined for the entire monitoring rule, ensure that the Inherit Setting from Rule check box is selected. The All methods option allows for processing all detected HTTP methods including the WebDAV HTTP extesion. The extended WebDAV methods automatically identified include: PROPFIND Retrieves properties and a directory hierarchy of a remote system. PROPPATCH Changes and deletes multiple properties is a single operation. MKCOL Creates directories or collections. COPY Copies a resource from one URI to another. MOVE Moves a resource from one URI to another. LOCK Puts a lock on a resource. UNLOCK NOTE Removes a lock from a resource. Monitoring WebDAV software services requires a specific configuration options. In order to properly report a hit as a separate operation, you must define a URL with regex matching all URLs ( and content types. Report long pages, incoming over many monitoring intervals This option allows for reporting so-called long pages (pages that load continually). This type of page is used, for example, to provide constantly updated information such as stock market reports. There are a number of different techniques for providing this functionality, such as by using streaming objects or server PUSH. All pages to be treated as long pages must be specified explicitly. Long pages are reported on reports, but no transaction-related information is included in reports. The only information collected for such pages are network metrics. Report Only URL Part When Parameters Do Not Match Select this option to cause this URL to be reported even if none of the parameter sets specified for the URL has been matched with the actual parameters seen in the monitored traffic. Parameters are defined in a separate configuration window. For 163

164 Chapter 6 Configuration Fine-Tuning more information, see Configuring Monitoring of URL Parameters for HTTP Express Analyzer [p. 164]. 6. Publish the draft configuration on the monitoring device. What to Do Next If you require URL recognition that includes parameter matching, you need to define parameter information for this URL definition. For more information, see Configuring Monitoring of URL Parameters for HTTP Express Analyzer [p. 164]. Configuring Monitoring of URL Parameters for HTTP Express Analyzer You can specify up to four parameters for a given URL definition using, among other ways, regular expressions. Pages with particular sets of parameters can be reported as separate pages in DC RUM reports. Before You Begin It is assumed for this task that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. For more information, see Configuring User-Defined Software Services Based on HTTP Express Analyzer [p. 159].Configuring URL Monitoring for HTTP Express Analyzer [p. 160] To specify parameter definitions for a URL definition, create or edit one or more parameter definitions as follows: 1. Open the Rules Configuration window for the service. 2. Click the URL Monitoring tab. 3. In the URL Definitions section, select the desired URL definition. To quickly navigate to an entry in the URL definitions table, click in the table and then type some or all of the IP definition. Click the magnifying glass icon or press [Ctrl+F] to open a search box to limit the table view to only those rows that contain a match (in any column) to the search string. 4. In the URL Parameters table, right-click and choose Add to create a new parameter definition, or choose Open to open an existing definition. The URL Parameters window will open. 5. Select a parameter matching method from the Parameter Match list and specify details for up to four parameters. The following matching methods are supported: Exact Report the specified parameter or the parameter and value. Usage syntax 'name=value' or just 'name'. 164

165 Chapter 6 Configuration Fine-Tuning Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. So, the wildcard character * is taken literally. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples You can specify 'john', to match though note that in this case will not be reported because the parameter value '=123' was not explicitly specified. To match it, you would need to specify 'john=123'. Start Report parameters that begin with a specified string; report only the matched pattern, truncate any remainder of the parameter. Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. 165

166 Chapter 6 Configuration Fine-Tuning Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples 'fred=5' will match but it will be reported as The value 'fred' will match as well as and it will be reported as Start (expand) Report parameters which begin with a specified string; report the entire parameter, not only the matched pattern. Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. 166

167 Chapter 6 Configuration Fine-Tuning End Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples 'fred=5' will match and it will be reported as The value 'fred' will match as well as and it will be reported as and respectively. Report parameters which end with a specified string; report the entire parameter, not only the matched pattern. Usage syntax 'name=value' or any final part of it this string, including string of the form '=value' or just 'value'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. 167

168 Chapter 6 Configuration Fine-Tuning Examples For to be matched, you can specify the following ends: '0', '00', '100', '=100', 'n=100' and so on, up to 'john=100'. Thus is reported. Value RegEx Report parameters which begin with a specified string; optionally attempt to match the remainder of the parameter with a regular expression; report the start string and selected portions of the regular expression, if any. Usage syntax Parameter is entered as name=value or any initial part of it this string including string of the form name= or just name. A regular expression (regex) is entered as an extended POSIX regular expression. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed on the Parameter part; the regex part is matched as a case-sensitive POSIX regular expression. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples parameter specification fred= and a regular expression AB(C?E) will match but it will be reported as because the AB portion of the regular expression was not included in round braces. 168

169 Chapter 6 Configuration Fine-Tuning Custom RegEx Report parameters that match the given regular expression; report those portions that have been selected within the regular expression. Usage syntax Enter an extended POSIX regular expression to match the desired string. Mark portions to be reported by using round braces ( and ). Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters The request URL, POST body, or HTTP header are not split into parameters prior to pattern matching. Instead, they are treated as single units of data and the regular expression is applied to their entire contents. Only the path part of the request URL is excluded from the matching process. Limitations The regular expression is entered according to POSIX syntax. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples Regular expression fred=ab(c?e) will match but it will be reported as Regular expression (.*=)AB(C?E) will match as well as and it will be reported as and as respectively. 6. Publish the draft configuration on the monitoring device. 169

170 Chapter 6 Configuration Fine-Tuning 170

171 CHAPTER 7 Monitoring Sequence Transactions You can manage the sequence transactions (operation sequences) that are defined on an individual AMD or manage each transaction that is monitored by a group of AMDs. Viewing All Defined Sequence Transactions To view all transactions, select Reporting Configuration Sequence Transactions from the console top menu. Click in the Sequenced Transactions list and then type the first letters of a sequenced transaction name to find a sequenced transaction whose name matches what you have typed. Click the magnifying glass icon or press [Ctrl+F] to open a search box to limit the table view to only those rows that contain a match (in any column) to the search string. For each transaction, the following information is shown: Sequence Transaction Name The name of a transaction. Application The application that includes the listed transaction. Type The protocol used to define the listed transaction: ASYNC-HTTP, CERNER, CERNER-RTMS, HTTP, OF, SAP GUI, SQL or XML. Packaged Applications Whether the listed transaction is a packaged application whose transactions are recognized by the report server automatically. When you select a transaction by clicking it once, you can see the list of AMDs that monitor this transaction. Viewing Sequence Transactions Defined on an Individual AMD To view the defined transactions monitored by a single AMD, select Devices and Connections Manage Devices from the console top menu. Next, select Open configuration from the context menu for the AMD to access the AMD Configuration screen. Finally, select Configuration Sequence Transactions. 171

172 Chapter 7 Monitoring Sequence Transactions The main Sequence Transactions table lists all of the currently defined transactions and their details: Sequence Transaction Name The name of a transaction. Application Name The application that includes the listed transaction. Type The protocol used to define the listed transaction. Steps The number of individual operations involved in the listed transaction. Priority The priority of the transaction. Possible values are 1 (highest priority), 2, and 3. Timeout The maximum time for the transaction to complete. Packaged Applications Identifies whether the listed transaction is a packaged application whose transactions are recognized by the report server automatically. Viewing Sequence Transaction Details On the Sequence Transactions screen, select Edit from the Actions menu for a given transaction to open the Edit Transaction window and examine the steps that make up the transaction. The listed details are as follow: Name, Application, Description: Timeout [s], Slow after [s], Priority URL, Timeout, Repetition. Managing Existing Sequence Transactions To manage all of the defined transactions, use the Sequence Transactions screen. To create new transactions, click Add Sequence Transaction. The Create Sequence Transaction screen appears, where you can select the AMD devices that will monitor this new transaction. To delete a transaction, select the check box for the transaction and click Delete. NOTE If the transaction you are deleting is monitored by more than one AMD, a new draft configuration must be published to all of the affected AMDs. To edit a transaction, from the Actions context menu for the transaction, select Edit. To copy a transaction to another device, from the Actions context menu for the transaction, select Copy. 172

173 Configuring the AMD to Monitor HTTP-Based Transactions The following configuration properties affect the monitoring of HTTP-based transactions. These properties reside in the /usr/adlex/config/page2trans.properties configuration file. process.http.records This property determines whether the HTTP-based transaction information is to be processed on this AMD. Note that while this configuration property does not affect the processing of XML-based transactions directly, it does influence processing performance, since, if set to true, the AMD will also have to process HTTP-based transactions. Therefore, if it is not required that HTTP-based transactions are to be processed in addition to XML-based ones, this property should be set to false. Default value: true. process.xml.records This determines whether XML-based transaction information is to be processed on this AMD. Note that while this configuration property does not affect the processing of HTTP-based transactions directly, it does influence processing performance, since, if set to true, the AMD will also have to process XML-based transactions. Therefore, if it is not required that XML-based transactions are to be processed in addition to HTTP-based ones, this property should be set to false. For more information, see Configuring the AMD to Monitor XML-Based Transactions in the RUM Console Online Help. Default value: true. Adding Transactions You can add a transaction to either an individual AMD or a range of AMDs using the RUM Console. To define a new transaction: 1. In the RUM Console, select Reporting Configuration Sequence Transactions. 2. Click Add Sequence Transaction. The Create Sequence Transaction pop-up window appears. 3. Enter the application and transaction names and a description. If you have configured the Dynatrace connection, click Browse to select a predefined application and a specific transaction within this application. For more information, see Configuring the BSM Connection in RUM Console in the Data Center Real User Monitoring Administration Guide. 4. Select the devices that will monitor the transaction. When you publish the new configuration, it is only applied to these devices. 5. Click OK. Chapter 7 Monitoring Sequence Transactions 173

174 Chapter 7 Monitoring Sequence Transactions On the screen, specify the configuration details for the transaction. 6. Provide the timing and priority values: Timeout [s] The maximum time for the transaction to complete. Transactions must complete in this time to be logged as successful transactions. Slow after [s] If the transaction execution time exceeds this value, the transaction is classified as slow. Specify this threshold as fractions of seconds, for example: 0.5. Priority Determines which transaction is recorded if two or more transaction definitions match the transaction detected in the monitored traffic. The valid priority values are 1 (highest priority), 2, and 3. A multiple transaction match can happen if, for example, you first create a generic transaction definition that can match a number of more specific transactions and then you create another transaction definition that matches a particular sub-type of that generic transaction type. If an observed transaction is found to match the latter definition, it also matches the first (more generic) definition, and the system will need to determine under which transaction name to record the observed transaction instance. By increasing the priority of the second, more specific definition, you can count the occurrences of this particular transaction sub-type, which are then not counted in the statistics for the generic transaction type. So you can use this feature to increase the priority of specific customized transaction definitions that should take precedence over more generic transaction templates. 7. Specify the operations that comprise the transaction steps. Depending on the transaction type, enter the following items to define the transaction steps: HTTP Transaction URL The URL can contain a contain optional wild-card character * or a regular expression. XML Transaction XML Action The XML action can contain a can contain a contain optional wild-card character *. SAP GUI transaction SAP GUI operation Cerner RTMS Transaction Cerner operation The Cerner operation can contain an optional wildcard character *. Oracle Forms Transaction Oracle Forms operation The Oracle Forms operation can contain an optional wildcard character *. 174

175 Chapter 7 Monitoring Sequence Transactions SQL Transaction (timer) SQL operation (command) and query type. Both SQL operation and query type can contain an optional wild-card character * to signify any number of any characters or a regular expression. You can use either of the two methods in one line. For example, you can use the regular expression based pattern for the query type, regex:rp[abc]$ and simple search pattern for the SQL operation, set*. For more information, see Reported Database Operation Types in the Data Center Real User Monitoring Database Monitoring User Guide. To maintain the sequence of these operations, use the navigation buttons on the right. When using the regular expression in defining the HTTP, Oracle Forms and SQL transaction steps, you have to start the search string with the phrase regex: and follow it a valid regular expression which is applied to the URL, Oracle Forms, SQL operation or query type, for example: regex: For more information, see Using Regular Expressions in URLs [p. 64]. Using the wildcard character *, you can signify any number of any characters. For more information, see Using Wildcards in URLs [p. 63]. You can Add, Delete, Move Up, Move Down, or Copy the defined steps by selecting the step and clicking one of these actions. You can also make changes in the table itself: click in any of the column cells to edit the values. Using the table, you can determine whether the selected operation was a request or a response and whether this particular operation may be repeated within this transaction. 8. Click OK to add your transaction definition to a draft configuration. 9. On the Sequence Transactions screen, click Publish Configuration. What to Do Next You can also add a transaction using the Sequence Transaction Inspector. For more information, see Monitoring Sequence Transactions [p. 171]. Adding Transactions for a Range of AMDs Transactions can be added to a number of AMDs at the same time. To add a transaction to several AMDs: 1. In the RUM Console, select Reporting Configuration Sequence Transactions. 2. Click Add Sequence Transaction. The Create Sequence Transaction pop-up window appears. 3. Enter the transaction and application names. 4. From the Type list, select the analyzer type for the transaction. 5. Select the AMDs to monitor this transaction by selecting the appropriate check boxes. 175

176 Chapter 7 Monitoring Sequence Transactions 6. To use a transaction inspector, select Open with Sequence Transaction Inspector and, from the drop-down list, select the device providing the XML data to the transaction inspector in the transaction editor. 7. Click OK to proceed to the transaction editor. 8. You can change the transaction and application names, or you can accept the current names and proceed to the next step. 9. Provide the timing and priority values: Timeout [s] The maximum time for the transaction to complete. Transactions must complete in this time to be logged as successful transactions. Slow after [s] If the transaction execution time exceeds this value, the transaction is classified as slow. Specify this threshold as fractions of seconds, for example: 0.5. Priority Determines which transaction is recorded if two or more transaction definitions match the transaction detected in the monitored traffic. The valid priority values are 1 (highest priority), 2, and 3. A multiple transaction match can happen if, for example, you first create a generic transaction definition that can match a number of more specific transactions and then you create another transaction definition that matches a particular sub-type of that generic transaction type. If an observed transaction is found to match the latter definition, it also matches the first (more generic) definition, and the system will need to determine under which transaction name to record the observed transaction instance. By increasing the priority of the second, more specific definition, you can count the occurrences of this particular transaction sub-type, which are then not counted in the statistics for the generic transaction type. So you can use this feature to increase the priority of specific customized transaction definitions that should take precedence over more generic transaction templates. 10. Specify the operations that comprise the transaction steps. Depending on the transaction type, enter the following items to define the transaction steps: HTTP Transaction URL The URL can contain a contain optional wild-card character * or a regular expression. XML Transaction XML Action The XML action can contain a can contain a contain optional wild-card character *. SAP GUI transaction SAP GUI operation Cerner RTMS Transaction Cerner operation 176

177 Chapter 7 Monitoring Sequence Transactions The Cerner operation can contain an optional wildcard character *. Oracle Forms Transaction Oracle Forms operation The Oracle Forms operation can contain an optional wildcard character *. SQL Transaction (timer) SQL operation (command) and query type. Both SQL operation and query type can contain an optional wild-card character * to signify any number of any characters or a regular expression. You can use either of the two methods in one line. For example, you can use the regular expression based pattern for the query type, regex:rp[abc]$ and simple search pattern for the SQL operation, set*. For more information, see Reported Database Operation Types in the Data Center Real User Monitoring Database Monitoring User Guide. To maintain the sequence of these operations, use the navigation buttons on the right. When using the regular expression in defining the HTTP, Oracle Forms and SQL transaction steps, you have to start the search string with the phrase regex: and follow it a valid regular expression which is applied to the URL, Oracle Forms, SQL operation or query type, for example: regex: For more information, see Using Regular Expressions in URLs [p. 64]. Using the wildcard character *, you can signify any number of any characters. For more information, see Using Wildcards in URLs [p. 63]. You can Add, Delete, Move Up, Move Down, or Copy the defined steps by selecting the step and clicking one of these actions. You can also make changes in the table itself: click in any of the column cells to edit the values. Using the table, you can determine whether the selected operation was a request or a response and whether this particular operation may be repeated within this transaction. 11. Click OK to add your transaction definition to a draft configuration. 12. Return to the Sequence Transactions screen and click Publish Configuration. The configuration is sent to all of the selected devices. Filters and Transaction Inspector for HTTP Transactions The Transaction Inspector enables you to select individual steps and construct your own transactions from live traffic or historical data. The Transaction Inspector consists of two main areas: the Filters and the Transaction Inspector itself. NOTE The search strings and transaction step definitions use regular expression (regex) format. For more information, see Regular Expression Fundamentals [p. 211] and Using Wildcards in URLs [p. 63]. 177

178 Chapter 7 Monitoring Sequence Transactions Filters Transactions can be defined manually by entering each step, however the Filters area enables you to examine the Current Stream or Recent Data and select the detected steps to build a transaction. The transaction filter consists of two tabs: Data Filter The Data Filter tab enables you to define your filter by selecting the source and range of data to be filtered. From the list, you can select the report server and create a user filter. Select User Name or User IP Address and either enter the data manually or click Browse to open the Select User window, where you can select the user identified in transaction traffic by the report server. You can highlight or search for the specific user or user IP address and filter the search query based on any of the columns in the transactions table. After the user or user IP address is selected, you can choose to either extract transactions from a Current Stream that is being monitored, or from Recent Data stored on the report server. The Recent Data option requires you to provide a Begin and End date for the time range to be processed. NOTE In the case of HTTP asynchronous transactions, you can only extract transactions from Recent Data and you cannot apply the user filters. Result Filter The Result Filter tab consists of a find field, a transaction detail field, and an interactive legend to filter transactions that have been classified as: Table 6. Result Filter Color Guide The transaction was recognized and matched with the transaction currently being defined. One or more steps One or more steps A Step or a URL Excluded URLs or in the transaction in the transactions was recognized as steps which did currently being for which an error an already defined not match any defined were not occurred. and saved definition. completed. transaction definition. By selecting and clearing the corresponding check boxes, you can filter the URLs from the data source. The color coding of the steps is based on your current transaction definition in the Transaction Definition area. To view the results and enable the filter to receive data, click, located on the right side of the Filters area. While viewing the data in Transaction Inspector, at any time you can force the data to be recalculated using your current transaction definition by clicking or, you can stop the filter by clicking, located on the right side of the Filters area. 178

179 Chapter 7 Monitoring Sequence Transactions Transaction Inspector Transaction Inspector consists of two tables that display the URLs and Sequenced Transactions detected in the data source that is defined in the Filters section. The Transaction Inspector enables you to select one or a number of detected steps and add them to your transaction definition. Select the check box corresponding to the URL that you want to add and then click located above the table. You can add the steps from both the URLs table and from the Transactions table. After the URL is moved to the Transaction Definition table, you can modify it, position it within a sequence of other steps, clone it as another step, or delete it using the operation buttons to the right of the Transaction Definition table. Modifying, Deleting, and Cloning Transactions for a Single AMD Modifying a Sequence Transaction To modify the definition of an existing transaction: 1. Open AMD configuration and click Edit as Draft to switch to draft mode. 2. In the Configuration tree, select Sequence Transactions. This opens the Sequence Transactions table, listing all of the defined transactions for this AMD. 3. Right-click the transaction to manage and select Open from the context menu. You can modify any of transaction details. For more information, see Adding Transactions [p. 173]. Deleting a Sequence Transaction To delete selected transactions: 1. Open AMD configuration and click Edit as Draft to switch to draft mode. 2. In the Configuration tree, select Sequence Transactions. 3. Click the transaction that you want to delete. To delete multiple transactions with one step, hold the [Ctrl] key as you click additional transactions. 4. Right-click and select Delete to remove the selected transactions from the list. Cloning a sequence transaction To clone selected transactions: 1. Open AMD configuration and click Edit as Draft to switch to draft mode. 2. In the Configuration tree, select Sequence Transactions. 3. Click the transaction that you want to clone. 179

180 Chapter 7 Monitoring Sequence Transactions To clone multiple transactions with one step, hold the [Ctrl] key as you click additional transactions. 4. Right-click and select Clone to duplicate the selected transactions. A cloned transaction is indicated by the original transaction name with (Clone) appended to it. There are differences between cloning and copying. For more information, see Monitoring Sequence Transactions [p. 171]. Using Correlation ID to Monitor Asynchronous HTTP Transactions In an asynchronous HTTP transaction, unlike in an ordinary HTTP transaction, the role of the client and the server is not persistently fixed, so the IP addresses of the client and server may differ between TCP sessions. The only parameter that associates the exchanged data is the correlation identifier, which is found in the HTTP header or body of the message. You can extract the correlation ID from the transaction response and request, making it possible to identify and report on a complete asynchronous transaction. To define an asynchronous HTTP transaction, first configure the AMD to extract a correlation identifier and then define the transaction itself, maintaining the sequence of the operations. 1. Determine the correlation identifier. Asynchronous data exchange consists of two or more independent HTTP/HTTPS operations. The correlation identifier is the information that associates one session with another and is passed in the HTTP/HTTPS header. The example transaction consists of two operations triggered by the following URLs: Request: Response: As shown in the example, the correlation identifier is passed by the id and originalid parameters. 2. Define monitoring of the operation URLs. Define the monitoring of each URL. For more information, see Configuring URLs for a Software Service Definition [p. 54]. 3. Configure extraction of the correlation identifier. When configuring the URL monitoring, both Configure Monitored URL and URL Parameters windows contain the Correlation ID tab which you use to extract the correlation ID from the URL. To extract the correlation ID from the example URLs in Step 1 [p. 180], the Simple Value search method is the most convenient. It matches the parameter that begins with a string defined in the phrase and reports only the value of parameter, not the prefix. 180

181 Chapter 7 Monitoring Sequence Transactions For the request URL, add a simple id= phrase and originalid= for the response. Both search phrases match the parameter that carries the jgk9ky19z8c5 string which is the example correlation identifier. For more information, see Metric Value Extraction Methods in the RUM Console Online Help. Click OK to save your changes. 4. Add the asynchronous HTTP transaction. For more information, see Adding Transactions [p. 173] and Adding Transactions for a Range of AMDs [p. 175]. When choosing the transaction type, select Asynchronous HTTP Transaction from the list. You must define the steps that comprise the full definition of the transaction. You must set the direction of the second step of the definition to response. In the example, is added as a first transaction step and its direction is set to request and is set as a second transaction step and its direction is set to response. Click OK to add the transaction definition to a draft configuration. 181

182 Chapter 7 Monitoring Sequence Transactions 182

183 CHAPTER 8 Web Tiers A tier is a specific point where DC RUM collects performance data. It is a logical application layer, a representation of a fragment of your monitored environment. There is one tier on the CAS that reports HTTP or SSL (decrypted) data: the Website tier. If your CAS is configured to report on web application traffic and receives web application data, the Website tier will automatically be displayed on the Tiers report. For more information, see Application and Transaction Management in the Data Center Real User Monitoring Administration Guide. 183

184 Chapter 8 Web Tiers 184

185 CHAPTER 9 Web Application Traffic on CAS Reports The HTTP and SSL (decrypted) traffic statistics will appear on CAS software service, transaction and application, or tier reports, depending on how you have configured your monitoring rules. For detailed description of the reports and explanation of dimension and metric definitions refer to the Data Center Real User Monitoring Central Analysis Server User Guide or CAS online help. Reports menu From the Reports menu, view the following reports: Applications: see the performance of applications for which data was detected on front-end tiers, if you have defined web applications and transactions on CAS. For more information, see Application and Transaction Management in the Data Center Real User Monitoring Administration Guide. Tiers: analyze statistics for the website tier. Sites: analyze statistics for specific sites. Top N View: view the most problematic software services, operations and sites. Software Services: analyze statistics for HTTP and SSL software services. Network Landing Page: analyze your network performance. User Activity: analyze traffic statistics for particular users. 185

186 Chapter 9 Web Application Traffic on CAS Reports 186

187 APPENDIX A Diagnostics and Troubleshooting Guided Configuration Issues After I upgraded to Data Center Real User Monitoring 11.5, why doesn't Guided Configuration work? On upgrade, the Guided Configuration connection is, by default, disabled on the AMDs. Enable the Guided Configuration connection on an AMD, see Step 14 [p. 83]. Note that if you add an AMD after you upgrade to DC RUM 11.5, the connection will be enabled for you on the new device. Another reason that it does not work is that the number of AMDs in your network exceeds the maximum number (15) of devices with a Guided Configuration connection enabled. Also note that automatic trace recording is, by default, disabled in all installations, so to see data on the Guided Configuration perspective, either enable automatic trace recording or record a trace manually. For more information, see Capturing Traffic Traces [p. 50]. The Guided Configuration is incorrectly displayed after a period of user inactivity. The watchdog mechanism for RUM Console Server frequently polls the server process for its activity. If no activity is detected after a certain timeout (default is 30 seconds), the RUM Console Server process is restarted. This restart causes a connection break between the active RUM Console and the RUM Console Server. The connection is automatically reestablished after RUM Console restart, but the Guided Configuration process may have to be restarted. The JVM restart will result in an entry in log file platform-system.log (located in the..\programdata\application Data\Compuware\Vantage Agentless EUE Configuration\workspace\log\kernel\) similar to this: ERROR wrapper 2010/06/29 17:13:14 JVM appears hung: Timed out waiting for signal from JVM. STATUS wrapper 2010/06/29 17:13:14 Dumping JVM state. ERROR wrapper 2010/06/29 17:13:19 JVM did not exit on request, terminated STATUS wrapper 2010/06/29 17:13:24 Launching a JVM... This usually happens on overloaded systems when another process is using 100% of the CPU, caused by low system memory and high disc swapping. In this situation, it is recommended that RAM be increased on the machine. Why can't I record a new traffic trace? You can diagnose and solve the problem in several ways: 187

188 Appendix A Diagnostics and Troubleshooting Read the message in the recording pop-up window. It may contain information about connection problems, the AMD receiving no traffic, or the Guided Configuration waiting for the top statistics data from the device. Check the connection status for the selected AMDs in the Device Status section on the Devices screen. You cannot record new traces if the monitoring device experiences connection problems. Issue the ndstat command on your AMD to check whether the CBA and the CBA Agent are working. The log should contain the following lines: === CBA watchdog process: 2018? S 0:00 /bin/sh /usr/adlex/cba/bin/cba.run === CBA module: 1 processes(threads) 20430? Sl 0:08 /usr/adlex/cba/bin/cba === CBA-Agent watchdog process: 2069? S 0:00 /bin/sh /usr/adlex/cba-agent/bin/cba-agent.run === CBA-Agent process: 2073? S 0:00 /bin/bash /usr/adlex/cba-agent/bin/cba-agent Using the ls -l /var/spool/adlex/cba command, check whether a trace file with a given name exists and, if it does, check its size. To determine whether an interface is configured and functioning, issue the ifconfig command two or more times and observe the number of packets. If there is traffic on the interface, this number should be non-zero and increasing from observation to observation. For example: [root@vantageamd ~]# ifconfig eth0 Link encap:ethernet HWaddr 00:0C:29:7B:32:70 UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:32692 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (15.9 MiB) TX bytes:0 (0.0 b) Base address:0x1070 Memory:ec ec [root@vantageamd ~]# ifconfig eth0 Link encap:ethernet HWaddr 00:0C:29:7B:32:70 UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:48991 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes: (19.2 MiB) TX bytes:0 (0.0 b) Base address:0x1070 Memory:ec ec You can also use the rcon tcdump command to check whether you can intercept any packets received through the traffic on the sniffing interfaces. Disable and then enable the Guided Configuration connection in the monitoring device settings. For more information, see Adding an AMD to Devices List [p. 19]. Restart the CBA Agent with the service cba-agent restart Linux command. You can also search for exceptions and error information in the available logs: cva\log\server.log in the RUM Console installation directory /var/log/adlex/cba-agent.log in the AMD installation directory /var/log/adlex/cba.log in the AMD installation directory This, however, requires advanced product knowledge. 188

189 Why does the Guided Configuration experience connection problems? To diagnose this problem: Check whether the default connection port (9094) is open on the firewall; this is required for the Guided Configuration to work. You can change the default port number if it is already used by another application or service. For more information, see Connection Settings for the CBA Agent and RUM Console Server [p. 208]. Using the command netstat -nat grep LISTEN grep -v to list the open external ports on the AMD. In the following screen output example, port 9094 is open: ~]# netstat -nat grep LISTEN grep -v tcp : :* LISTEN tcp : :* LISTEN tcp : :* LISTEN Why can't I find a certain URL, parameter, or cookie in the traffic? (I am sure it is there). Most likely the searched element did not make it to the top statistics that are displayed on the Application Traffic Categories screen. To find a specific element, consider using a filtered traffic trace. You may also increase the number of items in each processed wizard request: 1. Open the cva\config\amd\cba-config.xml file in the RUM Console installation directory. 2. In the file, search for the <numberofresults> element. The default setting is: <numberofresults>100</numberofresults> 3. Change the default number to a new value. 4. Restart the CBA with the service cba restart Linux command. NOTE Appendix A Diagnostics and Troubleshooting Increasing the number of items in each processed wizard request may negatively affect the overall system performance. Why can't I see the decrypted SSL traffic? First check whether there is any SSL (undecrypted) traffic detected. Select Devices and Connections Verify quality of monitored traffic, and select the Application Traffic Categories tab. If there are no results under SSL for a selected trace, it may indicate one of the following: There are no SSL data in the recorded traffic trace, which may be due to insufficient trace length. For the SSL data to appear in the Guided Configuration perspective, the trace must contain the session beginning together with the SSL key handshake. 189

190 Appendix A Diagnostics and Troubleshooting Your SSL port number is something other than 443, so change the configuration settings for Guided Configuration. For more information, see SSL Settings for the CBA Agent and RUM Console Server Connection [p. 209]. Your SSL key configuration is invalid. Why is integration with Dynatrace Synthetic Monitoring not working? First, verify whether the Dynatrace connection settings are correct. For more information, see Configuring the DPN Connection in RUM Console [p. 87]. Remember that the only Dynatrace tests that are imported to DC RUM are active backbone tests. If your test definitions are of a different type, they will not be downloaded to DC RUM. Also note that to integrate Dynatrace and DC RUM performance measurements, you must have traffic traces with data corresponding to Dynatrace test definitions. If, after importing Dynatrace test definitions to DC RUM, no matching URLs are found, it may mean that the trace is too short and does not contain the matching data. The RUM Console uses too much memory. How can I solve the problem? You can control the amount of used memory in several ways: Disable the automatic trace recording. For more information, see Capturing Traffic Traces [p. 50]. Disable the Guided Configuration connection on some of your AMDs. For more information, see Adding an AMD to Devices List [p. 19]. Reset the automatically recorded trace. Use this option carefully, because resetting the trace will cause all of the previously gathered statistics to be lost. For more information, see Capturing Traffic Traces [p. 50]. Restart the Dynatrace RUM Console service using the Windows services.msc utility. 1. Select Start Run. 2. Type the services.msc utility name in the Open box. 3. Click OK. 4. On the list of the running services, right-click the Dynatrace RUM Console service and select Restart from the context menu. Why only one out of, for example two, web monitoring enabled AMDs are collecting the monitoring data? This issue appears when Linux is not configured properly. Specifically, the hostname configuration. The hostname of the machine must be mapped to either the localhost or to the machine's public IP address. To map the hostname perform the following steps: 1. Edit /etc/hosts file and make sure it looks similarly to this: #/etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail localhost.localdomain localhost xxx.xxx.xxx.xxx servername.hummy.org servername someothernames 190

191 Appendix A Diagnostics and Troubleshooting 2. Edit /etc/sysconfig/network and change the value there: NETWORKING=yes HOSTNAME=servername NISDOMAIN=hummy.org GATEWAY= Restart the network: /etc/init.d/network restart 4. For these changes to take effect, either restart the machine or use the following command: echo servername >/proc/sys/kernel/hostname echo hummy.org >/proc/sys/kernel/domainname This command automatically loads the new hostname into memory. Troubleshooting SSL Monitoring Issues The AMD provides a wide range of diagnostic information and tools that can help you resolve issues with SSL monitoring. Before trying to find an answer to a specific question regarding SSL-related issues, you can use the built-in system diagnostics of Data Center Real User Monitoring. Inspect the AMD log files, especially rtm_perf_curr.log and check the system health reports. For more information, see Diagnostic Tools in the Data Center Real User Monitoring Administration Guide and Interpreting a System Problem in the Data Center Real User Monitoring Administration Guide. Why, even though the Agentless Monitoring Device has an SSL accelerator card, and the SSL card has been initialized, SSL is not being decrypted. The SSL card needs to operate in the Logged on mode. For security reasons, after each machine reboot, the card reverts back to the Initialized mode. To re-activate the card, log in to the card using the user login and password. How can I check whether SSL decryption is functioning properly? To see full status information about the current SSL operation, execute the SHOW SSLDECR STATUS rcon command. For more information, see SHOW SSLDECR STATUS in the Data Center Real User Monitoring SSL Monitoring Administration Guide. To see historical information about SSL decryption, open the /var/log/adlex/rtm_perf.log file. Output from the SHOW SSLDECR STATUS command is written there every monitoring interval (default: 5 minutes). When viewing CAS reports, note the number of SSL errors reported. In particular, if the error breakdown information shows a large number of Other SSL errors, this indicates that SSL decryption errors are a problem. 191

192 Appendix A Diagnostics and Troubleshooting What should I do if the SHOW SSLDECR STATUS command does not return engine status as OK or if the incorrect engine is used? To operate correctly, the engine and accelerator card should match. For example, when using a NITROX accelerator card, use the nitroxfips engine. If the engine status is not OK or an incorrect engine is listed as being in use, check the following: Installation: perhaps the wrong upgrade file has been installed. For more information, see Installing the AMD Software in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. Engine configuration. For more information, see Selecting and Configuring SSL Engine in the Data Center Real User Monitoring SSL Monitoring Administration Guide. Authentication: some cards require that you perform a login action before they can operate. Refer to the configuration instructions for the card. My SSL engine status is OK, but SSL decryption fails entirely, with no keys recognized. What is the likely cause? The AMD requires that the SSL card be in an authenticated mode. This allows the AMD to gain access to RSA private keys stored in the card. One common problem is that when an AMD is restarted, the user forgets to log in to the AMD and launch the SSL card configuration utility to authenticate user access (unlock access to RSA keys). The engine status will be given as OK, meaning that the card itself is functioning correctly and the correct system driver is loaded, but the number of keys recognized will be 0 because the AMD is not able to retrieve key information from the card. >$ SHOW SSLDECR STATUS SSL DECRYPTION STATUS: CONFIGURATION: Engine:openssl(thread) status:ok Keys: recognized=0 not recognized=18 SESSIONS:... To avoid this problem, remember to log in to the AMD and launch the SSL card configuration utility to authenticate user access (unlock access to RSA keys) after you restart the AMD. What should I do if the SHOW SSLDECR STATUS command reports that some keys were not recognized? This can happen if RSA private keys stored in.pem files or on the accelerator card do not match the keys used by the SSL servers being monitored. Private keys used by servers can change. Investigate the problem further by executing the SHOW SSLDECR KEYS command in rcon and check which keys have an error status. For example: >$ show ssldecr keys Configuration of SSL private keys: <key: s1.key, status: error (reading failed)> <key: strange.key, type: file, size: 1024, status: OK (matched)> Keys total: 2, ok: 1, failed: 1, matched: 1 If there are errors, check the following: 192

193 Appendix A Diagnostics and Troubleshooting Is the keylist file in the correct format? If not, correct the entries. For more information, see Management of RSA Private Keys on AMD in the Data Center Real User Monitoring SSL Monitoring Administration Guide. If.pem files are to be used, are there the correct.pem files in /usr/adlex/config/keys? If not, supply the missing files. If.pem files are to be used, are there any typos in the file names in the keylist file? Correct the file names or paths as needed. Are the.pem files encrypted? Open a key file and see whether the word ENCRYPTED appears near the top of the file. The keys stored on the disk may be in encrypted form. In this case, to make the keys available the administrator has to arrange for the keys to be decrypted before they can be read by the AMD process. This requires a password (one per key file) and is accomplished using the kpadmin utility and the KPA daemon. For more information, see Using KPA to Make Keys Available to the AMD Process in the Data Center Real User Monitoring SAP Application Monitoring User Guide. If keys from the accelerator card are used, are the key IDs and names given in the proper format in keylist? For more information, see Management of RSA Private Keys on AMD in the Data Center Real User Monitoring SSL Monitoring Administration Guide. If only keys from the accelerator are to be used, consider not using the keylist file at all by setting the ssl.import.all.keys.from.token configuration property to true. This ensures that all the keys on the card will be seen correctly regardless of any entries you might make in the keylist file. For more information, see Management of RSA Private Keys on AMD in the Data Center Real User Monitoring SSL Monitoring Administration Guide. What should I do if the SHOW SSLDECR STATUS command reports no sessions? If the number of sessions is reported as 0, check the following: Does your AMD installation have a license for SSL decryption? If not, you need to obtain one. For more information, see Licensing Data Center Real User Monitoring Components in the Data Center Real User Monitoring Administration Guide. Are there any SSL services defined? Remember that you need to define a service before you can monitor it. You can execute the SHOW SSLDECR SERVERS command in rcon to list all the servers for which SSL decryption is active. The analyzer for the software service must specify SSL with decryption. For more information, see SHOW SSLDECR SERVERS in the Data Center Real User Monitoring SSL Monitoring Administration Guide and Configuring User-Defined Software Services in the RUM Console Online Help. Is there any actual traffic for the servers for which SSL decryption is active? To find out, use the tcpdump command on the AMD. For example: tcpdump 1000 "/ssl.tcp" "host and port 443" or tcpdump 1000 "/ssl.tcp" "vlan and host and port 443" 193

194 Appendix A Diagnostics and Troubleshooting and check whether there is any traffic captured in the /ssl.tcp file. If SHOW SSLDECR STATUS reports decryption errors, what do they mean and what can I do to fix the problem? The following decryption errors can be reported: packet lost during payload data exchange Your network may be losing packets; check mirrored ports. corrupted payload data packet Some of the traffic is corrupted and may be incorrectly received by the AMD, potentially because of network problems. decryption failed during payload data exchange The symmetric decryption failed. no private key found You do not have a private key for this session or you have not listed it correctly in the keylist file. packet lost during handshake It may mean that your network is losing packets; check mirrored ports. corrupted handshake packet or incorrect handshake sequence Some of the traffic is corrupted and may be incorrectly received by the AMD. decryption broken during handshake The symmetric decryption failed. unsupported SSL version Traffic encrypted with SSL 2.0 has been encountered. These protocol versions are not supported by the AMD. unsupported SSL feature An unsupported SSL feature has been encountered. The area the feature relates to and the count of occurrences is in brackets: unsupported cipher, compression, server key exchange. re-used sessions with no matching master session seen before A so-called short handshake (a session with re-used ID) was observed, but the AMD has no record of the original session ( long handshake ) that established the security credentials. Note that some such errors are normal if you restart the AMD, which may cause some traffic not to be observed by the AMD. incomplete SSL handshake A TCP session was observed to terminate before a complete SSL handshake was seen. The server can refuse a connection and close the TCP session for various reasons. For example, this can occur if the client requested a particular version of SSL but the server requires a different version. terminated by alert A fatal SSL alert arrived. Technically, this is alert detection and not an error. 194

195 Appendix A Diagnostics and Troubleshooting session not seen from the beginning May be related to monitoring of sessions with missing start of session. Change your settings if required. For more information, see Monitoring of Persistent TCP Sessions in the RUM Console Online Help. I suspect that I do not have all the private keys necessary for decryption (for example, I observe sessions with no private key found ). How can I ensure that all the servers have their matching keys? Execute the SHOW SSLDECR SERVERS command in rcon to list the decryption information for each server. For example: >$ SHOW SSLDECR SERVERS Configuration for SSL servers: <server: :443, certs seen: 1, keys used: 1, status: key(s) found> <cert: [/C=PL/ST=woj pomorskie/l=tricit,//ou=lab/cn=sdfds/ address=sdklj@sdkjw.com], sent: 4, key: strange.key> Servers total: 1, keys required: 1, keys found: 1, keys missing: 0 For all servers, ensure that their key status is found. Also note the last summary line of the output, which states how many keys were required and how many keys were found or were missing. For more information, see SHOW SSLDECR SERVERS in the Data Center Real User Monitoring SSL Monitoring Administration Guide. There appear to be missing keys, but I know that I have provided all the necessary keys. How can I verify that the keys I have are correct. A monitored server may change its private key, making the key used by the AMD obsolete. To prove that a key is correct, perform a test encryption/decryption using that key: 1. Use the SSLDECR CERTS rcon command to extract the public keys from the traffic being seen by the AMD For more information, see SHOW SSLDECR CERTS in the Data Center Real User Monitoring SSL Monitoring Administration Guide. 2. Perform a test encryption of a short text string, such as today's date, using extracted certificates. Use OpenSSL to encrypt the string. For example: # date > txt # cat txt Wed Feb 3 16:13:01 CET 2010 # openssl rsautl -inkey /cert_ \:443_1.der -keyform der -certin -in txt -encrypt -out txt.enc where /cert_ \:443_1.der is the file saved by the SSLDECR CERTS command used earlier. 3. Decrypt the encrypted file using the private key you want to test. For example, using OpenSSL: openssl rsautl -inkey /usr/adlex/config/keys/www2.prod.ramq.gov_decr1.pem -decrypt -out txt.decr -in txt.enc If the key is correct, there should be no difference between the files txt and txt.decr. You can also use the key stored on the card to decrypt the test file. To do that, use the rsautil utility residing in /usr/adlex/rtm/bin/. (For full usage syntax of the utility, type rsautil -?) 195

196 Appendix A Diagnostics and Troubleshooting In the following example, the first decryption succeeds and the second one fails. Note the last line with decrypt simple failed: [root@hsekilx030 bin]# cd /usr/adlex/rtm/bin/ [root@hsekilx030 bin]#./rsautil -e nitroxfips -t token -k 7 -f /root/dt_00000_42494/cert_ \:443_1.enc L :33: @ssldecr/rsaeng.cpp:320 RSA engine mode auto set to native L :33: @ssldecr/rsaeng.cpp:80 Openssl version: OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008, L :33: @ssldecr/rsaeng.cpp:84 Initializing OpenSSL in thread safe mode with 41 locks L :33: @./ssldecr/sslnitroxfips.h:29 NitroxFips: blocking mode: 0 L :33: @ssldecr/rsautil.cpp:322 OK L :33: @ssldecr/rsautil.cpp:347 SSL RSA handler nitroxfips(native) created L :33: @ssldecr/rsautil.cpp:394 key ok: 7 L :33: @ssldecr/rsautil.cpp:67 30 (0x1e) bytes at 0xbfa d 6f 6e 20 4d a Mon May 31 13: a a :09 CEST [root@hsekilx030 bin]#./rsautil -e nitroxfips -t token -k 8 -f /root/dt_00000_42494/cert_ \:443_1.enc L :33: @ssldecr/rsaeng.cpp:320 RSA engine mode auto set to native L :33: @ssldecr/rsaeng.cpp:80 Openssl version: OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008, L :33: @ssldecr/rsaeng.cpp:84 Initializing OpenSSL in thread safe mode with 41 locks L :33: @./ssldecr/sslnitroxfips.h:29 NitroxFips: blocking mode: 0 L :33: @ssldecr/rsautil.cpp:322 OK L :33: @ssldecr/rsautil.cpp:347 SSL RSA handler nitroxfips(native) created L :33: @ssldecr/rsautil.cpp:394 key ok: 8 L :33: @ssldecr/rsautil.cpp:147 decrypt simple failed For more information on loaded keys, execute the SHOW SSLDECR KEYS command in rcon. Report-Related Issues Central Analysis Server automatically detects a range of exceptions (anomalies) and notifies the report users. Exception notifications are displayed as yellow (warning) or red (error) triangle icons in the upper-left corner of the report window. To see the notification message, position the cursor over the triangle icon. The Slow Operation Load Sequence report is empty for an operation which is part of an XML transaction. Why and how do I fix this? For XML and SOAP, Operation Elements data is identical to Operation Analysis data, so, to avoid unnecessarily keeping the duplicates in the database, a VDATA_FILTER_XMLSOAP filter is set to true by default. Keeping this filter set to true saves disk space but, because the XML and SOAP entries are filtered out, it makes reporting on the Operation Elements level (elements or headers) impossible. To change the value of VDATA_FILTER_XMLSOAP property in userpropertiesadmin, type in the Web browser's Address bar and press [Enter], change the filter's property value, and click Set value to accept the 196

197 Appendix A Diagnostics and Troubleshooting change. To access this screen, you need to have administrative privileges for the report server. The yellow triangle displays AMDs produce no performance data. What do I do? The message AMDs produce no performance data means that AMDs connected to the report server do not produce any new data. To resolve this issue, you have to investigate the configuration of the AMDs and determine why they do not produce the performance data. The yellow triangle displays An AMD produces data stamped with a time from the future. What do I do? The report server has a built-in protection from simple configuration mistakes. One of the related problems is when data is incorrectly time stamped by AMD. This happens when the AMD is running with the system clock incorrectly set and is not being synchronized with the report server. If you see this notification, check the system time on the report server and on the AMD. Ensure the time synchronization option is turned on. To check the time synchronization: 1. Launch the RUM Console. 2. Select the AMD, right-click it and choose Open Configuration. The AMD Configuration window appears. 3. Select Global General. Check the IP address of the server authorized to set the AMD time. Make sure it is the same as the report server IP address. 4. Check the report server time setting. Do this by reading the time that is displayed at the bottom of the reports. Ensure the report server has the time zone set correctly. Figure 19. Example of the Report Time Stamp The yellow triangle displays A daily maintenance task is in progress. Data processing suspended. What do I do? Once a day the report server has to perform a database maintenance and memory cleanup. During that time, the data processing has to be suspended and you will see delayed data 197

198 Appendix A Diagnostics and Troubleshooting on reports. The daily maintenance is usually performed as the first task after midnight and it takes up to half an hour in installations with a large database. It is normal and expected to see this warning just after midnight. But if you see the message during the day, it can be a symptom of incorrect system configuration (check the time settings on the server) or of system overload. The yellow triangle displays No contact with the primary AMD. What do I do? This message indicates that the report server has lost contact with at least one primary AMD. If an AMD is marked as primary and the report server cannot communicate with this AMD, even if the performance data can be downloaded from the other AMDs, the system will wait until the communication with the primary AMD is restored. The yellow triangle displays No contact with any of the AMDs. What do I do? This message indicates that the communication link cannot be established with any of the attached AMDs. Check the network settings on the report server or the configuration of AMDs. The yellow triangle displays Delay in data processing. What do I do? If the last processed data is significantly behind the current time due to slow data processing or idle periods that occurred in the past, the report server displays the triangle icon with the message Delay in data processing. If the server had a delay, but now it is catching up, this message will not appear anymore. To confirm that delay is decreasing, inspect server.log and search for messages similar to this: T REC :10: zdata_43f47e58_5_t is being processed. Sample begin ts = :25. Sample delay 17 min. If the delay becomes smaller, the server is catching up. If the delay values are growing, it can indicate a system overload. The yellow triangle displays The AMD has not yet generated performance data. What do I do? This message indicates that some data files have already been generated on some AMDs, but not on the others. This may not be an indication of a problem and, when you refresh the reports after 30 to 60 seconds, this message may disappear. If necessary, verify the time synchronization among all the AMDs. See The yellow triangle displays Delay in data processing. What do I do? [p. 198]. The yellow triangle displays Data processing is being performed in the debug mode. What do I do? Data processing can be manually suspended and controlled by so-called debug mode, which can be enabled using Control Panel. Open Control Panel by typing: in the Address field of the web browser and clicking Go, then select Controlled data processing from the Configuration Management section. The red exclamation mark displays Data loading is in progress. Reports may be incomplete. What do I do? This message indicates that the report server is currently starting up. Because of this the information presented on reports may be incomplete. Depending on the database size, the startup process may take up to several minutes. If the server restart was not done manually or was not planned, inspect server.log or contact Customer Support. 198

199 Appendix A Diagnostics and Troubleshooting The red exclamation mark displays Low memory. The real-time cache will only be updated. What do I do? This message indicates that the report server has no free memory to process new entities such as software services, servers, and URLs. This message will be cleared when some resources are freed, this usually happens at midnight during the scheduled database maintenance (see The yellow triangle displays A daily maintenance task is in progress. Data processing suspended. What do I do? [p. 197]). All the metric values presented on reports (except user/client counters) will show correct values. However, the predefined tabular reports may not show all the entities they are intended to show. All the charts and DMI reports show correct data. The mechanism of updating the real-time cache, as described above, is a protection that allows the report server to continue the operation instead of closing down due to lack of memory resources. The red exclamation mark displays The number of servers has reached the defined limit. What do I do? The report server has a built-in limit of the number of monitored servers. If the number of observed servers reaches a defined limit, the report server will not accept any new servers and will drop the collected data for those servers. The predefined value of the limit can be customized. However, the report server can automatically adjust the limit in low-resources situations. The red exclamation mark displays The number of clients has reached the defined limit. What do I do? The report server has a built-in limit of the number of monitored clients. If the number of registered clients (which also includes aggregated virtual clients such as Client from... ) reaches a defined limit, the report server will not accept any new clients and will drop the collected data for those clients. The predefined value of the limit can be customized. However, the report server can automatically adjust the limit in low-resources situations. The red exclamation mark displays The number of sites has reached the defined limit. What do I do? The report server has a built-in limit of the number of automatically created sites. If the number of observed automatic sites reaches a defined limit, the report server will not create any new automatic sites and such traffic will be allocated to All Other. The predefined value of the limit can be customized. However, the report server can automatically adjust the limit in low-resources situations. The Sites report for a selected application is empty. Why? If the Sites report for a selected application is filtered for a client tier, such as Synthetic or RUM sequence transactions, it will not show any data. To see statistics for sites, drill down from the Applications report as follows: 1. Click the application name on the Applications report. 2. Click the client tier name on the Tiers report for a selected application. For the Synthetic tier, you will see the Overview Application Status report; for the RUM sequence transactions tier, the Sequence Transactions Log report. 3. Depending on the type of report, click the Overview Site Status or the Sites tab. 199

200 Appendix A Diagnostics and Troubleshooting I see gaps on the chart reports. Why are the charts incomplete? Gaps in reports mean that the report server missed some data and was not able to get it into the database on time. Your reports may resemble the example below. Figure 20. Gaps in a Graphical Report There are several reasons why the graphical reports may have incomplete data: The AMD was not able to detect any traffic from the monitored network, so it was not able to produce any valid data for the report server. To confirm that this was the reason, connect to the AMD using an SSH client and check whether the files named zdata_xxxxx_x_x are located in the /var/spool/adlex/rtm directory. Similar symptoms can be observed if the AMD has been down for some time and data files were not produced for that time. If data files are present and the viewed chart displays only a fragment of the monitored traffic, for example, for a specific server or site, it may indicate that a part of traffic, which was indented to be monitored, is missing. In this situation, the data files are much smaller than usual for the corresponding period of the day. Similar situations, that is, gaps only on some reports, may occur in a multi-amd installation when some AMD s were down or disconnected from the network. In the case when only one AMD is connected to the report server, communication problems do not cause data gaps. If the report server cannot communicate with the AMD, it will wait until the communication is restored and then will process all the data from the past. When there are multiple AMDs connected to the report server and there is a break in communication with only some of them, the report server processes the data from the available AMDs, so in this case, gaps can appear on some reports. If it is a critical issue and your network (or its parts) require continuous monitoring and you cannot miss the data from some AMDs, you have to mark the AMDs as primary. In this case, the report server will wait until the communication with primary AMDs is restored, even if other AMDs are available. Gaps in charts on some reports in multi-amd installations may be caused by unsynchronized AMDs. The reason for that may be that if the report server sees a data file for a specific time period on one of the AMDs, it will wait only 30 seconds for data files covering the same period of time from other AMDs. The 30 seconds are the server's tolerance for time synchronization issues. To verify that this situation occurred, compare the clock readings from AMDs and then check the time synchronization settings (see The yellow triangle displays An AMD produces data stamped with a time from the future. What do I do? [p. 197]). 200

201 Appendix A Diagnostics and Troubleshooting It may happen that a part of data will be missing. This will result in a significant decrease of the aggregated data, used to render the chart bars. Note that this effect relates to metrics that are calculated as sums, for example, number of operations, number of errors, number of users, or bandwidth utilization. Charts showing the averages (RTT, loss rate, operation time) will not be affected. I see gaps on the log-term data chart reports. Why are the charts incomplete? The report server aggregates the data collected during the day into daily (and monthly) rollups. This is a scheduled process. If this process is not triggered, you will see gaps in the daily rollups. The most frequent reasons for missing rollups are: The report server was down in the night; report data generation starts at 12:10 AM local time and if the report server was down at that time, no aggregate data for long-term reports will be generated. The report server was overloaded and it took too much time for other crucial tasks; report data generation for long-term reports was canceled. You can always re-generate data for long-term reports. Open Control Panel by typing: in the Address field of the web browser and click Go, then select Regenerate Reports from the System Management section. I created a report that consists of several charts but it loads very slowly. How can I improve its performance? If you are using exactly the same set of dimensions and filters for every chart but would like to show different metrics on separate charts, there are two ways of improving such a report. In this example, it is assumed that you want a report that shows Client bytes, Server bytes, and Total bytes on separate charts for the HTTP analyzer. First, the simplest and recommended method, is to define one section that contains all these three metrics. Figure 21. Creating One Section with Three Metrics 201

202 Appendix A Diagnostics and Troubleshooting Open the Chart settings panel and from the single chart per list select Metric. If you are using metrics with different units, you can select the Metric unit option instead. For more information, see Displaying Multiple Charts in the Data Center Real User Monitoring Data Mining Interface (DMI) User Guide. The second method requires changes on the Subject Data and Result Display tabs. 1. For each report section (chart), create the same set of metrics. To do this, for each chart add metrics that are displayed on the other charts. Note that the order of metrics must be the same in every section. For example, each section must contain the Client bytes, Server bytes, and Total bytes metrics listed exactly in the same order. 2. Disable showing unnecessary metrics for each chart. Go to the Result Display tab and disable showing the redundant metrics. For example, for chart that is going to show only the Client bytes metric, disable showing the Server bytes and Total bytes metrics. Figure 22. Selecting Metrics to Display on a Chart Application performance and availability data is missing from the tabular reports. How can I fix this? The missing data manifests itself as zero or a hyphen. The most frequent reason for this situation is the incorrect setting of business hours and holidays. Inspect the business hours and holiday settings by choosing Settings Report Settings Business Hours. The following configuration screen shows the current settings. 202

203 Appendix A Diagnostics and Troubleshooting Figure 23. Business Hours Configuration Screen To collect performance data seven days per week, including non-business days and holidays, clear the Holidays check box and select the check boxes for weekend days. In addition, you can collect performance data in 24/7 mode, but be aware that this results in a higher database growth rate and a larger database. To enable collecting data all the time, open the Control Panel by opening the following page: In the Control Panel, click Advanced Properties Editor from the Configuration Management section. Set ONLY_BUSS_HOUR_REPORTING to OFF. To see whether your holiday definition is correct, click View Holidays. 203

204 Appendix A Diagnostics and Troubleshooting Figure 24. Defined Holidays Screen The list of holidays is hard-coded and the default set is for the USA. To select a set, click the Choose holiday definition list. To see the content of the selected set, click Preview. To store the newly selected set, click Save. Why are SQL queries in reports truncated even though full query logging is set on the AMD? By default, only the first 1024 bytes of a query are logged. This is sufficient in most cases to log full queries. However, if you deploy queries that are longer than 1024 characters, change the sql.query.length parameter in the rtm.config file. To edit this file: 204