Data Center Real User Monitoring

Transcription

1 Data Center Real User Monitoring Network Performance Monitoring User Guide Release 12.3

2 Please direct questions about Data Center Real User Monitoring or comments on this document to: Customer Support Copyright 2015 Compuware Corporation. All rights reserved. Unpublished rights reserved under the Copyright Laws of the United States. U.S. GOVERNMENT RIGHTS-Use, duplication, or disclosure by the U.S. Government is subject to restrictions as set forth in Compuware Corporation license agreement and as provided in DFARS (a) and (a) (1995), DFARS (c)(1)(ii) (OCT 1988), FAR (a) (1995), FAR , or FAR (ALT III), as applicable. Compuware Corporation. This product contains confidential information and trade secrets of Compuware Corporation. Disclosure is prohibited without the prior express written permission of Compuware Corporation. Use of this product is subject to the terms and conditions of the user's License Agreement with Compuware Corporation. Documentation may only be reproduced by Licensee for internal use. The content of this document may not be altered, modified or changed without the express written consent of Compuware Corporation. Compuware Corporation may change the content specified herein at any time, with or without notice. All current Compuware Corporation product documentation can be found at Compuware, FrontLine, Network Monitoring, Enterprise Synthetic, Server Monitoring, Dynatrace Network Analyzer, Dynatrace, VantageView, Dynatrace, Real-User Monitoring First Mile, and Dynatrace Performance Network are trademarks or registered trademarks of Compuware Corporation. Cisco is a trademark or registered trademark of Cisco Systems, Inc. Internet Explorer, Outlook, SQL Server, Windows, Windows Server, and Windows Vista are trademarks or registered trademarks of Microsoft Corporation. Firefox is a trademark or registered trademark of Mozilla Foundation. Red Hat and Red Hat Enterprise Linux are trademarks or registered trademarks of Red Hat, Inc. J2EE, Java, and JRE are trademarks or registered trademarks of Oracle Corporation. VMware is a trademark or registered trademark of VMware, Inc. SAP and SAP R/3 are trademarks or registered trademarks of SAP AG. Adobe Reader is a registered trademark of Adobe Systems Incorporated in the United States and/or other countries. All other company and product names are trademarks or registered trademarks of their respective owners. Local Build: April 1, 2015, 12:56

3 Contents Contents Introduction Who Should Read This Guide Organization of the Guide Related Publications Customer Support Information Reporting a Problem Documentation Conventions Chapter 1 Network Performance Monitoring Overview Deciding on NetFlow vs. Traffic Monitoring Analysis Using AMD Flow Collectors for NetFlow Analysis Flow Collector Placement and Usage Scenarios Placement of a Flow Collector Using AMD Flow Collectors to Supplement Passive Traffic Monitoring Using AMD Flow Collectors for Simple Network Performance Monitoring Using AMD Flow Collectors to Troubleshoot Network Problems Ad Hoc Configuration Overview for AMD Network Performance Monitoring Chapter 2 Adding Basic DC RUM Devices Adding an AMD to the Devices List Adding a CAS to Devices List Chapter 3 Verification of Traffic Monitoring Quality Sniffing Point Diagnostics Sniffing Point Diagnostics Reports Network Interface General Statistics Network and Transport Protocol Information Services Detected in the Traffic Session-Related Statistics SSL Diagnostics Application Overview Using RUM Console to Identify Problems Related to Network Hardware Operation

4 Contents Chapter 4 Propagating the Data Collector Configuration Importing the AMD Configuration Exporting the AMD Configuration Propagating the AMD Configuration Using RUM Console Propagating the AMD Configuration Automatically through RUM Console Server... Chapter 5 Basic Monitoring Configuration Configuring General Data Collector Settings General Configuration Options for HTTP Express Software Services Chapter 6 Configuring NetFlow Monitoring Configuring Remote NetFlow-enabled Devices Optimizing NetFlow Data Collection and Processing for NetFlow Version Configuring Fields to be Exported in NetFlow Version Configuring Flow Collector General Settings Observing and Fine-Tuning NetFlow Sources Chapter 7 Configuring AMD to Monitor User-Defined Software Services Defining Oracle Forms Software Services Configuring Rules for User-Defined Software Services Excluding IP Ranges from AMD Client Analysis HTTP Express Analyzer Configuring User-Defined Software Services Based on HTTP Express Analyzer... Configuring URL Monitoring for HTTP Express Analyzer Configuring Monitoring of URL Parameters for HTTP Express Analyzer Managing User-Defined Software Services Chapter 8 Network Tiers Modifying a Network Tier Adding a Rule Based on Sites Chapter 9 Configuring Sites, Areas, and Regions Adding Sites Manually Defining Sites, Areas, and Regions Using External Text Files Importing Site Definitions Automatic Creation of Sites from Enterprise Synthetic Formatting the Site, Area, and Region Definitions Importing and Exporting Site Definitions in RUM Console Exporting Site Definitions in Diagnostic Console Chapter 10 Network Analysis Reports Example Network Analysis Reports Usage Network Overview - Sites Report Network Status - Software Services View Report Network Status - Servers View Report Top Traffic Summary - Today Report Total Traffic by Hour - Today Report Top Software Services by Traffic - Today Report

5 Contents Total Traffic by Client Site - Today Report Traffic Analysis - Last 30 Days Report Traffic Fault Domain Isolation - Last Hour Report Top Clients by Traffic - Last Hour Report Traffic Analysis Details - Last 30 Days Report Software Services by RTT Report Software Services View - Charts - Network Report Software Services View - Charts - Performance Report Software Services View - Charts - Availability Report Software Services View - Charts - Clients Report Network View - Hosts and Users Reports Network Overview - Links Report Appendix A Diagnostics and Troubleshooting Report-Related Issues Appendix B Regular Expression Fundamentals Testing Regular Expressions Best Practices for Regular Expressions Glossary Index

6 Contents 6

7 INTRODUCTION Who Should Read This Guide This manual is intended for users of Data Center Real User Monitoring who want to monitor network activity in a single data center or in a distributed network with multiple sites. Organization of the Guide This guide is organized as follows: Network Performance Monitoring Overview [p. 11] Describes capabilities of network performance monitoring with Data Center Real User Monitoring and provides an overview of the configuration process. Adding Basic DC RUM Devices [p. 23] Describes how to add and configure the data sources and report servers using the RUM Console. Verification of Traffic Monitoring Quality [p. 27] Describes how to verify sniffing points traffic detection quality before the actual monitoring begins. Propagating the Data Collector Configuration [p. 39] Describes how to import, export, and propagate AMD configuration. Basic Monitoring Configuration [p. 45] Describes AMD general settings. Configuring NetFlow Monitoring [p. 53] Describes the process of configuring and fine-tuning network performance monitoring based on NetFlow. Configuring AMD to Monitor User-Defined Software Services [p. 63] Describes the creation and management of user-defined software services relevant to network performance monitoring. Network Tiers [p. 83] Describes tiers that present traffic for client sites. Configuring Sites, Areas, and Regions [p. 85] Introduces the concept of sites, areas, and regions. 7

8 Introduction Network Analysis Reports [p. 95] Describes the network reports, which provide a network view of the traffic, showing a picture of the monitored network operation and highlighting potential problems in the network, including excessive RTT or loss rate. Diagnostics and Troubleshooting [p. 111] Lists common CAS support issues in the form of questions and answers. Regular Expression Fundamentals [p. 121] Describes how to use regular expressions in CAS. Related Publications Documentation for your product is distributed on the product media. For Data Center RUM, it is located in the \Documentation directory. It can also be accessed from the Media Browser. Go online ( for fast access to information about your Dynatrace products. You can download documentation and FAQs as well as browse, ask questions and get answers on user forums (requires subscription). The first time you access FrontLine, you are required to register and obtain a password. Registration is free. PDF files can be viewed with Adobe Reader version 7 or later. If you do not have the Reader application installed, you can download the setup file from the Adobe Web site at Customer Support Information Dynatrace Community For product information, go to and click Support. You can review frequently asked questions, access the training resources in the APM University, and post a question or comment to the product forums. You must register and log in to access the Community. Corporate Website To access the corporate website, go to The Dynatrace site provides a variety of product and support information. Reporting a Problem Use these guidelines when contacting APM Customer Support. When submitting a problem, log on to the Dynatrace Support Portal at click the Open Ticket button and select Data Center Real User Monitoring from the Product list. Refer to the DC RUM FAQ article at to learn know how to provide accurate diagnostics data for your DC RUM components. Most of the required data can be retrieved using RUM Console. 8

9 Introduction Documentation Conventions The following font conventions are used throughout documentation: This font Bold Citation Documentation Conventions [p. 9] Fixed width Fixed width bold Fixed width italic Menu Item Screen Code block Indicates Terms, commands, and references to names of screen controls and user interface elements. Emphasized text, inline citations, titles of external books or articles. Links to Internet resources and linked references to titles in documentation. Cited contents of text files, inline examples of code, command line inputs or system outputs. Also file and path names. User input in console commands. Place holders for values of strings, for example as in the command: cd directory_name Menu items. Text screen shots. Blocks of code or fragments of text files. 9

10 Introduction 10

11 CHAPTER 1 Network Performance Monitoring Overview Data Center Real User Monitoring (DC RUM) enables network performance monitoring based on the analysis of NetFlow data and on passive monitoring of network traffic using network probes. Choice of data sources and reporting engines The analysis choices available to you depend on the product licenses you purchase. Data source options AMDs are used as primary network probes, combining the functionality of passive traffic listeners and NetFlow collectors. Reporting engine options Central Analysis Server (CAS) is the recommended reporting engine for data supplied by AMDs and Network Monitoring Probes, and is the recommended reporting engine for all new deployments. If you are an existing Network Monitoring user, you can continue to use the Interactive Viewer as your reporting engine for data supplied by Network Monitoring Probes. NetFlow options and features NetFlow version 5 NetFlow version 9 and IPFIX with elements of Flexible NetFlow: Ability to configure the NetFlow collector to retrieve and interpret client RTT information from a specified field. This feature has been tested for Riverbed and Cisco network devices. WAN optimization monitoring: ability to specify WAN interface names, to facilitate NetFlow monitoring of WAN-optimized traffic. Network performance monitoring options Data Center Real User Monitoring network performance monitoring covers layers 3 and up of the OSI model: it monitors the IP protocol and the layers above IP (transport, session and application); layer 2 and below are not analyzed. 11

12 Chapter 1 Network Performance Monitoring Overview Data source configuration For NetFlow-based analysis to function, you need to configure AMDs to perform NetFlow collection and SNMP polling. You should also configure specific NetFlow-enabled devices to generate and send FlowSets to the specific AMDs. For passive traffic monitoring analysis, you need to configure AMD so that the network traffic of interest is monitored by the appropriate traffic analyzers. To do this, you can configure custom software services in RUM Console. An alternative option is to use application recognition engines built into AMD and Network Monitoring Probe. To get maximum performance of traffic analysis in typical network performance monitoring use cases, you should use general traffic analyzers as opposed to analyzers for in-depth transactional or payload analysis. Reporting for network performance monitoring An extensive set of network performance monitoring reports has been provided. You can also design your own reports or customize some of the reports that have been provided. For more information, see Network Analysis Reports [p. 95]. Deciding on NetFlow vs. Traffic Monitoring Analysis The advantage of NetFlow-based analysis is that NetFlow data can be easily collected from remote locations. However, passive traffic monitoring can provide in-depth information not available through NetFlows. Use the following decision table to decide which type of analysis is suitable for your network architecture and specific monitoring needs. You may decide to use both types of analysis at the same time: traffic monitoring analysis for your local data center and NetFlow analysis for remote locations. Table 1. Advantages and uses of NetFlow and traffic monitoring analysis NetFlow analysis Suitable for monitoring remote sites. Does not provide time-based statistics. Traffic monitoring analysis Most suitable for monitoring the local data center. Requires a remote probe for monitoring remote sites (which may result in a heavy data footprint over the connection link). Provides time-based statistics, such as analysis of response time, server time, or network time; Provides only general network statistics. Provides detailed breakdown of network protocols and sessions seen on the network links by NetFlow-enabled device. Provides in-depth analysis for a number of network protocols, including HTTP, XML, SQL, etc. Using AMD Flow Collectors for NetFlow Analysis In the Data Center Real User Monitoring deployment architecture, a flow collector is one of the services running on an AMD, while other services may at the same time be used for passive traffic monitoring of a data center. 12

13 Chapter 1 Network Performance Monitoring Overview Information available through NetFlow analysis In a NetFlow-enabled device, each IP packet that passes through the router is examined for a set of IP packet attributes: IP source and destination addresses Source and destination ports Layer 3 protocol type Class of Service Router interface These attributes comprise an IP packet s identity, which is used by the router to determine whether the packet is unique or is similar to other packets. All packets with the same source/destination IP address, source/destination ports, protocol interface, and class of service are grouped into a flow. The flows are stored as flow records in the router s NetFlow cache database. At the expiration of device timers, all flow records in the NetFlow cache are exported as NetFlow packets to the destinations listed in the router's export configuration settings. AMDs as Flow Collectors An AMD can be named as a NetFlow export destination. An AMD flow collector process (service) operates in the same way as a traffic monitoring service, in that it analyzes received data and stores the statistics from a given period (monitoring interval) in a database record. The data record is then forwarded to a requesting reporting server. Before storing the statistics, AMD flow collectors analyze the raw FlowSet information by applying built-in and user-defined software service definitions. The definitions allow for the identification of observed software services, based on IP address and TCP/UDP port and socket number. Flow Collector Placement and Usage Scenarios A flow collector's traffic analysis represents a subset of the information available from a probe; however, flow collectors are a good choice for supplementing the probes when monitoring border routers of a large distributed network. A flow collector can also be used within a single data center to monitor internal switches. It should be noted however that indiscriminate monitoring of all interfaces on hot/core devices can generate very large numbers of flows and create significant CPU load on these devices. In this case CPU load monitoring is particularly important and port pairing may need to be performed to reduce the load and avoid duplication. In particular, the following scenarios can be distinguished: Using AMD Flow collectors to supplement passive traffic monitoring. For more information, see Using AMD Flow Collectors to Supplement Passive Traffic Monitoring [p. 14]. Using AMD Flow collectors to troubleshoot network problems ad hoc. For more information, see Using AMD Flow Collectors to Troubleshoot Network Problems Ad Hoc [p. 18]. 13

14 Chapter 1 Network Performance Monitoring Overview Using AMD Flow collectors for simple network performance monitoring. For more information, see Using AMD Flow Collectors for Simple Network Performance Monitoring [p. 16]. Placement of a Flow Collector Before installing flow collectors, plan deployment based on location relative to the routers and network traffic monitoring requirements. Location relative to the routers is the primary consideration in placement of a flow collector. The flow collector s monitoring activities result in a continuous increase in network overhead traffic, which includes the flow packets coming from the routers and SNMP traffic generated when the flow collector polls the routers. It therefore may be necessary to place the flow collector at a location that avoids sending the additional traffic over critical areas of the network such as high-utilization WAN links. For similar reasons, you should also consider the optimal location of the flow collector relative to the report server, because data will be sent to the server every monitoring interval. Using AMD Flow Collectors to Supplement Passive Traffic Monitoring You can use flow collection if continuous visibility across the entire network is needed but it is not practical (because of the network size or budget constraints) to monitor all of the traffic using probes. Possible situations in which to use flow collectors: In a distributed network, to monitor traffic between remote sites In the case of a distributed network, using flow analysis is especially useful if remote sites communicate with each other without the traffic coming to a central enterprise router. Traditional hub-and-spoke networks are being replaced with meshed networks. In the hub-and-spoke model, placing a probe at the core of the network provided complete visibility to all the traffic going to all sites. With meshed networks, there is no visibility on traffic going from site to site, unless each remote site is monitored with a probe. However, placing a probe at each site may not be practical. In such situations, to extend monitoring to the entire network, place an AMD probe in front of key components such as data center servers and Internet gateways and then enable NetFlow on the routers at these locations (to enable capture of application traffic between remote locations) and use an AMD flow collector (probably the same physical device as the probe) to receive and decode the NetFlow records. In a single data center, if not all traffic needs to be seen by probes It may be that in a large data center there is no need to collect detailed traffic monitoring information from all of points of the network. Connecting probes requires following hardware maintenance procedures, which may require issuing maintenance and authorization requests, whereas arranging for flow collection over the existing network is relatively easy. 14

15 Chapter 1 Network Performance Monitoring Overview NOTE Indiscriminate monitoring of all interfaces on hot/core devices can generate very large numbers of flows and create significant CPU load on these devices. In this case CPU load monitoring is particularly important and port pairing may need to be performed to reduce the load and avoid duplication. Figure 1. Using AMD flow collectors to supplement passive traffic monitoring in a distributed network The following figure illustrates how a combination of AMD flow collectors and AMD passive traffic monitoring probes can be used to gain complete coverage on a large distributed network. Branch A Internal Router NetFlow Passive Monitoring AMD Passive Monitoring Border Router NetFlow Internal Router Data Center at HQ Branch B Figure 2. Using AMD flow collectors to supplement passive traffic monitoring within a single data center The following figure illustrates how an AMD can be used as both a passive traffic monitoring probe and as a flow collector to gain complete coverage of a large data center. Note that, while it is likely that the same physical unit will be used as both a probe and a flow collector, it is also possible to use two separate AMDs. 15

16 Chapter 1 Network Performance Monitoring Overview As noted above, if such a scenario is used and core devices are equipped with more than two interfaces, particular care must be taken to avoid generating huge numbers of flows (thus causing duplication and heavy CPU load on the devices). Border Router Internal Router NetFlow Internal Router NetFlow NetFlow Passive Monitoring AMD Data Center Using AMD Flow Collectors for Simple Network Performance Monitoring If you do not require in-depth traffic analysis, but only simple network performance monitoring, you can use AMDs as flow collectors without using any AMDs as probes. Possible situations in which this solution can be used: In a distributed network To enable capture of application traffic between remote locations, enable NetFlow on the routers at these locations and use an AMD flow collector to receive and decode the NetFlow records. In a single data center Collect flows from all the NetFlow-enabled devices in a single data center. 16

17 Chapter 1 Network Performance Monitoring Overview NOTE Indiscriminate monitoring of all interfaces on hot/core devices can generate very large numbers of flows and create significant CPU load on these devices. In such cases, CPU load monitoring is particularly important and port pairing may be needed to reduce the load and avoid duplication. Figure 3. Using AMD flow collectors for simple network performance monitoring in a distributed network The following figure illustrates how an AMD flow collector can be used to gain complete coverage on a large distributed network. Branch A Internal Router NetFlow AMD NetFlow Border Router NetFlow Internal Router Data Center at HQ Branch B Figure 4. Using AMD flow collectors for simple network performance monitoring within a single data center The following figure illustrates how an AMD can be used as a flow collector to monitor a large data center. 17

18 Chapter 1 Network Performance Monitoring Overview As noted above, if such a scenario is used and core devices are equipped with more than two interfaces, particular care must be taken to avoid generating huge numbers of flows (thus causing duplication and heavy CPU load on the devices). Border Router Internal Router NetFlow Internal Router NetFlow NetFlow AMD Data Center Using AMD Flow Collectors to Troubleshoot Network Problems Ad Hoc The ease of deploying AMD flow collectors and activating NetFlow make flow collectors a useful tool for troubleshooting networks and applications. Fast results are needed in a short-term troubleshooting project. It is desirable to avoid potential network downtime and time delays required to install probes to support the troubleshooting. Such consideration are particularly applicable for large distributed networks, if remote sites communicate with each other without the traffic coming to a central enterprise router where a probe could be placed. To troubleshoot a distributed network, place an AMD probe in front of key components such as data center servers and Internet gateways. Then, to enable troubleshooting for remote locations, enable NetFlow on the routers at problem locations and use an AMD flow collector (probably the same physical device as the probe) to receive and decode the NetFlow records. Subsequent deployment of probes may be needed for deeper application understanding; the troubleshooting analysis accomplished with the flow collector likely will help in that decision. 18

19 Chapter 1 Network Performance Monitoring Overview Figure 5. Troubleshooting a distributed network with an AMD flow collector The following figure illustrates the placement of an AMD flow collector for troubleshooting purposes. The AMD flow collector is deployed as the network troubleshooting tool and NetFlow is enabled only on the routers in the problem locations. NetFlow data is immediately available for analysis and troubleshooting can be performed using available report engines. Branch A Passive Monitoring Internal Router NetFlow Branch B Site with a network problem AMD Passive Monitoring Border Router Branch C Branch D Internal Router Data Center at HQ Branch E Configuration Overview for AMD Network Performance Monitoring Before You Begin Before you start the configuration process: You should be familiar with DC RUM components and basic monitoring concepts. For more information, see Data Center Real User Monitoring Overview in the Data Center Real User Monitoring Getting Started.. You need to install the following DC RUM components: AMD. For more information, please refer to the Dynatrace Agentless Monitoring Device Installation Guide. RUM Console. For more information, please refer to the RUM Console Installation Guide. CAS. For more information, please refer to the Central Analysis Server Installation Guide. Follow these steps to set up network performance monitoring using AMDs as primary network probes and CAS as the reporting engine. 19

20 Chapter 1 Network Performance Monitoring Overview Defining data sources and servers 1. Add Agentless Monitoring Device (AMD) AMD is the main data source (Data Collector) for DC RUM; it collects and presents the monitored data to DC RUM report servers for analysis and reporting. You need to add at least one AMD to the list of devices in RUM Console. For more information, see Adding an AMD to the Devices List [p. 23]. 2. Add Central Analysis Server (CAS) CAS is the main report server for DC RUM. It uses data provided by the AMD and its monitoring and alerting mechanisms to identify, track, and report on issues affecting the security, performance, and reliability of your services. Add at least one CAS to the device list and configure its connection with the AMD. Adding a report server to a list of devices is similar to adding the AMD. For more information, see Adding a CAS to Devices List [p. 25]. 3. Verify the traffic monitoring quality and completeness You can verify traffic quality and completeness before the actual monitoring begins. Sniffing point diagnostics allows you to perform pre-monitoring tasks without the need of accessing the AMD console and executing a series of Linux commands which usually serve the purpose of validating AMD physical installation and connection. For more information, see Verification of Traffic Monitoring Quality [p. 27]. Deciding on NetFlow vs datacenter analysis and configuring NetFlow-enabled devices 4. Deciding on NetFlow vs datacenter analysis Decisions taken at this point will affect the configuration steps performed later. Network performance monitoring can be based on NetFlow analysis as well as on passive analysis of network traffic on selected data center devices. You can use both of these types of analysis to satisfy your requirements. Options available to you depend on the product license you have purchased. For more information, see Deciding on NetFlow vs. Traffic Monitoring Analysis [p. 12]. 5. Optional: Configuring NetFlow-enabled devices to deliver NetFlows to the AMD. If NetFlow analysis is required, the specific routers and other NetFlow-enabled devices have to be configured to send NetFlows to the AMD. Refer to appropriate documentation of your NetFlow-enabled devices for instructions on how to perform the configuration. For more information, see Configuring Remote NetFlow-enabled Devices [p. 53]. Configuring AMD 6. Specifying basic AMD settings. Before you proceed to detailed monitoring rules, you need to define global settings that will be applied to all the monitoring and analysis performed on a given AMD. These global settings include: 20

21 Chapter 1 Network Performance Monitoring Overview The monitoring interval You are unlikely to need to modify this setting for the purpose of network analysis, but you can adjust it if you need a different granularity of results. Operation time threshold This setting will affect one of the protocol analyzers used for network performance monitoring: Generic (with transactions). This setting can be overridden by settings for specific software services you define later, based on this analyzer. For more information, see Configuring General Data Collector Settings [p. 45]. 7. Optional: Configuring general NetFlow settings on the AMD For more information, see Configuring Flow Collector General Settings [p. 58]. 8. Optional: Fine-tuning flow sources After you have configured flow-enabled devices to send flows to the AMD, the observed devices will start appearing in the AMD Flow Sources table. You may then need to adjust the settings for individual devices.for more information, see Observing and Fine-Tuning NetFlow Sources [p. 60]. 9. Optional: Viewing NetFlow reports on the report server and re-adjusting monitoring configuration settings After viewing network reports on the report server, you may need to modify NetFlow monitoring settings to better suit your purposes, remove or add more flow sources, or modify flow analysis settings. For more information, see Step 5 [p. 20], Step 7 [p. 21], and Step 8 [p. 21]. An extensive set of network performance monitoring reports has been provided. You can also design your own reports or customize some of the reports that have been provided. For more information, see Network Analysis Reports [p. 95]. 10. Optional: Deciding on traffic monitoring analyzers and configuring global monitoring setting for these analyzers. Select one or more traffic monitoring analyzers to be used for network performance monitoring, and define global settings for these analyzers if any are available. These are settings that affect monitoring of all services based on the given analyzer. For network performance monitoring, you are advised to select those analyzers that provide network-related information without in-depth transactional or payload analysis. The recommended analyzers and their global settings are: TCP This analyzer provides only basic measurements (no realized bandwidth and transaction monitoring). There are no global settings specific to this analyzer. Generic This protocol provides basic measurements and realized bandwidth. There are no global settings specific to this analyzer. Generic (with transactions) This protocol provides basic measurements, realized bandwidth and transaction metrics. There are no global settings specific to this analyzer except the transaction threshold as set for all analyzers supporting transactions. For more information, see Step 6 [p. 20]. 21

22 Chapter 1 Network Performance Monitoring Overview HTTP Express Use this analyzer if it is known that HTTP traffic is present and you require basic HTTP information about servers and URLs. This analyzer has a number of general options for all the services based on it. For more information, see General Configuration Options for HTTP Express Software Services [p. 49]. 11. Optional: Defining custom software services for passive traffic monitoring Define custom software services for the selected analyzers. For more information on software service configuration, see Configuring User-Defined Software Services in the RUM Console Online Help. There are additional specific settings for the HTTP Express analyzer that should also be configured for software services based on that analyzer. For more information, see Configuring User-Defined Software Services Based on HTTP Express Analyzer [p. 70]. Fine-tuning and troubleshooting 12. Viewing reports and fine-tuning configuration. Determining the best possible configuration for your needs may be an iterative process, where you will fine-tune the configuration incrementally after viewing your report results. For more information, see Network Analysis Reports [p. 95]. 13. Troubleshoot problems You can review the answers to the most common questions and troubleshoot your setup and report configurations. For more information, see Diagnostics and Troubleshooting [p. 111]. 22

23 CHAPTER 2 Adding Basic DC RUM Devices In a DC RUM configuration, there are two device types: data collectors and report servers. To start using the product, add and configure at least one AMD data collector and one CAS report server. You manage these devices using a configuration tool called the RUM Console. Adding an AMD to the Devices List Before you can monitor traffic with DC RUM, you have to add and configure an Agentless Monitoring Device using the RUM Console. To add an AMD to the list: Adding an AMD 1. Start and log on to RUM Console. 2. Select Devices and Connections Add device from the top menu. The Add Device pop-up window appears. 3. From the Device type list, select AMD. 4. In the Description box, type a description of the device. TIP It is recommended that you include the parent device name in the description of each device you add and to add these names consistently. This enables you to easily find your device in the list. Specifying the Connection Information 5. In the Device IP address box, type the device IP address. 6. In the Port number box, type the port number for communication with this device. The standard port number used by AMD is Optional: Select Use secure connection if you want to use HTTPS (secure HTTP) for communication between the console and the device you are adding. 23

24 Chapter 2 Adding Basic DC RUM Devices Providing the Authentication Details 8. Type the user name and password of the account that will be used for managing this device. By default, the AMD user is set to compuware and the password is set to vantage. The credentials entered here are used by the RUM Configuration to communicate with the device and are also passed to the report servers so that they can collect monitoring data for processing. Note that the values used here for authentication are not the same as the values you use for logging in to the device via SSH or local console. Configuring Advanced Settings 9. Select the Advanced options tab. 10. Optional: Under Secondary device connection, provide an alternative IP address for this device. 11. Optional: Enable SNMP connection. Optionally, you can define the SNMP connection parameters so that you can obtain more detailed health information about the device. To define SNMP connection parameters: a. Select SNMP Connection check box. b. Type the read community name and port number. 12. Enable Guided Configuration. By default, the Guided Configuration connection is enabled when you add an AMD. However, for performance reasons, the number of AMDs with enabled Guided Configuration is limited to 50. Any additional AMDs do not feed data to the Guided Configuration perspective. This means that the monitoring data from the additional AMDs is not available for generating the web traffic statistics or defining the web software services with a wizard. By default, the port number for communication between the Console Basic Analyzer Agent and the RUM Console Server is set to 9094 and the secure connection is enabled. In most cases, it is not necessary to modify this setting. If the default port number is already in use by other services, however, type the new port number in the Port number box. In this case, you also have to manually change the port number setting on the Console Basic Analyzer Agent side. For more information, see Modifying Connection Settings for Guided Configuration in the Data Center Real User Monitoring Administration Guide. 13. Click Next to test your connection parameters. If your configuration fails the test, you can go back and adjust your settings. Note that if the device fails to respond correctly, it may take several seconds before the test times out. 14. Click Finish to save the configuration. As a result, your device appears on the Devices list. To view the list, go to Devices and Connections Manage Devices in the top menu of the RUM Console. The Devices screen presents a comprehensive view of all the devices that you add, including their IP Address, Port, Description, Type, Version, Connection, Hardware Health, and Configuration. 24

25 Chapter 2 Adding Basic DC RUM Devices Adding a CAS to Devices List To view reports based on the data from the AMD, use the RUM Console to add and configure a CAS report server. Adding a CAS 1. Start and log on to RUM Console. 2. Select Devices and Connections Add device from the top menu. The Add Device pop-up window appears. 3. From the Device type menu, select CAS. 4. In the Description box, type a description of the device. TIP It is recommended that you include the parent device name in the description of each device you add and to add these names consistently. This enables you to easily find your device in the list. Specifying the Connection Details 5. In the Device IP address box, type the device IP address. 6. In the Port box, type the port number for communicating with this device. The standard port number used by the CAS when communicating over HTTP is Select Use secure connection if you want to use HTTPS (secure HTTP) for communication between the console and the device you are adding. Providing the Authentication Details 8. Choose whether authentication should occur via CSS. 9. Type the user name and password of the account that will be used for managing this device. Configuring the Advanced Settings 10. Select the Advanced options tab. 11. Optional: Under Secondary device connection, provide an alternative IP address for this device. 12. Click Next to test your connection parameters. If your configuration fails the test, you can go back and adjust your settings. Note that if the device fails to respond correctly, it may take several seconds before the test times out. 13. Click Finish to save the configuration. Configuring the CAS-AMD Connection 14. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 15. Select a report server from the list of devices. Click the server once to display the detailed information for the device. 16. Select the Data Sources tab. 25

26 Chapter 2 Adding Basic DC RUM Devices 17. Click Add Data Source. 18. Select your AMD from the list and then click the button. 19. Click Finish to save the configuration. As a result, your device appears on the Devices list. To view the list, go to Devices and Connections Manage Devices in the top menu of the RUM Console. The Devices screen presents a comprehensive view of all the devices that you add, including their IP Address, Port, Description, Type, Version, Connection, Hardware Health, and Configuration. What to Do Next It is important to keep the devices synchronized to avoid improper data interpretation. For more information, see Synchronizing Time Using the NTP Server in the Data Center Real User Monitoring Smart Packet Capture User Guide and Time Synchronization Between AMD and Server in the Data Center Real User Monitoring Administration Guide. 26

27 CHAPTER 3 Verification of Traffic Monitoring Quality Use the RUM Console to verify the traffic monitoring quality using two tightly connected solutions: Sniffing Point Diagnostics and Application Overview. We highly recommend that you perform this step at the beginning of your DC RUM deployment to verify that your hardware is working properly and that the applications you intend to monitor are detected. You can verify the test results and repeat them as needed at any time and for any network conditions. IMPORTANT All verification is based on a traffic recording, either manual or automatic. The outcome may not be representative if the target traffic is low at the time of recording or if you are unable to capture a satisfactory number of complete sessions. Choose automatic or manual traffic recording to capture unfiltered or filtered traffic. Enable automatic recording only during the configuration process and then disable it. It can negatively affect the performance of the AMD during normal operations, especially if you are running a 32-bit AMD in a high-traffic environment or a 64-bit AMD with the native driver. For the most complete and reliable statistics, use the 64-bit customized driver on the AMD. The verification of traffic monitoring quality is possible only for AMD 11.7 or later. Sniffing Point Diagnostics Sniffing Point Diagnostics is a type of hardware state analysis that enables you to perform pre-monitoring tasks without the need to access the AMD terminal. You can use it to validate the operation of the sniffing points, instead of using a series of UNIX or rcon commands. This step can be performed at the DC RUM deployment stage or at any other time to determine if the AMD performance is affected by malfunctioning hardware or external networking conditions. The Sniffing Point Diagnostics analysis can detect issues, such as: No traffic detected on sniffing interfaces. Interface or link overload. 27

28 Chapter 3 Verification of Traffic Monitoring Quality Poor quality of traffic due to mirrored ports on switching hardware configuration. Dropped packets (indicates AMD overload). Network conditions when unidirectional traffic prevails. Rejected packets, invalid packets, wrong check sums for packets. Missing packets (either lost or dropped). Missing bytes (how much traffic is lost in general). Conditions affecting AMD calculations, such as: Duplicate traffic that cannot be handled by the AMD. Incorrect choice of packet deduplication method. Incorrect settings for packet deduplication buffer. Incorrect settings for maximum packet size or huge packet size. Conditions affecting AMD performance, such as: Duplicate traffic handled by the AMD. Large percentage of non-ip traffic (noise). Large percentage of non-tcp or non-udp traffic (noise). Reordered sessions. Miscellaneous SSL problems: Unsuccessful decryption (in general). Uninitialized SSL cards unable to decrypt traffic. The ratio of encrypted and successfully decrypted traffic to encrypted and non-decrypted traffic. Incorrect or missing private keys. No match between the key and server certificate. Dropped or corrupted packets preventing decryption. Unsupported cipher methods (for example, Diffie-Hellman based key infrastructure). Unsupported SSL versions or features. Prerequisites and Best Practices To diagnose application detection problems and sniffing point connection problems, ensure that: All cables are connected correctly. The AMD is properly installed and configured. This includes the post-installation steps, such as interface identification and network configuration. Traffic recording lasts long enough to capture a reasonable amount of traffic volume, for example, 20 to 30 minutes of traffic. 28

29 Chapter 3 Verification of Traffic Monitoring Quality Do not use specific capture profiles when recording traffic. Always use the All available option for capture profiles when you do manual recording. When you need to diagnose traffic or capture port problems, enable automatic trace recording. Trace recording provides access to regular and fresh snapshots of the traffic that is traveling on your network. Sniffing Point Diagnostics Reports Sniffing Point Diagnostics reports are organized into several sections, each representing a separate set of metrics related to either hardware or network traffic. This topic provides directions for viewing the reports, but you can follow each step or skip steps to view the only the information important to you. 1. Start either by looking at device health or from the reports section directly. If you enabled automatic trace recording, you can monitor the device state on the Device Status tab of the Devices screen. A separate set of statistics is provided for each AMD added to the console. If there are any alarm messages, go to Devices and Connections Verify quality of monitored traffic. Inspect network interfaces in detail for a selected AMD. Open the Overview report to verify that the proper type of network driver is being used (custom or native) and that traffic has been detected, and check the number of dropped packets and other performance related issues. You can also verify that the NIC drivers are operational. For more information, see Network Interface General Statistics [p. 29]. 2. Switch to the Protocols section to inspect protocols. See whether network protocols are detected (IPv4 or IPv6) and verify detection of transport protocols (TCP or UDP). For more information, see Network and Transport Protocol Information [p. 32]. 3. Switch to the Services section to see the most active services. For more information, see Services Detected in the Traffic [p. 32]. 4. Depending on your goals, switch to the Sessions section either by selecting a particular service on the Services report to see session details or by choosing the Sessions section to see general statistics for all sessions. For more information, see Session-Related Statistics [p. 32]. 5. If you use SSL decryption, you can inspect whether there are problems detected for the currently used SSL engine or keys. For more information, see SSL Diagnostics [p. 34]. Network Interface General Statistics The Overview section of the Sniffing Points Diagnostics reports enables you to verify the general state of capture ports on a selected AMD. The information in the Overview section is gathered directly from the NIC driver operating on the AMD. For the most reliable results, use the 64-bit customized drivers. 29

30 Chapter 3 Verification of Traffic Monitoring Quality Calculation of Analyzed Traffic The calculation of analyzed traffic is performed in several stages, gradually excluding the irrelevant statistics: 1. The overruns are excluded first. When the received packets are counted, the overruns are omitted. 2. The calculation of the received packets depends on the subtraction of errors and filtered-out packets. 3. The dropped packets are counted after the filtered packets are disregarded. 4. The number of analyzed packets is the count of packets remaining after all of the previous categories are subtracted. In default AMD installations, non-tcp/udp packets are not part of this process and are never counted when the number of analyzed packets is given. Non-TCP/UDP traffic increases the amount of analyzed traffic only if you enable the monitoring of the default software services. Figure 6. Graphical Explanation of Analyzed Traffic Calculation for an AMD with 64-bit Customized Network Interface Driver All network packets Overruns Packets not received Received packets Errors and non-conditional filtering Errors: length or bad checksum; filtered out: non-ip Load balancing If active, fraction of the traffic Configuration filtering Based on defined software services Sampling and dropped packets Packets not analyzed due to performance issues Non-TCP, non-udp If default software services enabled Analyzed packets 30

31 Chapter 3 Verification of Traffic Monitoring Quality Interface Operation-Related Metrics The statistics presented on this screen include: Overruns Overruns may indicate a link overload. The overload is typically caused by an exceptionally high traffic volume. This value may also indicate that the network interface or network interface driver cannot manage the amount of traffic received. Other hardware-related issues may also cause overruns. If a high overrun occurs, limit the traffic volume received by the card. Errors (length) Packets of erroneous length are reported when they are too big (such as jumbo frames) or are bigger than the maximum transmission unit (MTU). To avoid such problems, you can increase Maximum packet size in the Entire Configuration perspective. For more information, see Configuring General Data Collector Settings [p. 45]. Errors (bad checksum) Checksum-related errors are typically caused by insufficient signal strength on an optical link. In other cases, checksum errors may indicate Ethernet distortion, such as duplex problems, where the checksum errors may result, for example, when the duplex auto-negotiation process fails. Check the host switch and AMD duplex settings. Filtered out (non-ip) Non-IP packets, such as ARP traffic. Even large numbers of such packets are generally considered harmless. They are not analyzed by the AMD software and are regarded as noise. Preventing such traffic from reaching the AMD may reduce the possibility of performance degradation. Filtered out (load balancing) This setting is only applicable in deployments with multiple AMDs where each device only analyzes a certain part of the same traffic. Filtered out (configuration) Provides additional filtering based on software service definitions. In default installations, where monitoring of the default software services is turned off, the driver limits the number of processed packets to only those that are relevant to the IP addresses included in user-defined software service definitions. Dropped (sampling) Sampling here means dropping packets when the driver performance is degraded. Packets are dropped in a controlled manner, and always with care, to preserve complete and consistent sessions. The packet drops almost always mean that traffic is too heavy for a complete analysis and that, with packet drops, the precision of CAS reports is diminished. Sampling is only active with the customized 64-bit driver and diagnostics always use this sampling mechanism regardless of the settings used in the general AMD configuration. Dropped (driver performance) Drops are always a symptom of problems, especially when SSL analysis is deployed. Drops occur when AMD software is unable to analyze all of the packets it receives from the driver. If you use 32-bit or native drivers, you may experience uncontrolled packet dropping. If you use the 64-bit customized driver, packet dropping may occur, but in a software-controlled manner with care for monitored data contingency. 31

32 Chapter 3 Verification of Traffic Monitoring Quality To avoid packet dropping, decrease the traffic volume that your AMD analyzes or reduce the number of monitored software services. Non TCP/UDP Whether these statistics are classified as analyzed or not depends on the default software service monitoring. The numbers in this section are mostly relevant if you enabled monitoring of default software services. In this case, ICMP traffic is also analyzed. If monitoring of the default software services is disabled and you still see a large percentage of non-tcp and non-udp traffic, it is possible that AMD performance will be affected. Network and Transport Protocol Information Use the Protocols report to check the ratio of supported transport or network protocols. Only supported protocols are shown. In general, this report enables you to check whether traffic that makes sense (from the DC RUM perspective) is present and is heavy enough to give meaningful results for report servers. NOTE To obtain the most reliable results, use 64-bit customized drivers. The limited approximation algorithms used by native and 32-bit customized network interface drivers may lead to differences between the packet count in this and the Overview sections. Problem Detection Low traffic for the IPv4 or IPv6 network protocols may indicate further monitoring problems. The presence solely of multicast or broadcast traffic is an indication that port mirroring is not enabled or inactive. Services Detected in the Traffic This overview report enables you to identify the most active services on your network. You can see what their load is and what protocols they use, and filter the results to display all data, monitored services, or unmonitored services. You can also use filters to display statistics for all, monitored, or unmonitored services with additional protocol filtering. For each service, you can open the Sessions report to verify session-level statistics. NOTE To obtain the most reliable results, use 64-bit customized drivers. The limited approximation algorithms used by native and 32-bit customized network interface drivers may lead to differences between the packet count in this and the Overview sections. Session-Related Statistics The Sessions section enables you to view detailed information about traffic quality. The statistics presented on this screen include: 32

33 Chapter 3 Verification of Traffic Monitoring Quality Duplicates, Unhandled duplicates The value presented on the Sessions screen depends on the currently selected deduplication method in your AMD configuration. Packet duplicates may indicate incorrect configuration of mirroring ports. While this may be a sign of a problem, values of 10 to 20 percent typically are no reason for concern. The AMD is capable of packet deduplication. Higher numbers of duplicate packets will degrade the AMD performance and may negatively influence the monitoring results. The diagnostics mechanism for duplicate detection and counting for this report works with different settings than the network monitoring processes on the AMD. Duplicate detection is performed using both methods of duplicate detection and with different settings (buffer and delay detection size). Based on these settings and calculations, Sniffing Point Diagnostics provides suggestions concerning duplicate handling, such as increasing buffer size or changing the deduplication mechanism. You should check whether there are unhandled duplicates detected, in which case it is suggested that you switch the detection method in the AMD general settings. For more information, see Configuring General Data Collector Settings [p. 45]. Unidirectional TCP sessions and UDP streams This may indicate a problem related to incorrectly configured mirroring ports. If the value of unidirectional traffic exceeds 90 percent, the RUM Console always marks it as an error. The numbers on the Sessions screen are the sums of many measurements; you are able to go deeper and analyze details for each server and check whether this is a problem related to a significant service or protocol. Insignificant traffic may be recorded and included in the general analysis, so always check the detailed reports when you see alarming numbers on the Sessions report. TCP sessions with missing packets Missing packets may result from interface or driver packet drops. If a session with missing packets is shown, the percentage value is counted with regard to all sessions. For example, if two percent of sessions have missing packets reported, this means that two out of a hundred sessions have missing packets. TCP sessions with missing packets and TCP bytes lost in missed packets may provide valuable insight into SSL decryption problems, especially in the case of long SSL sessions. TCP bytes lost in missing packets This is a complementary value to the TCP sessions with missing packets. Verify the number of lost bytes with regard to missing packets to see whether the problem is serious (if there are large sums of missing bytes). This is useful additional information in the case of long TCP sessions; because one lost packet is enough to classify a session as having missing packets, the number here gives insight into the actual loss rate. TCP sessions with reordered packets Reordered packets are typically found when there is a WAN link enabled. Devices transferring WAN packets may affect the packet order. The existence of reordered packets is not a problem in itself, because the AMD software can restore original packet order, but an excessive number of such packets may cause performance degradation. 33

34 Chapter 3 Verification of Traffic Monitoring Quality NOTE To obtain the most reliable results, use 64-bit customized drivers. The limited approximation algorithms used by native and 32-bit customized network interface drivers may lead to differences between the packet count in this and the Overview sections. SSL Diagnostics The traffic for this report is dependent on capturing complete sessions. Incomplete sessions, missing packets, or missed handshakes cause a large number of errors and a large number of errors results in unreliable reports. Always be sure to record enough traffic for an adequate length of time to allow you to capture complete sessions. The Statistics for encrypted traffic, SSL card and keys report is only available after the traffic trace recording is finished. Partial statistics for SSL are not provided for unfinished sessions. General Statistics for Encrypted Traffic For a given time range, defined by the scope of the recorded traffic traces, you can see the recognized SSL engine (for example, OpenSSL or ncipher) and the number of keys exchanged in the traffic. The remaining sections of this diagnostic report show the detailed information about the keys, the overall summary of the captured SSL traffic, and whether there are errors. The servers section shows information for all SSL traffic captured during the traffic trace recording. All of the detected encrypted protocols are listed together with their matching keys, if they are seen in the traffic. You can see whether the key exchange was successful; the matched keys are indicated by the icon. Key and certificate matching enables you to verify that certificates were found and were valid. No matching may indicate that the certificates are out of date. SSL Server Status The Status column shows whether there are errors or whether erroneous sessions prevail. A traffic capture sometimes does not contain session beginnings, or it contains incomplete handshakes, or it has no master session; these sessions are marked as ignored, as indicated by the gray ( ) color bar. The sessions with errors are marked by a red ( ) color bar. The main causes of errors are missing packets or missing keys. Other causes of errors are listed in detail on the Detailed SSL Statistics for servers report. Detailed SSL Statistics for Servers Detailed SSL statistics for servers are accessed from the Server or Status columns. This report shows: The percentage of the sessions without error, with errors, or ignored. The counts of each problem, in detail, for the error or ignored sessions. The number of decrypted sessions if there are no problems. 34

35 Chapter 3 Verification of Traffic Monitoring Quality You can filter the results. Use Sessions finished to display the data for completed sessions. Use Sessions in progress to display the sessions that are still in progress (sessions that did not end before the traffic capture stopped; to see those session statistics). Figure 7. Example of Detailed SSL Statistics for Server, Errors Detected Due to Private Key Mismatch SSL Keys Because invalid or outdated keys are usually not removed from SSL cards, the list of keys for which an error status is indicated may be considerably long. In such cases, sort by the Status column to see keys correctly matched. Note that it may be necessary to format the SSL card storage area to refresh the key list. Application Overview The Application Overview screen enables you to answer several questions about your applications at the onset of your monitoring configuration. Are all my applications or servers detected? What applications or servers are detected? Can the detected applications or servers be successfully monitored? How heavy is the traffic for each application or server? What services are detected on each server? How heavy is the traffic for each detected service? Note that incomplete sessions are not analyzed. If no beginning is recorded for a session, that session is not analyzed. 35

36 Chapter 3 Verification of Traffic Monitoring Quality The Application Overview screen is an optional step towards defining new software services. To access it, select Software Services Add Software Service in the console top menu, then select By traffic lookup. Figure 8. Example of the Application Overview Screen Showing Detected Applications From this screen, you can configure software services either manually or by using the wizard. If it is possible to go through a step-by-step configuration, a wizard icon ( ) is displayed for the given protocol or service. Application Detection Mechanism Application detection is a three-stage process: 1. To provide the most accurate results, packet analysis for SSL, HTTP, HTTPS, SOAP, and related protocols is performed as a first step toward application type detection. Application recognition is based on the first matching pattern found. This means that some services may not be properly classified if multiple protocols are used in one session. For example, if your application uses HTTP and SOAP over HTTP protocols, and plain HTTP communication opens a session, the application is classified as HTTP. 2. Applications are also detected based on discovery of well-known ports. The default protocol definitions are stored on the AMD and can be exported from the RUM Console. For more information, see Exporting the AMD Configuration [p. 40]. At times applications may use ports commonly used for other purposes. The AMD is unaware of these circumstances and will report well-known protocol names. For example, if one of your web applications uses port 8080 and uses HTTP for communication, the AMD reports this as an HTTP proxy. 3. If none of the selected conditions matches, the application is labeled as Unknown TCP or Unknown UDP. 36

37 Chapter 3 Verification of Traffic Monitoring Quality Server recognition in application detection is based on heuristic session analysis; results may vary depending on the type of network interface driver used. Using RUM Console to Identify Problems Related to Network Hardware Operation Typical configuration errors related to port mirroring can, at times, severely affect the AMD software traffic analysis capabilities. Faulty hardware configuration may result in no data seen by the AMD, a large number of duplicate packets reaching the AMD, or only a limited portion of traffic visible to the monitoring software. Use the Application Overview and Sniffing Point Diagnostics sections as tools to solve issues related to the switching hardware configuration. The following list describes several common problems and some possible causes and solutions. No data seen by the AMD The cable is connected to the wrong physical port on the destination switch. This can be checked by physically tracing the cable directly to the switch and confirming the port ID. The port mirroring configuration (for example, SPAN on Cisco hardware) has been set or changed to mirror incorrect ports or an incorrect destination. This can be resolved by logging on to the source switch and checking the mirroring ports configuration relevant to the requirements (see the vendor-specific documentation for details). No data seen on Application Overview but non-tcp/udp traffic seen in interface statistics The port mirroring configuration (for example, SPAN on Cisco hardware) has been set or changed to mirror incorrect ports or an incorrect destination. This can be resolved by logging on to the source switch and checking the mirroring ports configuration relevant to the requirements (see the vendor-specific documentation for details). Application Overview does not show all expected data The port mirroring destination may be oversubscribed or dropping packets. Check this by logging on to the switch and checking the SPAN or mirror destination interface. If it is recording many drops, review the configuration of source ports to understand the ratio of source interface bandwidth to destination interface bandwidth. If the ratio is excessive (for example, greater than 4:1), consider reducing the number of source interfaces. If applicable, consider using device-specific filtering to reduce the load on the destination interface (for example, VACL, Rx-only, or Tx-only sources). By design, port mirroring does not forward faulty frames. Check the source device interface statistics to ascertain the nature of the drops (see the vendor-specific documentation for details). Check the interface-related metrics. If there is a high rate of Errors (bad checksum), consider hard-configuring one end of the AMD SPAN connection to prevent auto negotiation. Session-related report shows a high rate of packet duplicates A SPAN or mirror operates by copying frames from source interfaces and directing them to the destination interface. In effect, configurations often result in two copies of a packet. 37

38 Chapter 3 Verification of Traffic Monitoring Quality For example, if the source of a SPAN or mirror is set as a VLAN, any traffic that goes from one switch port to another switch port within the VLAN appears twice on the mirrored port. If the number of duplicates starts to affect AMD performance, consider reducing the number of source interfaces. If applicable, consider using a device-specific filtering control to reduce packet duplication (for example, VACL, receive-only, or transmit-only sources) or consider using tap technology as opposed to port mirroring to collect the data. Only unidirectional streams are seen on session-related overview If the AMD is connected via a SPAN or mirror, the configuration has been set incorrectly to send only one side of a receive or transmit stream to the destination. Log on to the local source switch to check the configuration (see the vendor-specific documentation for details). 38

39 CHAPTER 4 Propagating the Data Collector Configuration Using the RUM Console, you can edit, import, export or propagate the configuration of a Data Collector. Importing the AMD Configuration You can import the entire configuration from XML files, or selectively import specific configuration settings. The imported configuration becomes Draft by default. After an AMD is listed as one of the managed devices in a RUM Console, you can import a set of AMD configuration settings from an XML file. NOTE If you are importing older configurations for global settings and services (applications.xml) and for predefined software services (protocols.xml), they will be upgraded to validate against the XML schema installed on the RUM Console. To import an AMD configuration: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. In the upper-right corner of the screen, click the Import Configuration icon. 6. On the Import AMD Configuration screen, select the check box for the configuration file to import and click Browse to select the XML file for a specific configuration. Global Settings and Software Services {prefix}applications.xml Transactions {prefix}page2trans.xml 39

40 Chapter 4 Propagating the Data Collector Configuration Pre-Defined Software Services {prefix}protocols.xml Dynatrace Network Analyzer Agent Settings {prefix}avagt.xml Flow Collector Settings {prefix}nfc.xml 7. Click OK to import the configuration from XML files. Exporting the AMD Configuration You can export the configuration for every AMD managed in RUM Console. You can export the AMD configuration to a series of XML files: Global settings and software services Transactions Pre-defined software services Dynatrace Network Analyzer agent settings. To export an AMD configuration: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Select the Configuration type to export: Current Draft Pending. 5. Click the icon in the upper-right corner and select the export destination folder. 6. Define a prefix for each of the file names. The description will be added automatically as part of a file name for individual XML files: applications.xml for global settings and services avagt.xml for Dynatrace Network Analyzer agent settings. nfc.xml for Flow Collector settings. page2trans.xml for transaction definitions protocols.xml for pre-defined software services 40

41 Propagating the AMD Configuration Using RUM Console After an AMD has been added to Devices, RUM Console may distribute that configuration to a number of devices of the same type. To propagate the AMD configuration to a number of AMDs: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. On the Devices screen, select Current configuration Copy from the context menu for the AMD from which to copy the configuration (the source AMD). Optionally, you can propagate an unpublished configuration by selecting the Draft configuration Copy from the context menu for a given AMD. Note that propagating a draft configuration will overwrite the draft configurations on all of the destination AMDs. 4. Select the AMDs to which to copy the configuration (the destination AMDs). To ensure configuration compatibility, verify that all AMDs involved are of the same version. 5. Specify whether to propagate the entire configuration or whether to propagate selected parts of the configuration. To propagate the entire configuration, select Whole configuration. To propagate only specific parts of the configuration, select Part(s) of configuration and then select the configuration components to propagate. 6. Click OK. NOTE Chapter 4 Propagating the Data Collector Configuration After you add an AMD as a device, if its configuration is older than the RUM Configuration's, the current AMD configuration for global settings and services (applications.xml) and pre-defined software services (protocols.xml) will be upgraded automatically. The configurations for all AMDs connected as devices are also updated after each RUM Console upgrade. After the configuration is propagated to the selected destination AMDs, a message appears indicating that the draft configuration has been updated. What to Do Next The distributed configuration is presented as an updated draft for each of the destination AMDs. You must still publish all of the drafts before the new configuration takes effect. You can publish draft configurations individually for each AMD or click Publish Configuration on the Devices screen to publish them all at once. 41

42 Chapter 4 Propagating the Data Collector Configuration Propagating the AMD Configuration Automatically through RUM Console Server To copy configurations to AMDs, in addition to using the RUM Console, you can also use automatic propagation through the RUM Console Server. An AMD configuration, when exported properly, is expressed in four XML files. For more information, see Exporting the AMD Configuration [p. 40]. applications.xml for global settings and services avagt.xml for Dynatrace Network Analyzer agent settings. nfc.xml for Flow Collector settings. page2trans.xml for transaction definitions protocols.xml for pre-defined software services The computer hosting the RUM Console Server maintains a copy of the current configuration for each of the AMD devices in the <install_dir>\cva\eclipse\workspace\configuration\amd\export folder. The RUM Console Server automatically generates four XML configuration files per AMD in the RUM Console devices list. The name of each of the exported XML files contains the IP address and port number of the associated AMD. These files are located on the RUM Console Server in the folder <install_dir>\cva\eclipse\workspace\configuration\amd\export. Modify these files then propagate the configuration to several or all of the AMDs in the RUM Console Devices list. A configuration file must have one of four standard names (application.xml, avagt.xml, page2trans.xml, or protocols.xml) and may be further qualified with an IP address and port number. The following examples illustrate how this works: Suppose the \import folder contained XML files named as follows: application.xml avagt.xml page2trans.xml protocols.xml In this case, the RUM Console Server applies these configuration files to all AMDs available in the RUM Console Devices list, regardless of IP address. Suppose the \import folder contained the following set of XML files: _9091_application.xml _9091_avagt.xml _9091_page2trans.xml _9091_protocols.xml 42

43 Chapter 4 Propagating the Data Collector Configuration Note that the file names now contain an IP address and port number. In this example, the RUM Console Server automatically applies each of these configuration files only to the AMD with IP address using port number Any other AMDs are not affected. Suppose the \import folder contained just a subset of those files: _9091_avagt.xml _9091_protocols.xml In this example, the RUM Console Server applies the listed XML configuration files to that AMD. Because the _9091_application.xml and _9091_page2trans.xml files are not available, the parts of the configuration affected by the missing configuration files remain unchanged on that machine. Suppose the \import folder contained two of the same type of XML file, one with a specific IP address and the other without: application.xml _9091_application.xml In this case, the RUM Console Server applies the application.xml configuration file to all AMDs except the AMD at address , which receives the contents of _9091_application.xml instead. An address-specific configuration file always overrides a general configuration file of the same type. Suppose the \import folder contained XML files whose names did not adhere to the above file naming syntax, such as: my_configuration_application.xml some-file_avagt.xml test-amd-page2trans.xml all_protocols.xml In this case, the RUM Console Server ignores these files and would not propagate any settings they might contain. To propagate a configuration set from one AMD to the rest of the AMDs, copy the XML configuration file set from the..\cvaconfig\amd\export folder to the..\cvaconfig\amd\import folder and rename all four file names so they do not include the IP address and port number. The RUM Console Server will publish the configuration within 60 seconds of a change detection. Considering that a default directory check occurs every 60 seconds and by default the publish delay is set to 60 seconds, the maximum wait for the configuration to begin propagating is 120 seconds after a new configuration file has been added to the..\cvaconfig\amd\import folder. 43

44 Chapter 4 Propagating the Data Collector Configuration 44

45 CHAPTER 5 Basic Monitoring Configuration You can define many configuration settings globally for all software services for a given protocol and Data Collector, or locally for specific user-defined software services. If you specify both types of settings, the settings for a user-defined software service take precedence over the corresponding global settings. Use the RUM Console to perform basic monitoring configuration, including the global settings for Data Collectors, operations, and the analyzer, as well as configuring Dynatrace to recognize WAN-optimized traffic. Configuration and recognition of optimized WAN traffic in Dynatrace is optional and depends on whether WAN optimization is used in your network. Refer to the Data Center Real User Monitoring WAN Optimization Getting Started. NOTE If you make any significant changes in the configuration, such as removing defined software services or operations, your are advised to restart the AMD. This is to prevent persistent TCP sessions from blocking your changes. Configuring General Data Collector Settings For any given data collector device such as the AMD, you can set a variety of options, such as time thresholds. The general settings affect the monitoring of default and user-defined software services. Some of these settings can then be overridden by settings for a particular analyzer, software service, or URL. To define the general settings for an AMD: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Select Configuration Global General to access the list of general configuration settings. 45

46 Chapter 5 Basic Monitoring Configuration While some of the options control only general AMD behavior, some options in the Advanced group affect more specific configurations in application monitoring. For example, if Inherit from global settings is selected in your other configurations while configuring user-defined software services, the global setting takes precedence over the specific monitoring configuration. Configuration options include: Monitoring interval The monitoring interval in minutes. Increasing this value reduces the number of chunks of data that need to be transferred and processed. Default: 5 minutes. Verify that the monitoring interval is synchronized between the data collectors. Operation time threshold The number of seconds after which an operation is considered to be slow. The global threshold value depends on the analyzer. This threshold is used by the following analyzers: Cerner Cerner over MQ Epic Generic with transactions HTTP MS Exchange over HTTP MS Exchange over HTTPS Oracle Applications over HTTP Oracle Applications over HTTPS SAP GUI SAP RFC SAP GUI over HTTP SAP GUI over HTTPS SMTP SSL SSL Decrypted Server time threshold The Server time threshold relates to the server time portion of an overall operation time. Server times above the threshold limit are considered to be slow due to poor datacenter performance. This threshold is used by the following analyzers: HTTP SAP GUI over HTTP SAP GUI over HTTPS IP address of the server authorized to set AMD time The IP address of the report server that has the authority to synchronize the time with this AMD. 46

47 Chapter 5 Basic Monitoring Configuration In an environment with a number of servers sharing the same AMD, it is good practice to designate only one of these servers as a time synchronization server to make changes to AMD settings. Otherwise, the server used for time synchronization will change inadvertently every time you save an AMD configuration. Default analyzer The default setting for the TCP analyzer is Generic (with transactions). To change it, select another analyzer from the list. Client RST packet timeout to mark session as CLOSED If the time between the last ACK for data sent by the server and an RST packet sent by the client is greater than this value, the session is treated as closed instead of aborted. Huge packet size The upper size limit, in bytes, of an HTTP request to be processed successfully by the AMD. Maximum packet size The AMD is capable of processing packets of up to bytes, besides the Ethernet standard MTU (Maximum Transmission Unit) of 1536 bytes. Choose one of the predefined values (2048, 4096, 8192, or bytes) to enable the AMD to process non-standard MTU packets. When you have chosen the Maximum packet size value, make sure that you also set the Huge packet size to an applicable value. Enabling theamd to process nonstandard MTU packets without extending RAM on the machine and leaving Packet buffer size (64-bit AMDs only) and Data memory limit unchanged can cause an excessive packet loss. To avoid this, extend RAM and configure its usage as recommended in the tables below. For more information, see Setting Packet Buffer Size in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide and Setting Data Memory Limit in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. NOTE Do not enable the processing of large packets for a Small AMD. These devices are not designed to process larger packets. For more information, see Small AMD in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. Table 2. Recommended RAM Configuration for Maximum Packet Size Values for AMDs Maximum packet size 8192 B or less 8192 B B Recommended RAM size for 64-bit platforms 64 GB 96 GB 128 GB 47

48 Chapter 5 Basic Monitoring Configuration Sampling enabled Supported in 64-bit customized AMD drivers and all- native drivers. The sampling mechanism is beneficial when heavy traffic may negatively affect AMD performance and there is a risk of losing IP session consistency. When this option is enabled, the AMD tries to analyze the greatest possible portion of traffic. It drops packets in a controlled manner that preserves complete and consistent sessions. Note that statistics for dropped packets are not shown on the report server. If packets are dropped because of sampling, the CAS shows notification messages. For percentages between 75 and 99, a warning icon is displayed; for values below 75, the report server issues error messages. When this option is disabled and the network interface driver performance is degraded, random packets are dropped. Default: enabled. For more information, see Using Network Interfaces with Native Drivers in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide and Driver, Network, and Interface Configuration in the Data Center Real User Monitoring Agentless Monitoring Device Installation Guide. NOTE When capturing packets on an AMD with sampling disabled, if the AMD experiences packet drop due to high traffic volume, the packet capture is not automatically canceled. If this occurs, select Tools Packet Data Mining Tasks on the CAS, find the task that was using the AMD in question, and click to cancel that task. Deduplication method You can choose one of four methods for eliminating duplicate packets: Based on TCP checksum and IP ID Using this method, duplicate packets are detected based on their TCP checksum and IP ID. Based on TCP checksum and IP ID (excluded SEQ and ACK numbers) Using this more complex, two-stage method, duplicate packets are detected based on a modified packet KCP checksum (SEQ and ACK numbers are excluded) and IP ID. This method is useful if the AMD captures packets on various interfaces of the router, rewriting SEQ and ACK numbers. A packet is considered a duplicate when the modified checksum, IP ID, and SEQ and ACK numbers are identical. First, a packet checksum with SEQ and ACK numbers is created and compared to the packets stored in the detection buffer. If the comparison indicates that the packet is not a duplicate, it is checked to determine whether it matches the current session. A packet matches the current session when its SEQ and ACK numbers are different from processed and cached numbers by the value defined in TCP duplicate window. If the difference exceeds the defined value, the AMD assumes the ACK and SEQ numbers were rewritten by the router and the packet is considered a duplicate. 48

49 Chapter 5 Basic Monitoring Configuration TCP checksum, IP ID and MAC address (excluded SEQ and ACK) Using this method, the deduplication process is similar to the one based on TCP checksum and IP ID (excluded SEQ and ACK numbers), but in addition to TCP checksum and IP ID, the source/destination MAC addresses are also taken into account for the calculation. TCP checksum, IP ID and MAC address Using this method, duplicate packets are identified based on their TCP checksum, IP ID and source/destination MAC addresses. TCP duplicate window This setting is useful only if Deduplication method is set to Based on TCP checksum with excluded SEQ and ACK numbers. It is used for determining whether a packet, based on its SEQ and ACK numbers, belongs in the session. If a packet's SEQ and ACK numbers differ from the current session's SEQ and ACK numbers by a value larger than TCP duplicate window, the packet is considered a duplicate. Default: Packet buffer size The number of packets to keep in the buffer for use as a basis for comparison in duplicate packet detection. Newly captured packets are sequentially compared to the packets in the buffer. A newly captured non-duplicate packet (all packets in the buffer are unique) is placed on the top of the stack and the oldest is removed. Range: 6 to 24 packets. Default: 16. Reset duplicate detection time threshold Time of inactivity (in seconds) after which the duplicate packets elimination mechanism is reset. If Deduplication method is set to Based on TCP checksum with excluded SEQ and ACK numbers or TCP checksum, IP ID and MAC address (excluded SEQ and ACK), and the Reset duplicate detection time threshold should be greater than every response generation time (server time). 5. Save or publish the configuration. Click Save to save your changes and continue with configuration. Click Save and Publish to immediately update the devices configuration. 6. Close the AMD Configuration window. General Configuration Options for HTTP Express Software Services HTTP general configuration options for the HTTP Express analyzer are limited to the options related to the HTTP sessions and session timeouts. They can be set globally for the AMD or individually for particular software services. 49

50 Chapter 5 Basic Monitoring Configuration Before You Begin It is assumed for this task that you have already created one or more user-defined software services for this protocol and that you know how to access and modify global settings for an AMD and settings for a service. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Front-End-Monitoring Web HTTP Express General. NOTE Configuration options related to general settings for an AMD for this protocol analyzer are also under the HTTP Options tab for individual user-defined services. 6. Configure options available in the General section. The list of configuration options includes: Last packet HTTP session timeout If the time since the last packet for an HTTP session is longer than this value (in seconds), the hit is considered finished and closed. This timeout period is configured globally for all software services. Report URL prefixed with analyzed HTTP method If this option is selected, the string POST or GET is prefixed to the reported URL. This option can be set globally for all software services or configured for a specific user-defined software service. Service-specific settings take precedence over global settings. Treat a client RST packet sent by the session as closing session If this option is selected, the protocol analyzer treats a client RST packet sent by the session as closing the session instead of aborting it if there was no content length header. It is configured globally for all software services. 7. Optional: Configure URL Auto-Learning. For more information, see Configuring URL Auto-Learning in the Data Center Real User Monitoring Web Application Monitoring User Guide. 8. Optional: Configure Character Encoding Support. For more information, see Character Encoding Support for HTTP Services in the Data Center Real User Monitoring Web Application Monitoring User Guide. 9. Save or publish the configuration. Click Save to save your changes and continue with configuration. 50

51 Chapter 5 Basic Monitoring Configuration Click Save and Publish to immediately update the devices configuration. 51

52 Chapter 5 Basic Monitoring Configuration 52

53 CHAPTER 6 Configuring NetFlow Monitoring To configure NetFlow monitoring, you first configure NetFlow-enabled devices to forward flows to AMD, configure AMD to set a number of global parameters for all flow sources, observe the detected flow sources, and adjust the settings for specific flow sources with problems. 1. Configuring remote NetFlow-enabled devices to send flows to AMD. Refer to appropriate documentation of your NetFlow-enabled devices for instructions on how to perform the configuration. For more information, see Configuring Remote NetFlow-enabled Devices [p. 53]. 2. Configuring general NetFlow settings on the AMD. For more information, see Configuring Flow Collector General Settings [p. 58]. 3. Waiting until flows start appearing in the AMD Flow Sources table and adjusting the settings for individual devices. For more information, see Observing and Fine-Tuning NetFlow Sources [p. 60]. 4. Viewing NetFlow reports on the report server and repeating the above tasks to increase or decrease the number of flow sources as required. An extensive set of network performance monitoring reports has been provided. You can also design your own reports or customize some of the reports that have been provided. For more information, see Network Analysis Reports [p. 95]. Configuring Remote NetFlow-enabled Devices To configure remote NetFlow-enabled devices to send flows to AMD, refer to appropriate vendor documentation. For NetFlow version 9, configure only selected fields. Ensure that NetFlow reporting does not create an unacceptable load on the CPU. Supported NetFlow versions The following NetFlow versions are supported: NetFlow version 5 53

54 Chapter 6 Configuring NetFlow Monitoring NetFlow version 9 and IPFIX with elements of Flexible NetFlow: Ability to configure the NetFlow collector to retrieve and interpret client RTT information from a specified field. This feature has been tested for Riverbed and Cisco network devices. WAN optimization monitoring: ability to specify WAN interface names, to facilitate NetFlow monitoring of WAN-optimized traffic. Choice of NetFlow-enabled devices and CPU load considerations Border routers are the natural choice for gathering NetFlow information. While it may be useful to monitor NetFlows on internal data center devices, indiscriminate monitoring of all interfaces on hot/core devices can generate very large numbers of flows and create significant CPU load on these devices. In such cases, CPU load monitoring is particularly important and port pairing may needed to reduce the load and avoid duplication. Transport mechanism In general, NetFlow-enabled devices can send flows over UDP or SCTP. DC RUM requires that UDP is used for flows sent to AMD. Special considerations for configuring NetFlow version 9 When configuring NetFlow version 9 you should: Decide if egress flows should also be sent to the AMD for processing: This is an important consideration, potentially allowing you to optimize how much data is collected and processed by DC RUM. If egress flows are to be monitored, care should be taken to export the DIRECTION field, else data duplication may occur. For more information, see Optimizing NetFlow Data Collection and Processing for NetFlow Version 9 [p. 54]. Specify which fields are to be exported: If spurious fields are exported, they will be ignored by the AMD NetFlow collector, though they will affect performance, as they will generate additional traffic. Therefore it is recommended that only the fields that are specified as processed by the NetFlow collector should be exported. For more information, see Configuring Fields to be Exported in NetFlow Version 9 [p. 56]. Optimizing NetFlow Data Collection and Processing for NetFlow Version 9 When configuring NetFlow version 9 you should decide if egress flows should also be processed: This is an important consideration, potentially allowing you to optimize how much data is collected and processed by DC RUM. However, if egress flows are to be sent to AMD, you must remember to export the DIRECTION field. In the case of older versions of NetFlow (version 5), where only ingress flows can be produced, you may be forced to monitor a larger volume of traffic, in order to be sure that you have analyzed all of the traffic of interest. As, for example, in a situation where you need to monitor the traffic between a server and a number of clients on the other side of a router. In this case you would need to monitor ingress flows on all of the interfaces, as pictured: 54

55 Chapter 6 Configuring NetFlow Monitoring Figure 9. Monitoring only ingress flows in NetFlow version 5 Client 1 Client 2 Client 3 ifc A ifc B AMD monitor ingress ifc C Router Server Indiscriminate monitoring of all of the interfaces may, however, force you to analyze a significant amount of other, unrelated traffic as in this case could occur on interfaces A and B on client side. If you are using NetFlow version 9, you can avoid this problem by limiting your monitoring to ingress and egress traffic on the single interface C at servers side, since both ingress and egress flows can be interpreted by the current version of DC RUM: 55

56 Chapter 6 Configuring NetFlow Monitoring Figure 10. Monitoring ingress and egress flows in NetFlow version 9 Client 1 Client 2 Client 3 ifc A ifc B monitor ingress and egress AMD ifc C Router CAUTION Server If a flow has no DIRECTION field exported, the assumption is that it is an ingress record. Therefore, if both ingress and egress records are being sent, you need to make sure that the DIRECTION field is exported, else the data count performed by DC RUM will be doubled. Configuring Fields to be Exported in NetFlow Version 9 When configuring NetFlow version 9 you should specify which fields are to be exported: If spurious fields are exported, they will be ignored by the AMD NetFlow collector, though they will affect performance, as they will generate additional traffic. The following table shows which NetFlow version 9 fields need to be configured for export so that DC RUM NetFlow analysis functions correctly. CAUTION If a flow has no DIRECTION field exported, the assumption is that it is an ingress record. Therefore, if both ingress and egress records are being sent, you need to make sure that the DIRECTION field is exported, else the data count performed by DC RUM will be doubled. 56

57 Chapter 6 Configuring NetFlow Monitoring Table 3. Supported NetFlow v9 fields Field Type ID Processing by DC RUM Length Description IN_BYTES 1 Yes Length not fixed; default is 4. Incoming counter with length IN_BYTES x 8 bits for number of bytes associated with an IP Flow. IN_PKTS 2 Yes Length not fixed; default is 4. Incoming counter with length IN_PKTS x 8 bits for the number of packets associated with an IP Flow. PROTOCOL 4 Yes 1 IP protocol byte. SRC_TOS 5 Yes 1 Type of Service byte setting when entering incoming interface. TCP_FLAGS 6 Yes 1 Cumulative of all the TCP flags seen for this flow. L4_SRC_PORT 7 Yes 2 TCP/UDP source port number (for example, FTP, Telnet, or equivalent). IPV4_SRC_ADDR 8 Yes 4 IPv4 source address. If both IPV4 and IPV6 are present only IPV4 is reported. INPUT_SNMP 10 Yes Default length Input interface index. is 2 but higher values could be used. L4_DST_PORT 11 Yes 2 TCP/UDP destination port number e.g. FTP, Telnet, or equivalent. IPV4_DST_ADDR 12 Yes 4 IPv4 destination address. If both IPV4 and IPV6 are present only IPV4 is reported. OUTPUT_SNMP 14 Yes Default length Output interface index. is 2 but higher values could be used. IPV6_SRC_ADDR 27 Yes If both IPV4 and IPV6 are present 16 IPv6 Source Address. 57

58 Chapter 6 Configuring NetFlow Monitoring Table 3. Supported NetFlow v9 fields (continued) Field Type ID Processing by DC RUM Length Description only IPV4 is reported. IPV6_DST_ADDR 28 Yes 16 IPv6 Destination Address. If both IPV4 and IPV6 are present only IPV4 is reported. SAMPLING_INTERVAL 34 Yes If not specified, 1 is assumed. 4 When using sampled NetFlow, the rate at which packets are sampled e.g. a value of 100 indicates that one of every 100 packets is sampled. DIRECTION 61 Yes If not specified ingress flows are assumed. 1 Flow direction: 0 - ingress flow, 1 - egress flow. SUM_RT Yes Configurable. Vendor dependent. 4 The time taken by an application to respond to a request. It is also called Application Delay (AD) or Application Response Time. Configuring Flow Collector General Settings Flow collector general settings enable you to define settings common to all NetFlow sources. The process for classification and identification within the flow collector is in the following order: defined SS, followed by NBAR name, followed by port lookup in protocols.xml. To specify flow collector general settings: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Configuration Global Flow Collector General to access the list of general configuration settings for the flow collector. The list of options is organized into the following sections and includes: Flow listening port (UDP) In general, NetFlow-enabled devices can send flows over UDP or SCTP. DC RUM requires that UDP be used for flows sent to AMD. The network device is configured with an IP address and port number of the NetFlow collector that will receive the 58

59 Chapter 6 Configuring NetFlow Monitoring records and process them. This is the port number that the NetFlow collector will listen to for incoming flow records. Default: 2055 (This can be configured to something more suitable to your environment. Other but less commonly used ports are 9555 and 9995.) Flow Source SNMP polling The NetFlow collector actively polls the network devices from which it receives flow records. Information received through SNMP polling augments the information received through flows. In particular, it helps to associate the interface name and index, thus allowing for byte and packet counts to be calculated for the named interfaces. The incoming flow records contain an interface index (decimal number) that, when correlated with the interface name, makes reading reports much easier. For example, VPN to Tokyo is much easier than 26 to understand in the context of the reporting data. Also, interface index numbers can change across router and switch reboots, so without the name association you would constantly need to manually update a mapping table to really understand which link the interface index was referring to. By doing the SNMP query, the index and name are correlated automatically. In addition, when retrieving the byte counts over an interval time frame, the utilization for that link can be calculated and provided in the report. SNMP read community name This is a string used by SNMP V1 and V2 clients to authenticate with a SNMP agent. A common community string used is public. Note that this allows read-only access and does not compromise the security of your switch. SNMP port The SNMP port to be used for SNMP polling. Default: 161 Timeout This setting determines the number of seconds that the NetFlow collector will wait for an SNMP query request to return from a SNMP agent before it aborts the request. Default: 1 Retries The number of retries for attempting SNMP polling. Default: 5 Advanced Remove inactive Flow Source from status update after The number of days after which flow sources that have stopped reporting are removed from the table of flow sources. 6. Publish the draft configuration on the monitoring device. 59

60 Chapter 6 Configuring NetFlow Monitoring Observing and Fine-Tuning NetFlow Sources A list of the detected NetFlow sources appears automatically on the Flow Sources screen in the AMD configuration settings of the RUM Console. You can subsequently edit settings for individual NetFlow sources to modify the SNMP community name and SNMP port if the device status indicates an error. To access the table of the detected NetFlow sources and modify individual source settings: 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Navigate to Configuration Global Flow Collector Flow Sources 6. Review the information and settings for the displayed NetFlow sources. The status of the devices can assume the following values: OK The SNMP connection is working as configured. Error There is an SNMP connection error due to, for example, an incorrect community string or port number. None No status information has been received for this device since the device settings were last modified. N/A When a configuration change is made to specific NetFlow source settings (community name or port number), time is needed to obtain a new device status based on the new settings. This time delay should take about one monitoring interval, but could take up to two intervals, depending on where in the monitoring cycle the configuration change occurred. During this time, a None status is reported. The SNMP status not available for this outdated entry. Normally entries that have stopped communicating are automatically deleted from the status table. However, entries containing custom address or port numbers are retained for future reference but with a status of N/A. More detailed information for the selected row of the Devices table is displayed immediately below the table. In particular, the following information can be used for troubleshooting purposes: Last failed SNMP connection The time and date of the last failed connection with the device. This is useful if you are troubleshooting. If the last failed connection is a couple of minutes ago, you know this issue is current. 60

61 Chapter 6 Configuring NetFlow Monitoring Last flow received The time and date of the last flow received from the device. This shows you which devices may no longer be sending NetFlows. When they reach an inactivity time, they are automatically purged from the list. However, if they are reactivated, they will appear back on the list automatically. 7. Modify settings for problem entries. To correct SNMP settings for problem entries, modify the community name or port number as required. 8. Specify WAN interface names, if required. The WAN Interface configuration setting allows you to specify network interfaces for expected WAN optimization which allows the AMD to determine whether traffic compression occurs during the optimization. 9. Configure RTT-related settings, if required. For NetFlow version 9 it is possible to configure the NetFlow collector to retrieve and interpret client RTT. a. Configure RTT template value. The value entered is this field allows the analyzer to identify which field carries the RTT information. This value is vendor-specific and it should not be regarded as a simple field number. This feature has been tested for Riverbed with the value of 110 and for Cisco with the value of b. Configure RTT measurement type. The value in the field specifies how to interpret the RTT information received from the NetFlow device. For example, single measurements vs. cumulative measurements. Note that this is vendor-specific and should not be regarded as a simple factor to divide the metric by. For tested Riverbed equipment this parameter was not used (enter 0), for Cisco enter Publish the draft configuration on the monitoring device. 11. Wait and re-examine the updated status information After you publish the configuration, you will have to wait up to two monitoring intervals for the displayed status of the modified entries to be updated. During that time the status will appear as None. After waiting of maximum two monitoring intervals, the status information will be updated for the entries for which configuration was modified. 12. If problems persist, repeat Step 7 [p. 61] to Step 11 [p. 61] until all problems have been resolved. 61

62 Chapter 6 Configuring NetFlow Monitoring 62

63 CHAPTER 7 Configuring AMD to Monitor User-Defined Software Services If you require detailed analysis of traffic, you need to specify software services to be monitored on specific IP addresses and ports. Defining Oracle Forms Software Services For software services you intend to monitor that do not work on a well-known port, you can use the specific IP address and port of the service when defining the software service configuration. For such software services, you can measure a wide range of metrics and perform detailed traffic analysis. To add a new software service: 1. Start and log on to RUM Console. 2. In the top menu, select Software Services Add Software Service. The Add Software Service pop-up window appears, listing all ways of adding a new service. 3. Select Manually as a method of adding a new software service definition. The Add Software Service window appears. 4. Specify basic information for your software service. Provide a software service name. Select appropriate analyzer to monitor the traffic. Using check boxes, select the devices that will monitor the new software service. When you later publish the software service definition, the new configuration will be applied only to the selected devices. 5. Click OK to proceed to monitoring rules configuration. 6. Right-click in the Rules table and select Add from the context menu. The Rule Configuration appears. 63

64 Chapter 7 Configuring AMD to Monitor User-Defined Software Services 7. Proceed with the rules configuration.for more information, see Configuring Rules for User-Defined Software Services in the Data Center Real User Monitoring Oracle Forms Application Monitoring User Guide. 8. On the Software Services screen, click Publish Configuration. Configuring Rules for User-Defined Software Services Each software service can have a number of specific rules that define what is to be monitored and what additional options are in effect. You can also assign each software service to existing or newly created tiers and applications. Before You Begin It is assumed that, for this task, you are already familiar with the concept of software services and that you know how to create and edit software services and how to open the Rules window. For more information, see Configuring User-Defined Software Services in the RUM Console Online Help. After a user-defined software service is created, create a group of settings that comprise the rules for the software service. It is necessary to specify, at minimum, the IP addresses and port numbers for the software service. To configure rules for a user-defined software service: 1. Open the Services tab. 2. On the Services tab, select or clear Enabled to activate or de-activate the service definition. 3. In Rule description, type a brief description to identify the rule. The description you enter is shown in the Rules table, in the column Rule Name. If no text is entered here, the IP address specified later is used as the description for this rule. 4. Right-click in the Services table and select Add or Open from the context menu. To quickly navigate to an entry in this table, click in the table and then type some or all of the IP address. Click the magnifying glass icon or press [Ctrl+F] to open a search box to limit the table view to only those rows that contain a match (in any column) to the search string. The Service Details window appears when adding or editing the rules. 5. In the Service Details window, in the IP address(es) fields, enter the server IP address, or enter a range of IP addresses to monitor more than one server. 6. In the Port(s) fields, enter the port number of the monitored service. You can provide a range of port numbers if such a range of ports is used in your environment. Some software services may be active on a number of predefined ports or may change ports dynamically. To allow for this, you can specify a range of ports. However, specifying more than one port for a service prevents the port number from being reported for that service. If you define more than one port for a particular service name and server IP address (by either specifying a range of ports or by creating two or more distinct rules for the same 64

65 Chapter 7 Configuring AMD to Monitor User-Defined Software Services service name and server IP address but with different port numbers), the AMD reports the port number for this service as 0, causing the port number to be ignored in traffic reports. NOTE You can define up to 5000 definitions containing a server and a port. Each association of a server and a port counts as a single definition. Specifying a range of ports counts as providing many individual definitions. On CAS, the number of processed server definitions is limited by the license. For more information, see Per-Measurement Licensing in the Data Center Real User Monitoring Administration Guide. Advanced Configuration 7. Optional: Select Client port(s) for reversed-direction protocols. This option applies only to protocols such as X-Window whose client-server meanings are reversed. If you are uncertain, leave this option cleared. 8. Optional: Select or enter a Group name Part of URL auto learning configuration. By default, the URL auto-learning mechanism stores the URLs from all the the servers defined in the software service in one pool. You can create separate pools within a single software service based on a number of servers. This way, you ensure the URLs monitored on a server with a lower traffic do not have to compete with URLs from a much larger server in terms of volume. You achieve this by assigning servers to groups within a single software service which translates to separate pools. To create a seperate pool for a group of server, keep them under a common group name of your choice. For more information, see Details of the URL Auto-Learning Algorithm in the Data Center Real User Monitoring Web Application Monitoring User Guide. NOTE It is important that grouping within the services definition is consistent. Defining services with that same IP address but different ports and assigning them to different groups results in the generation of redundant and irrelevant data. 9. Optional: Enter the main server IP address. If the monitored application runs on several servers that are linked together in a farm, you can monitor the farm as one virtual server. Type the IP address that you want to use as your main server IP address. 10. Optional: Enter the IP address of the server masking the addresses of monitored servers. If the servers you intend to monitor reside behind an appliance that masks and replaces the addresses of the target servers, you need to set NLB NAT masking IP address to the IP address of the masking server. Without doing so, the AMD will see two unidirectional conversations instead of one bi-directional conversation between the servers and appliance: The conversation between the client and server is observed and recorded (IP address A talking to IP address B) 65

66 Chapter 7 Configuring AMD to Monitor User-Defined Software Services When a response travels to the client, a different session (IP address C talking to IP address A) is recorded due to the server's IP address being replaced by the load balancer's IP address. Unless you account for this, CAS reports will return ambiguously granulated data. Using the NLB NAT masking IP address option will ensure that the AMD monitors contiguous conversations. 11. Optional: Map client IP to client group name. The mapping allows you to catalog and report traffic going to the same server IP and port by associating client group names with the originating client IP. On the report, the client group name is be reported as a suffix to the software service name. For example, a software service named SQL configured on a server located at can be configured the following way: Client IP Address Software Service name suffix _ATLANTA _BOSTON The system will differentiate the SQL software service traffic going to the server based on the client IP definition and report data for software service SQL_ATLANTA and SQL_BOSTON individually. The default configuration containing no client IP definitions results in an empty client group name. Similarly, an empty group name is used if a client IP is not included in any of the defined IP ranges. This configuration makes it possible to obtain only the client group name. The same client group name can be used in many client IP ranges. The configuration of each software service is individual per client group name. No cross-relations or cross-checks are performed between the definitions. It is possible to use a different name for the same client IP in each of the software services. 12. Click OK to confirm your changes and close the Service Details window. 13. Configure the settings on the available tabs. The number of available configuration options depends on the analyzer. See the analyzer-specific section for more information. 14. Optional: On the Options tab, define analyzer-specific options. The following list describes all possible options. Depending on the analyzer, some may be unavailable: Operation load time threshold An operation that takes more than this many seconds is considered slow. When Inherit from global setting is selected, the global setting is used. The global threshold value depends on the analyzer. Operation time threshold An operation that takes more than this many seconds is considered slow. When Inherit from global setting is selected, the global setting is used. To edit the global setting, 66

67 Chapter 7 Configuring AMD to Monitor User-Defined Software Services open the AMD configuration, select Global General and set the Operation time threshold. Server time threshold Server time threshold relates to the server time portion of an overall operation time. Server times above the threshold limit are considered to be slow due to the poor datacenter performance. When Inherit from global setting is selected, the global setting is used. To edit the global setting, open the AMD configuration, select Global General and set the Server time threshold. SQL query time threshold A database query that takes more than this many seconds is considered slow. When Inherit from global setting is selected, the global setting is used. To edit the global setting, open the AMD configuration, select Global Database Monitoring General and set the SQL query time threshold. Enable monitoring of persistent TCP sessions When this option is selected, the TCP sessions that do not start with SYN packets are monitored. By default, this option is selected. Persistent TCP sessions are TCP sessions for which the start was not recorded. They are also referred to as non-syn sessions. These sessions can be included in the TCP statistics, based on the configuration properties you enable in RUM Console. The inclusion of these sessions may render the statistics somewhat inaccurate and must be undertaken with care. Generate transactions and ADS data Select this option to provide the report server with, for example, raw HTTP traffic data enabling you to view the full HTTP request-response dialog. SQL Server uses dynamic ports This option only applies to the TDS analyzer. Select this option if the database engine you intend to monitor does not have a static port number assigned (for example, a named instance). In this case SQL Server Browser Service (SSBS) is used to discover the actual port of the service. The AMD uses additional UDP analysis of the SSBS to discover the port number for the service you intend to monitor. If you select this option make sure that the connection details specified on the Services tab identify the SQL Server Browser Service (use the IP address of the server and the port number of the SSBS). Do not enable this option if your SQL Server uses static ports. Convert the XML content URL-encoding This check box defines whether the XML URL-encoding content is enabled. When Inherit from global setting is selected, the global XML setting is used. URL parameter name that contains URL encoded XML document Provide the parameter name that contains a URL encoded XML document. If this field is empty, the AMD will not analyze XML documents sent in URL parameters. 15. Configure availability 67

68 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Select the Availability tab to configure the availability reporting at the software service level, overriding the global settings. The scope of failure reporting depends on an analyzer. For more information, see Configuring Availability in the Data Center Real User Monitoring Administration Guide. 16. Click OK. Excluding IP Ranges from AMD Client Analysis You can exclude particular client IP address ranges from AMD analysis. 1. Start and log on to RUM Console. 2. Select Devices and Connections Manage Devices from the top menu, to display the current device list. 3. Select Open Configuration from the context menu for an AMD. The AMD Configuration window appears. 4. Click Edit as Draft to set your configuration to draft mode (if you are not in draft mode already). 5. Select Global Advanced Excluded Client ranges. 6. Provide the start and end IP addresses for each range to exclude from AMD analysis. Be sure not to filter everything out or there will be no data in your reports. 7. On the Devices screen, click Publish Configuration. HTTP Express Analyzer The HTTP Express analyzer is a simplified version of the HTTP analyzer. Use this analyzer for network performance monitoring when you know that HTTP traffic is present and you require basic HTTP information about servers and URLs but not in-depth transactional or payload analysis. The HTTP Express analyzer supports the basic HTTP monitoring features, enabling you to create a simple software service used to monitor URLs. It provides basic HTTP analysis limited to the hit identification and per-url monitoring. Table 4. Comparison of the HTTP and HTTP Express Analyzers URL and URL with parameters monitoring. For more information, see Configuring URL Monitoring in the Data Center Real User Monitoring SAP Application Monitoring User Guide and Configuring URL Monitoring for HTTP Express Analyzer [p. 71]. URL auto-learning. HTTP Yes Yes HTTP Express Yes, with limitations 1 Yes 1 Many of the DC RUM monitoring features are available at the level of a software service, URL, and URL with parameters. All monitoring features not supported by the HTTP Express analyzer are naturally not available for URL and URL with parameters monitoring. 68

69 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Table 4. Comparison of the HTTP and HTTP Express Analyzers (continued) HTTP HTTP Express For more information, see URL Auto-Learning in the Data Center Real User Monitoring Web Application Monitoring User Guide. Recognition and parsing of URLs. For more information, see Global Settings for Recognition and Parsing of URLs in the Data Center Real User Monitoring Web Application Monitoring User Guide. Yes Yes Character encoding support. For more information, see Character Encoding Support for HTTP Services in the Data Center Real User Monitoring Web Application Monitoring User Guide. Content type monitoring. For more information, see Content Type URL Monitoring in the Data Center Real User Monitoring Web Application Monitoring User Guide and Monitoring of Non-HTML Objects Based on Content Type in the Data Center Real User Monitoring Web Application Monitoring User Guide. Extracting additional dimensions. For more information, see Extracting Grouping Attributes in the Data Center Real User Monitoring SAP Application Monitoring User Guide and Extracting Miscellaneous Parameters in the Data Center Real User Monitoring SAP Application Monitoring User Guide. Yes Yes Yes, in HTTP mode. Yes No No Operation attributes reporting. For more information, see Operation Attributes in HTTP Monitoring in the RUM Console Online Help. Custom metrics reporting. For more information, see Custom Metrics in HTTP Monitoring in the RUM Console Online Help. Yes Yes No No Page name recognition. For more information, see Automatic Page Name Recognition in the Data Center Real User Monitoring Web Application Monitoring User Guide. Yes. From responses in HTTP and HTTP legacy mode. From requests only in HTTP mode. No Defining end-of-page components. For more information, see End-of-Page Components in the Data Center Real User Monitoring Web Application Monitoring User Guide. Excluding elements from orphaned redirect reporting. Yes Yes, in HTTP mode. No No 69

70 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Table 4. Comparison of the HTTP and HTTP Express Analyzers (continued) HTTP HTTP Express For more information, see Excluding Elements from Orphaned Redirects Reporting in the Data Center Real User Monitoring Web Application Monitoring User Guide. User identification. For more information, see User Name Recognition Configuration in the Data Center Real User Monitoring Web Application Monitoring User Guide. Yes No Transaction reporting, including asynchronous HTTP transactions. For more information, see Logging Transactions, ADS Data and ADS Header Data in the Data Center Real User Monitoring Web Application Monitoring User Guide and Using Correlation ID to Monitor Asynchronous HTTP Transactions in the Data Center Real User Monitoring Web Application Monitoring User Guide. Yes No Browser, operating system, and hardware recognition. For more information, see Configuring Synthetic Agents, Browsers, Operating System and Hardware Recognition in the Data Center Real User Monitoring Web Application Monitoring User Guide. Assembling pages. Assembling Pages in the Data Center Real User Monitoring Web Application Monitoring User Guide Multi-frame pages reporting. For more information, see Multi-Frame Pages in the Data Center Real User Monitoring Web Application Monitoring User Guide. SSL monitoring. Yes Yes Yes Yes No No No No Within the features supported, the HTTP Express analyzer configuration are similar to the standard HTTP analyzer. Configuring User-Defined Software Services Based on HTTP Express Analyzer HTTP options for software services based on the HTTP Express analyzer can be set globally for the AMD or individually for particular software services. To modify the configuration options related to service specific settings for an individual HTTP software service based on the HTTP Express analyzer: 1. Start and log on to RUM Console. 2. From the top menu, select Software Services Manage Software Services. 3. Select a software service from the list. 70

71 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Click in the row corresponding with your service to display a set of rules for this service on the Configuration tab. 4. On the Configuration tab, select Edit manually from the Actions context menu for a selected rule. The Edit Rule pop-up window appears. In this window you can edit and delete the existing rules, or add new rules. 5. Configure URL monitoring. For more information, see Configuring URL Monitoring for HTTP Express Analyzer [p. 71]. 6. Optional: Configure URL parameters monitoring. For more information, see Configuring Monitoring of URL Parameters for HTTP Express Analyzer [p. 75]. 7. Optional: Configure URL Auto-Learning. For more information, see Configuring URL Auto-Learning in the Data Center Real User Monitoring Web Application Monitoring User Guide. 8. Optional: Configure Character Encoding Support. For more information, see Rule-based Character Encoding for HTTP Services in the Data Center Real User Monitoring Web Application Monitoring User Guide. 9. Optional: Switch to the HTTP Options tab. 10. Configure report URL prefixed with analyzed HTTP method. If this option is selected, the string POST or GET is prefixed to the reported URL. This option can be set globally for all software services or configured for a specific user-defined software service. Specific settings take precedence over global settings. 11. Optional: Select analyzed HTTP methods. Choose between Only POST and GET and All Methods. This option is configured individually for user-defined software services. 12. Optional: Switch to the Options tab. If you select Enable monitoring of persistent TCP sessions, TCP sessions not starting with SYN packets are monitored. 13. Publish the draft configuration on the monitoring device. Configuring URL Monitoring for HTTP Express Analyzer You can create named URL definitions to monitor specific URLs and you can specify URLs to be excluded from monitoring. You can also specify a virtual HTTP server to handle scenarios in which many web sites reside under a single IP address. Before You Begin It is assumed for this task that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the 71

72 Chapter 7 Configuring AMD to Monitor User-Defined Software Services IP address and port of the software service to be monitored. For more information, see Configuring User-Defined Software Services Based on HTTP Express Analyzer [p. 70]. To specify definitions for the monitoring of URLs for a user-defined HTTP software service, create or edit one or more URL definitions: 1. Open the URL Monitoring screen for the service. In the Rules table for the service, select the URL Monitoring tab. 2. Add or open a definition for a URL to be monitored or to be excluded from monitoring. In the URL Definitions table, right-click and choose Add Monitored URL to create a new definition for monitoring URLs, Add Excluded URL to create a new definition for URLs excluded from monitoring, or Open to open an existing definition. The Configure Monitored URL or Configure Excluded URL window will open. The order in which you arrange URLs is important. When adding several URLs of the same type, make sure that you arrange the definitions from the most specific to the most general, because the URLs are processed from top to bottom. In particular, if you add a specific excluded URL, make sure that you place it before a more general monitored URL, or the exclusion will be ignored. 3. Select a URL type. The option you select here determines the type of URL information that you will need to enter further down in the URL Definition section: Virtual HTTP Server Static URL Part URL as Regular Expression. 4. Enter a URL definition string. The information you enter here depends on the URL Type selection you made in the previous step. Virtual HTTP Server This option refers to monitoring a host where many web sites reside under a single IP address. Using a virtual HTTP server causes all reported pages that have no separate definitions to be aggregated to one record and reported together. This does not apply to those pages from the IP address that are defined separately in a monitoring configuration. Such individual definitions do not require that you select this option. A valid virtual HTTP server address to enter would be, for example, without a trailing slash. Static URL Part A fully qualified URL (one containing the protocol to be used, the server to be contacted, and the file to be requested) such as This URL will be added to the list of monitored URLs regardless of the limit of monitored URLs. 72

73 Chapter 7 Configuring AMD to Monitor User-Defined Software Services URL as Regular Expression An extended POSIX regular expression describing a set of URLs. For more information, see Regular Expression Fundamentals [p. 121]. The syntax allows you to use parentheses () to select one or more sub-expressions (specific portions of the results). If this mechanism is used, only the specified portions are reported; if more than one portion is specified, the portions are concatenated. NOTE When using a regular expression to specify a set of URLs to monitor: Explicitly include the string in the expression. You can not, for example, start the expression with.* and expect that the string will be assumed or resolved as a part of the regular expression. The parentheses you use to select the part of the URL to be extracted must include and the name of the host. However, the name of the host does not have to be provided explicitly, but can be resolved by the regular expression. Thus, for example, ( is correct, and so is ( The regular expression must be constructed such that, after extracting the portions delimited by parentheses, the resulting string does not end with a slash character ( / ). This rule applies to all URLs except home pages (URLs consisting only of a protocol specification and a host name). Such URL specifications should end with a slash. For example ( is valid, but ( is not valid. Note also that a specification ending with (myreport/*) is not valid because it can be matched by a string ending with a slash, as the asterisk can match an empty string. You can click the Test button located beside the regular expression pattern field to use the Regular Expressions Test tool to test patterns that will be used by the AMD. For more information, see Testing Regular Expressions [p. 123]. Example 1. A simple example of using a regular expression to specify monitored URLs The use of parentheses in a regular expression is demonstrated in the following example: ( The above expression will match URLs such as but only the bracketed portion ( ) will be reported. 73

74 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Example 2. A more complex example of using a regular expression to specify monitored URLs The following is a more complex example that demonstrates concatenation of bracketed portions: A site contains URLs of the form: 04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLN4o39w0BSYGYRiGBpFoYsamaEIG8Y4IEW99X4_83FT9AP2C3NDQiHJHRQDwwo2X/ delta/base64xml/l3djdyevuud3qndnqsevnelvrs82xzjfnvvn?wcm_global_context=/ assurance/wcm/connect/my Life.fr/Aide/Accueil Aide&WT.tz=1&WT.bh=12&WT.ul=en-us&WT.cd=32&WT.sr= 1400x1050&WT.jo=Yes&WT.ti=AssuranceRetraitePERP&WT.js=Yes&WT.jv=1.3&WT.fi= Yes&WT.fv=3.0&WT.sp=@@SPLITVALUE@@ where only the part coming after...wcm/connect/, in this case My Life.fr/Aide/Accueil Aide, is relevant for differentiating this page from other pages of this site, the rest being session ID and various parameters. If you use ( assurance/wcm/connect/([^&]*) to define monitored URLs, the reported URL for this page will be: life.fr/aide/accueil Aide NOTE Because resolving regular expressions is processor-intensive, defining a large number of URLs with regular expressions can have an adverse effect on the performance of the AMD. If you are configuring excluded URLs, this step completes this particular definition. If you are defining monitored URLs, proceed to the next step. 5. Optional: Select additional options. In the Options section, select or clear the desired options as required: Report URL Prefixed with Analyzed HTTP Method All methods of passing HTTP parameters can be distinguished if this option is selected. To use the value defined for the entire monitoring rule, ensure that the Inherit Setting from Rule check box is selected. The All methods option allows for processing all detected HTTP methods including the WebDAV HTTP extesion. The extended WebDAV methods automatically identified include: PROPFIND Retrieves properties and a directory hierarchy of a remote system. PROPPATCH Changes and deletes multiple properties is a single operation. MKCOL Creates directories or collections. COPY 74

75 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Copies a resource from one URI to another. MOVE Moves a resource from one URI to another. LOCK Puts a lock on a resource. UNLOCK NOTE Removes a lock from a resource. Monitoring WebDAV software services requires a specific configuration options. In order to properly report a hit as a separate operation, you must define a URL with regex matching all URLs ( and content types. Report long pages, incoming over many monitoring intervals This option allows for reporting so-called long pages (pages that load continually). This type of page is used, for example, to provide constantly updated information such as stock market reports. There are a number of different techniques for providing this functionality, such as by using streaming objects or server PUSH. All pages to be treated as long pages must be specified explicitly. Long pages are reported on reports, but no transaction-related information is included in reports. The only information collected for such pages are network metrics. Report Only URL Part When Parameters Do Not Match Select this option to cause this URL to be reported even if none of the parameter sets specified for the URL has been matched with the actual parameters seen in the monitored traffic. Parameters are defined in a separate configuration window. For more information, see Configuring Monitoring of URL Parameters for HTTP Express Analyzer [p. 75]. 6. Publish the draft configuration on the monitoring device. What to Do Next If you require URL recognition that includes parameter matching, you need to define parameter information for this URL definition. For more information, see Configuring Monitoring of URL Parameters for HTTP Express Analyzer [p. 75]. Configuring Monitoring of URL Parameters for HTTP Express Analyzer You can specify up to four parameters for a given URL definition using, among other ways, regular expressions. Pages with particular sets of parameters can be reported as separate pages in DC RUM reports. 75

76 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Before You Begin It is assumed for this task that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. It is also assumed that you have created one or more URL definitions for your rules. For more information, see Configuring User-Defined Software Services Based on HTTP Express Analyzer [p. 70].Configuring URL Monitoring for HTTP Express Analyzer [p. 71] To specify parameter definitions for a URL definition, create or edit one or more parameter definitions as follows: 1. Open the Rules Configuration window for the service. 2. Click the URL Monitoring tab. 3. In the URL Definitions section, select the desired URL definition. To quickly navigate to an entry in the URL definitions table, click in the table and then type some or all of the IP definition. Click the magnifying glass icon or press [Ctrl+F] to open a search box to limit the table view to only those rows that contain a match (in any column) to the search string. 4. In the URL Parameters table, right-click and choose Add to create a new parameter definition, or choose Open to open an existing definition. The URL Parameters window will open. 5. Select a parameter matching method from the Parameter Match list and specify details for up to four parameters. The following matching methods are supported: Exact Report the specified parameter or the parameter and value. Usage syntax 'name=value' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. So, the wildcard character * is taken literally. 76

77 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples You can specify 'john', to match though note that in this case will not be reported because the parameter value '=123' was not explicitly specified. To match it, you would need to specify 'john=123'. Start Report parameters that begin with a specified string; report only the matched pattern, truncate any remainder of the parameter. Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. If more than one option is selected, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples 'fred=5' will match but it will be reported as The value 'fred' will match 77

78 Chapter 7 Configuring AMD to Monitor User-Defined Software Services as well as and it will be reported as Start (expand) Report parameters which begin with a specified string; report the entire parameter, not only the matched pattern. End Usage syntax 'name=value' or any initial part of it this string, including string of the form 'name=' or just 'name'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples 'fred=5' will match and it will be reported as The value 'fred' will match as well as and it will be reported as and respectively. Report parameters which end with a specified string; report the entire parameter, not only the matched pattern. 78

79 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Usage syntax 'name=value' or any final part of it this string, including string of the form '=value' or just 'value'. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed; no wildcard characters are permitted in the string. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Examples For to be matched, you can specify the following ends: '0', '00', '100', '=100', 'n=100' and so on, up to 'john=100'. Thus is reported. Value RegEx Report parameters which begin with a specified string; optionally attempt to match the remainder of the parameter with a regular expression; report the start string and selected portions of the regular expression, if any. Usage syntax Parameter is entered as name=value or any initial part of it this string including string of the form name= or just name. A regular expression (regex) is entered as an extended POSIX regular expression. Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. 79

80 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Method of matching parameters Parameters are identified in the request URL, POST body, or HTTP header by searching for the appropriate separator characters, as defined for the analysis of HTTP. After individual parameters have been identified a match is attempted for each parameter. Limitations A case-insensitive match is performed on the Parameter part; the regex part is matched as a case-sensitive POSIX regular expression. Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples parameter specification fred= and a regular expression AB(C?E) will match but it will be reported as because the AB portion of the regular expression was not included in round braces. Custom RegEx Report parameters that match the given regular expression; report those portions that have been selected within the regular expression. Usage syntax Enter an extended POSIX regular expression to match the desired string. Mark portions to be reported by using round braces ( and ). Source By selecting the appropriate check box, you can cause the parameter to be searched for in the request URL, POST body, or HTTP header. Note that more than one option can be selected, and in such a case, the selected parameter sources are searched for in sequence, in the order specified for HTTP analysis, until the first match is found. Method of matching parameters The request URL, POST body, or HTTP header are not split into parameters prior to pattern matching. Instead, they are treated as single units of data and the regular expression is applied to their entire contents. Only the path part of the request URL is excluded from the matching process. Limitations The regular expression is entered according to POSIX syntax. 80

81 Chapter 7 Configuring AMD to Monitor User-Defined Software Services Combining parameters If you have defined more than one parameter for a given URL, for a match to be successful all specified parameters have to be matched. When all matches are found, the reported string then contains a concatenation of all the matched parameters, separated by the ampersand & character. Note that for a single URL, different parameters can be extracted from different portion of the HTTP packet, request URL, POST body, or HTTP header, and combined into a single match. Decoding and decompression The string to match the regular expression is first optionally decoded and decompressed, if the appropriate encoding and compression is selected. Examples Regular expression fred=ab(c?e) will match but it will be reported as Regular expression (.*=)AB(C?E) will match as well as and it will be reported as and as respectively. 6. Publish the draft configuration on the monitoring device. Managing User-Defined Software Services As a console user you have the ability to add, remove, and edit the properties of any software service defined on any of the AMDs. You can view all user-defined software services on the Software Services screen in the RUM Console. To access this screen, select Software Services Manage Software Services from the top menu. This screen contains information about the user-defined software services that were created on all devices managed from this console. You can add new services, delete existing ones, and copy them to other AMDs. Note that the default software services are specific to a single AMD and cannot be managed centrally. To view user-defined software services monitored by a selected device, access the configuration for this device. Select Devices and Connections Manage devices from the top menu, a list of all devices managed by the console appears. Next, select Open configuration from the context menu for the device. Assigning Software Services to Devices Software services mirrored across different AMDs (that is, having the same name and identical rules) are grouped together. The service name is the name for the group. Whenever you change rules on any of the AMDs in the group, the software services are separated into single entries. After a software service is created on one of the AMDs in your network, you can copy it to another device. To do so, on the Software Services screen, select Copy from the Actions 81

82 Chapter 7 Configuring AMD to Monitor User-Defined Software Services context menu for a given software service. A dialog box appears where you can choose the AMDs by selecting the check box next to their IP addresses. After you click OK, the software service definitions are copied to the selected AMDs. On the Software Services screen, select the Deployment tab and click Change Assignment to modify the list of devices that are monitoring a given software service. 82

83 CHAPTER 8 Network Tiers If you have configured sites on the CAS or if sites have been configured automatically, the Client network and Network tiers will automatically be displayed on the Tiers report. By default, the Client network tier shows traffic for all sites except the All other site and the Network tier shows traffic for the All other site only. Figure 13. Client Traffic and Data Center Traffic Site 1 Data center Site 2 Site 3 Client network (WAN) Network (LAN) If you suspect that the All other site may include client traffic, you should modify the default configuration of the Client network and Network tiers. For more information, see Modifying a Network Tier [p. 84]. When the All Other Site Includes Client Traffic The following are examples of when the All other site may include client traffic: 1. The assignment of clients to sites is imprecise. This may happen when you are not sure whether all client locations were assigned to the appropriate sites. 83

84 Chapter 8 Network Tiers 2. Sites were configured automatically based on either AS names or CIDR blocks, where a block is defined by an IP address and a network mask. In either case, modify the Client network and Network tier definitions. Follow these steps to configure the network tiers so that they are used as designed: 1. Create a new Data center site and specify a set of IP addresses or a range of IP addresses corresponding to your data center. 2. Modify a Client network tier definition. a. Remove the rule that excludes the All other site. b. Add a rule that excludes the Data center site. 3. Modify a Network tier definition. a. Remove the rule that includes the All other site. b. Add a rule that includes the Data center site. The Network tier now shows the data center traffic and the Client network tier now shows the traffic for the client sites. For more information, see Modifying a Network Tier [p. 84]. Modifying a Network Tier To modify a network tier: 1. Select a tier in the Network tiers section. 2. In the Rules for this tier section: Click Add rule to define a new rule. For more information, see Adding a Rule Based on Sites [p. 84]. Select a rule and change its definition by selecting other region, area, or site in the designated fields. Select a rule and click Delete rule to delete that rule from the tier definition. Adding a Rule Based on Sites In the Rules for this tier section: 1. Select Client network or Network tier. 2. Select a region, area, or site to add it to the rule. Select Exclude to exclude a particular region, area, or site from a tier definition. 84

85 CHAPTER 9 Configuring Sites, Areas, and Regions The monitored traffic information can be organized in the way the report server sees traffic via AMDs. The hierarchy of sites, areas, and regions provides an organized view of the monitored network. A site is a term for a group of users that are located in the same IP network or group of networks sharing similar routing properties. Sites can be grouped together into areas, which in turn can be grouped together into regions. Use site definitions to help you identify IP networks. The report server enables you to use two types of sites: Manual sites Specified as explicit IP address ranges Support both IPv4 and IPv6 addresses Configured through the Site Configuration screen Automatic sites Public Internet sites based on either an IP address and a network mask or on information retrieved from the BGP routing table and AS (autonomus system) name to number mapping in the bgp.zip and asn.zip files Support only IPv4 format Configured directly in the report server database All server IP addresses that are not covered by any of the manual or automatic sites are treated as belonging to the Default Data Center. In addition, there are external sites, which are specified based on the same type of definitions as imported manual sites, but are imported from external text files rather than defined using the report server GUI. Automatically detected sites, such as sites originating from Enterprise Synthetic, can be viewed by clicking Show Detected. All of the sites listed in the column Type as Active originated from Enterprise Synthetic data. Once modified, the site type becomes Manual (user-defined). 85

86 Chapter 9 Configuring Sites, Areas, and Regions NOTE Sites cannot overlap. The report server will not allow a situation where there would be addresses that match the definitions of two different sites. Adding Sites Manually Creating site definitions manually enables you to monitor specific networks that use both IPv4 and IPv6 addresses. To add a new site: 1. Open the site definition screen From the RUM Console top menu, choose Devices and Connections Manage Devices, then choose Open configuration from the context menu for your server on the Devices screen. Finally, on the Server Configuration screen, choose Sites from the menu. NOTE All manual sites defined through external files are of type External. This type of site cannot be removed or modified through the RUM Console user interface. 2. Optional: Add Areas and Regions. The sites are geographicaly situated using the Region -> Area -> Site hierarchy. Before adding a site, you need to create a region and area the site belongs to. Type in the region name, and click Add. The region name appears in the table. Click the name to move to the Areas tab. Do the same for the area, completing the process on the Sites tab. 3. Add Site. When in the Sites, type in the site name and click Add. Site names can contain spaces but cannot contain special characters such as & or *. 4. Optional: Select region and Area. You can select a region and area from the list. 5. Optional: Select the User Defined Link check box. This check box enables you to select a manually defined site as a UDL, which will make it appear on the CAS Link View reports. This is helpful if a particular link cannot be automatically detected by the monitoring device. NOTE For sites that are not defined manually, this option will not be available from the server GUI. However, you can configure any site as a user-defined link through external files. For more information, see Formatting the Site, Area, and Region Definitions [p. 90]. A UDL will appear in the Link View reports as soon as the first monitoring interval data collection is ready after the UDL has been defined. The report does not show any historical data. 86

87 Chapter 9 Configuring Sites, Areas, and Regions If you change the configuration and clear User Defined Link for a selected site, the system will stop collecting information for that link but the UDL will still display in the Link View reports to allow access to the historical data already stored in the report server database. 6. Optional: Select the WAN Optimized Link check box. This check box enables you to select a manually defined site as a WAN optimized link. This option is not available if WAN optimized links are discovered automatically as controlled by the RTM_AUTO_WANOPT_DISCOVERY property in the Advanced Properties Editor. You can turn off the automatic detection of WAN optimized links if a particular link cannot be automatically detected as WAN optimized by the monitoring device. The WAN optimized links information is available as Client site WAN optimized and Server site WAN optimized dimensions with the plain text value Yes or No on the DMI Software service, operation, and site data and Software service, operation, and site data baselines data views. 7. Optional: Specify the User Defined Link speeds. Specify the incoming and outgoing link speeds in bits per second. 8. Specify a set of IP addresses or a range of IP addresses in the IP Addresses field. Make sure to specify each IP address or IP address range on a separate line. To define a range of IP addresses, you have to specify the starting and ending addresses in the following way: An IP address range has to be continuous. You can specify IP addresses both in IPv4 and IPv6 format and mix them within the same site. For example: :db8::4ab5 Note that you can type an IPv6 address in any of the following ways (they are equivalent): 2001:0db8:0000:0000:0000:0000:1622:35bd 2001:0db8:0:0:0:0:1622:35bd 2001:db8::1622:35bd NOTE Note that this mode of specifying IP addresses does not allow you to enter an IP address followed by a mask, as in Site_name = IP_address / IP_mask. However, using masks to specify sites is permitted when importing site definitions using the external site definition files or text fragments pasted in the site import window. For more information, see Defining Sites, Areas, and Regions Using External Text Files [p. 88], Importing Site Definitions [p. 88], and Formatting the Site, Area, and Region Definitions [p. 90]. 9. Optional: Add a comment in the Comment field. For each site, in addition to the site name, you can add a comment that is displayed on the list of sites. 87

88 Chapter 9 Configuring Sites, Areas, and Regions 10. Click OK. Defining Sites, Areas, and Regions Using External Text Files The report server enables you to use external text files to specify the site, area, and region definitions. The files are read (imported) by the reporting server on server startup and at every monitoring interval and the site information is analyzed and stored in the report server configuration. The configuration is non-volatile (preserved between server restarts). Changes to the configuration can be made by supplying new site definition files. NOTE All manual sites defined through external files are of type External. This type of site cannot be removed or modified through the RUM Console user interface. The text files containing site definitions must have names matching the following pattern: locations-*.config and must be placed in the <install_dir>\config directory. In the definition files, the lines that define the sites must be structured according to a specific syntax. For more information, see Formatting the Site, Area, and Region Definitions [p. 90]. Importing Site Definitions Manual site definitions can be specified in the form of text fragments containing already formatted site definitions, according to the required syntax. For more information, see Formatting the Site, Area, and Region Definitions [p. 90]. Access the Site Configuration Import screen by doing the following: On the report server, open the Diagnostic Console by typing diagconsole in the browser Address field: Select Import Sites Hierarchy. Note that administrative rights are required to configure the site definitions. On the Site Configuration Import screen, paste the site definition text in the input window. The syntax of the imported site definitions is verified before they are added to the server configuration and any invalid definitions are rejected. To import data, enter the site definitions in the edit box in the following format: Location_name=IP_address1-IP_address2 Location_name.area=Area_name Location_name.region=Region_name The following example defines the site New England in the East Coast area of the USA region: New England= New England.area=East Coast New England.region=USA 88

89 Chapter 9 Configuring Sites, Areas, and Regions Importing the sites configuration using the RUM Console You can also import the sites configuration, you previously exported in the RUM Console using the CSV format. For more information, see Importing and Exporting Site Definitions in RUM Console [p. 93]. Automatic Creation of Sites from Enterprise Synthetic The data coming from Enterprise Synthetic to CAS may contain the agent name or site name or both. In each case, the process of the automatic site creation is slightly different. Site only If the data contains information on an active site name, a site with the same name is created in CAS and an agent is assigned to this site. However, exceptions are possible: If the information for an active site name is provided, but the agent IP address already belongs to a non-active site, a new site will not be created and the agent will belong to the existing site. If the active data contains the information for a site name, but the agent is already assigned to an active site, it will be removed from it and assigned to a new one. This means that the agent can change sites provided both the old and the new site are active type. Agent only If only the agent name is provided, three scenarios are possible before a site is created: The agent name is first mapped to a site, area, or region name using the locations-*.config files in the config folder of the report server. NOTE The locations-sample.config file is a sample source of mapping information; other files named using the locations-*.config pattern can be specified in the report server configuration. If no mapping information for the agent is available in the locations-*.config files, the agent is assigned to the Default site. If the agent IP number is already assigned to an existing site, the mapping information from the locations-*.config files will not be taken into account and the agent will be assigned to the detected site. To view the automatically detected sites from Enterprise Synthetic, choose Settings Report Settings Sites and click Show detected in the bottom-right part of the Site Configuration screen. All detected sites from Active data are marked Active in the Type column. The information about detected sites is automatically updated when new data arrives, ensuring that new users detected in a specific site appear on the CAS reports. However, if you modify the site definition by, for example, changing its name, the sites become Manual (user-defined) and no updates are performed. 89

90 Chapter 9 Configuring Sites, Areas, and Regions Synchronizing the Site Definitions with Business Service Management CAS supports a three-level site hierarchy, with a single site at the basic level, then an area, and then a region at the top level. In Enterprise Synthetic there is no such hierarchy, so when CAS receives active data, the site names included with the active data have to be mapped to sites, areas, and regions in CAS. Automatic mapping depends on the information that Business Service Management delivers about the site synchronization level and also on the configuration settings in CAS. The synchronization level always allows mapping of sites, but may not include areas and regions. If it does include them and the CAS allows synchronization at a level other than site level, the agent site names are also automatically mapped to areas and regions. In practice, this means that a site with a name corresponding with the active site name is always created. If the synchronization level in Business Service Management includes areas, an area with the same name is also created. Otherwise, the site is assigned to the Default area. If the synchronization level in Business Service Management is set to regions, an area and a region with the same name will be created. If not, the site will be assigned to the Default region. The default synchronization level is the area level. To check whether the current CAS configuration enables automatic creation of regions and areas based on data from Enterprise Synthetic: 1. Open Control Panel by typing into the browser's Address field. 2. In the Control Panel, select Advanced Properties Editor. 3. Find the VSM_USE_SYNC_LEVEL property and ensure its value is ON. Formatting the Site, Area, and Region Definitions Specifying Sites Ensure that the lines in the external site definition files and the text fragments pasted in the site import window are structured according to the syntax shown below. Note the use of hash characters (#) to delimit the comments: Explicit Definition of a Site Site_name=IP_address for example # define location Mylocation Mylocation = Specifying an Address Range Site_name = IP_address_1 - IP_ address_2 for example # Define a location as a range of addresses Mylocation = Specifying an IP Address and Mask Site_name = IP_address / IP_mask 90

91 Chapter 9 Configuring Sites, Areas, and Regions for example # Define a location as a IP address and mask Mylocation = /24 Sites cannot overlap. Ensure that there are no addresses that match the definitions of two different sites. For example, do not specify two different address ranges for two different sites that include a common subrange, or a mask that defines addresses that are already defined by a different mask or address range. External files containing overlapping site definitions will not be accepted by the report server. If your external definitions conflict with the definitions that already exist (if the name of a site in an external file duplicates that of a site already defined in the report server configuration), the following rules are applied: If a site defined in an external file already exists in the report server configuration database and it is of a different type (if it is not a definition that came from an external file), the external definition is ignored and an error message is written in the log. If a site defined in an external file already exists in the report server configuration database and it is the same type (if it is a definition that came from an earlier application of an external file), the definition of the site is updated according to the new external definition. Also note that to remove the definition of an external site from the report server configuration, omit this site from the next set of external files. If a site in the report server configuration database comes from an external file and no such definition is found in the new, currently read set of external files, the definition is removed from the database. There are no changes on reports for updated/added sites until new traffic has come for these sites. Assigning Areas and Regions Sites can be assigned to areas and regions in the following way: Site_name.area = Area_name Site_name.region= Region_name Example Mylocation.area = Myarea Mylocation.region = Myregion NOTE If the area or region name used in the assignment does not exist, it will be created. Assignment of an area to a region can be done explicitly with the syntax: Area_name.region = Region_name Example: Myarea.region = Myregion or it can be done by assigning one of the sites from the given area to the appropriate region. This will cause the area to be assigned to the same region. 91

92 Chapter 9 Configuring Sites, Areas, and Regions Adding a Comment to Site Site_name.comment = comment Example Mylocation.comment=Location sample Specifying User Defined Links Site_name.UDL = true or Site_name.UDL = false Example: Mylocation.UDL=true Specifying UDL Speeds Specify the speeds in bits per second. Site_name.speed_in = incoming_speed Site_name.speed_out = outgoing_speed Example: Mylocation.speed_in= Mylocation.speed_out= Specifying a WAN Optimized Link Site_name.WAN = true Example Mylocation.WAN=true Site Sample Definitions # Defining a location using an IP address Loc_1= # Defining a location using an IP address range Loc_1= # Defining a location using an IP address and mask Loc_1= /24 # Note: For a single location, multiple IP # ranges/addresses/masks can be defined # A location can be assigned to an area Loc_1.area = Area_1 # A location can be assigned to a region # Note: in this case area assigned to this location is # also assigned to the given region assigned Loc_1.region = Region_1 # An area can also be assigned to a region explicitly Area_1.region = Region_1 # Location can be defined as UDL - user defined link. This option is used by CAS only. Loc_1.UDL=true 92

93 Chapter 9 Configuring Sites, Areas, and Regions # Location is defined as WAN Optimized Link. Loc_1.WAN=true Importing and Exporting Site Definitions in RUM Console To export or import the RUM Console managed site definitions to a CSV text file, open the RUM Console Site Configuration screen. After you log on to the console, choose Devices and Connections Manage Devices, and, on the Devices screen, choose Open configuration for the selected server. The Server Configuration screen appears. To access site configuration, choose Sites from the menu. To export the sites configuration, click Export to CSV. To import, click Import from CSV Use the CSV file import and export in RUM Console only. The CAS managed property, YAML and JSON files cannot be used in RUM Console. For more information, see Importing Site Definitions [p. 88] and Exporting Site Definitions in Diagnostic Console [p. 93]. Exporting Site Definitions in Diagnostic Console To export the current site definitions to a text file, all or only manual, open the CAS Diagnostic Console screen. On the report server, open the Diagnostic Console by typing diagconsole in the browser Address field: Select Export Sites. You can choose among the JSON, YAML and properties format. The properties format file contains only the manual defitnitions and is compliant with the locations-sample.config file stored on the report server in the config folder. For more information, see Formatting the Site, Area, and Region Definitions [p. 90]. Exportring the sites configuration using the RUM Console You can also export the sites configuration, you previously exported in the RUM Console using the CSV format. For more information, see Importing and Exporting Site Definitions in RUM Console [p. 93]. 93

94 Chapter 9 Configuring Sites, Areas, and Regions 94

95 CHAPTER 10 Network Analysis Reports The reports grouped in the Reports Network Analysis menu provide a network view of the traffic. They show a picture of the monitored network operation and highlight potential problems in the network, including excessive RTT or loss rate. The Network Analysis reports show metrics for all the detected or defined regions, areas, and sites and for all analyzed software services. They help you to quickly troubleshoot software service or network performance problems by answering these questions: Who (which clients and servers) is sending what data? Which software services are competing for network resources? When was the software service used? Where is the troublesome software service traffic flowing through the network? Why is the software service performing poorly? How much data (bytes) is being sent/received? A healthy network is necessary as a platform for the IT applications that support key business services. The Network Analysis reports deliver metrics on the performance levels of various software services across the network. They help you find and eliminate performance problems before they affect end users. They help you to identify the business impact of the performance problem and to plan for the growth of network infrastructure by showing network performance trends for critical software services with respect to traffic load and loss rate baselines. Example Network Analysis Reports Usage The following is an example of how the Network Analysis reports can help you find the cause of network-related problems. This example shows you connections between particular reports and briefly explain the aim of those reports. Example: Network Anomaly for a Specific Day and Time Start from the Total Traffic by Hour - Today report. 1. Click Reports Network Analysis Traffic Summary and then click the Total Traffic by Hour - Today link. 95

96 Chapter 10 Network Analysis Reports The report shows today's network traffic. Notice that the loss rate was acceptable at 08:00, but changed to red (warning) at 09:00. Click the 6/2/14 09:00 time stamp. 2. On the Traffic Fault Domain Isolation - Last Hour report, note the usage-related metrics for software services, sites, servers, and clients. One client generated much more traffic than others, so click the client IP address in the Top Client Traffic table to see which software services this client was using. 96

97 Chapter 10 Network Analysis Reports 3. The selected client was using five different software services and was talking mostly to one server: the server that had the highest value of all transmitted bytes on the Traffic Fault Domain Isolation - Last Hour report. Now you are aware of the client activity and you can do something about it to avoid such situations in the future. You can also display clients for a specific software service, site, or server to investigate why the total bytes value is outside the normal range. Network Overview - Sites Report The Network Overview - Sites report shows an overall view and heath of sites and software services from the network perspective. If you select a table row with a specific site name, the report shows contextual data for that site. How to Access the Report You can access the Network Overview - Sites report from the Reports Network Analysis menu or from the Application Health Status report. To access it from the Application Health Status report, click the number of applications and display network details for the selected application. Report Contents and Usage Use the Network Overview - Sites report to identify the root cause of network problems. This report shows which software services are competing for network resources and helps you identify the bandwidth usage anomalies. For example, a high RTT can indicate network routing issues; a low bandwidth usage and a high loss rate in one direction can indicate the quality of service (QoS) issues. The first table lists local sites and shows whether they are configured with WAN optimization. The graph shows bandwidth usage for the selected site. The second table lists software services in the context of the selected site. The full set of all the available statistics is provided on the Internetwork traffic data data view. For more information, see Internetwork traffic data in the Data Center Real User Monitoring Central Analysis Server User Guide. 97

98 Chapter 10 Network Analysis Reports Drilldown Reports You can access more detailed reports from the following columns: Local site Network Status - Software Services View report. For more information, see Network Status - Software Services View Report [p. 98]. Capture packets dialog box. For more information, see Starting a Packet Capture in the Data Center Real User Monitoring Smart Packet Capture User Guide. Network View - Users report. Optimized For more information, see Network View - Hosts and Users Reports [p. 108]. Optimized WAN Environment Performance Overview - Sites report. For more information, see Optimized WAN Environment Performance Overview - Sites Report in the Data Center Real User Monitoring Central Analysis Server User Guide. Software service Voice and Video Status - Software Services (Codecs) report (for VoIP traffic). For more information, see Voice and Video Status - Software Services (Codecs) Report in the Data Center Real User Monitoring Central Analysis Server User Guide. Network Status - Software Services View Report The Network Status - Software Services View report displays metrics for all detected sites and all analyzed software services from the software services point of view. Metrics presented on this report are specific to the AMD, but performance data can come from AMDs and Flow Collectors. How to Access the Report From the CAS top menu, choose Reports Network Analysis Traffic Summary and click the Software Services tab. Report Contents and Usage With this report, you can identify software services experiencing performance problems and the number of users affected by those problems. By default, the time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. Metrics are grouped into three sections: Usage, Performance, and Network. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. 98

99 Chapter 10 Network Analysis Reports Drilldown Reports You can access the following reports by clicking links in the report table: Network Status - Servers View, from the Software service column. For more information, see Network Status - Servers View Report [p. 99]. Network View - Users, from the Software service or Unique users column. For more information, see Network View - Hosts and Users Reports [p. 108]. Software Services View - Charts - Network, from the Software service column. For more information, see Software Services View - Charts - Network Report [p. 105]. Network Status - Servers View Report Depending on the way you access this report, it shows measurements either for all servers monitored in all defined sites or for servers filtered for the selected software service. Metrics presented on this report are specific to the AMD, but performance data can come from AMDs and Flow Collectors. How to Access the Report From the CAS top menu: Choose Reports Network Analysis Traffic Summary and click the Servers tab. Choose Reports Network Analysis Traffic Summary, click the Software Services tab, and then click a software service name. Report Contents and Usage With this report, you can identify servers that are experiencing network performance problems and the number of users affected by those problems. By default, the time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. Metrics are grouped into three sections: Usage, Network, and Availability. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown Reports You can access the following reports by clicking links in the report table: Network View - Hosts, from the Name column. For more information, see Network View - Hosts and Users Reports [p. 108]. Software Services View - Charts - Performance, from the Name column. For more information, see Software Services View - Charts - Performance Report [p. 106]. Software Services View - Charts - Network, from the Name column. For more information, see Software Services View - Charts - Network Report [p. 105]. 99

100 Chapter 10 Network Analysis Reports Network View - Users, from the Unique users column. For more information, see Network View - Hosts and Users Reports [p. 108]. Software Services View - Charts - Clients, from the Affected users columns. For more information, see Software Services View - Charts - Clients Report [p. 107]. Top Traffic Summary - Today Report The Top Traffic Summary - Today report shows a traffic summary for the top five software services, sites, clients, servers, time intervals, and web pages. It enables you to analyze the network traffic from different perspectives in one screen. How to Access the Report From the CAS top menu, choose Reports Network Analysis Traffic Summary. Report Contents and Usage Using this report, you can identify software services, sites, or clients that generate the most traffic. You can drill down to a report that shows details (for example, the top clients by total bytes for the selected site) for a selected entity. This information can help you identify the performance bottleneck. By default, the following charts are displayed: Top Software Services by Traffic Top Sites by Traffic Top Clients by Traffic Top Servers by Traffic Top Time Intervals by Traffic Top Web Pages The time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown Reports You can access the following reports by clicking bars on the charts: Top Software Services by Traffic -Today, from the Top Software Services by Total Bytes chart. For more information, see Top Software Services by Traffic - Today Report [p. 101]. Top Clients by Traffic - Last Hour, from the Top Sites by Traffic, Top Clients by Traffic, and Top Servers by Traffic charts. For more information, see Top Clients by Traffic - Last Hour Report [p. 104]. 100

101 Chapter 10 Network Analysis Reports Traffic Fault Domain Isolation - Last Hour, from the Top Time Intervals by Traffic chart. For more information, see Traffic Fault Domain Isolation - Last Hour Report [p. 103]. Total Traffic by Hour - Today Report The Total Traffic by Hour - Today report compares today's network traffic with baseline values. When a network anomaly occurs, you can drill down to a report that shows the details for the selected time frame. How to Access the Report From the CAS top menu, choose Reports Network Analysis Traffic Summary and click the Total Traffic by Hour - Today link. Report Contents and Usage The Total Traffic by Hour - Today report shows how the network traffic changes over time, and highlights deviations from the baseline. Data is presented both on a chart and in a table. In addition to numeric data, the tabular report uses cell background colors (red, orange, yellow, and green) for all metrics to indicate different levels of problem severity. By default, the time range for the report is Today (the last calendar day for which data is available). The resolution is set to 1 hour. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown Reports Click the metric value in the Time column to access the Traffic Fault Domain Isolation - Last Hour report filtered for the selected time frame. For more information, see Traffic Fault Domain Isolation - Last Hour Report [p. 103]. Top Software Services by Traffic - Today Report The Top Software Services by Traffic - Today report helps you identify how the number of all transmitted bytes changes for the top ten software services over one day. How to Access the Report From the CAS top menu, choose Reports Network Analysis Traffic Summary and click the Top Software Services by Traffic - Today link. You can also access this report by drilling down from the Top Traffic Summary - Today report. For more information, see Top Traffic Summary - Today Report [p. 100]. Report Contents and Usage The Top Software Services by Traffic - Today report helps you analyze the number of all transmitted bytes for the top ten software services. Data is presented both on a chart and in a 101

102 Chapter 10 Network Analysis Reports table. The tabular report shows more detailed information and enables you to see how the total bytes metric was changing over one day with a five-minute time interval. By default, the time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown Reports Click the link in a chart tooltip or any value in the table to access the Traffic Fault Domain Isolation - Last Hour report filtered for the selected time frame. For more information, see Traffic Fault Domain Isolation - Last Hour Report [p. 103]. Total Traffic by Client Site - Today Report The Total Traffic by Client Site - Today report displays network statistics for the top 30 sites for today. It helps you identify client sites for which the RTT or loss rate is outside the norm. How to Access the Report From the CAS top menu, choose Reports Network Analysis Traffic Summary and click the Total Traffic by Client Site - Today tab. Report Contents and Usage The Total Traffic by Client Site - Today report helps you analyze performance problems for client sites. Data is presented both on a chart and in a table. In addition to numeric data, the tabular report uses cell background colors (red, orange, yellow, and green) for the client RTT, server RTT, client loss rate, and server loss rate metrics to indicate different levels of problem severity. By default, the time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown reports Click the link in a chart tooltip or the client site name in the Client site column to access the Top Clients by Traffic - Last Hour report filtered for the selected client site. For more information, see Top Clients by Traffic - Last Hour Report [p. 104]. Traffic Analysis - Last 30 Days Report The Traffic Analysis - Last 30 Days report shows how the number of all transmitted bytes changes over 30 days for the top software services, servers, and sites. 102

103 Chapter 10 Network Analysis Reports How to Access the Report From the CAS top menu, choose Reports Network Analysis Traffic Summary and click the Traffic Analysis - Last 30 Days tab. Report Contents and Usage This report enables you to see when and for which software service, server, or site the number of all transmitted bytes is the highest; and to identify the anomalies for the total bytes metric. By default, the following charts are displayed: Total Traffic - Last 30 Days Top Software Service Traffic - Last 30 Days Top Server Traffic - Last 30 Days Top Site Traffic - Last 30 Days The time range for the report is Last 30 days and the resolution is set to 1 day. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown Reports Click the link in a chart tooltip to access the Traffic Analysis Details - Last 30 Days report filtered forthe selected entity and time interval. For more information, see Traffic Analysis Details - Last 30 Days Report [p. 104]. Traffic Fault Domain Isolation - Last Hour Report The Traffic Fault Domain Isolation - Last Hour report focuses on top software services, top sites, top servers, and top clients for the time frame in question. In this report, you can easily see which software services, sites, servers, or clients were outside the norm during that time. How to Access the Report To access this report, drill down from the Top Traffic Summary - Today, Total Traffic By Hour - Today, or Top Software Services by Traffic - Today report. Report Contents and Usage The Traffic Fault Domain Isolation - Last Hour report highlights the total traffic changes from the baseline for top software services, sites, servers, and clients. The Total bytes column is automatically color-coded from baseline data to call attention to those areas where the traffic has changed. By default, the time range for the report is Last one hour and the resolution is set to 1 period. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. 103

104 Chapter 10 Network Analysis Reports Drilldown Reports Click the entity name in the Software service, Client site, Server IP address, Server name, or Client IP address column to access the Top Clients by Traffic - Last Hour report filtered for the selected time frame. For more information, see Top Clients by Traffic - Last Hour Report [p. 104]. Top Clients by Traffic - Last Hour Report The Top Clients by Traffic - Last Hour report is a drilldown report that shows the top users filtered by time frame and the area of concern. It enables you to quickly identify users that generate the most traffic. How to Access the Report To access this report, drill down from the Top Traffic Summary - Today, Total Traffic by Client Site - Today, or Traffic Fault Domain Isolation - Last Hour report. Report Contents and Usage The Top Clients by Traffic - Last Hour report helps you analyze performance problems for software services, servers, sites, or end users. You can use this report to display the top clients that use a specific software service and to see which servers they were talking to; or to see which users generate the most traffic in a specific site. By default, the time range for the report is Last one hour and the resolution is set to 1 period. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Capturing Packets Click the client IP address to open the Capture packets dialog box. For more information, see Starting a Packet Capture in the Data Center Real User Monitoring Smart Packet Capture User Guide. Traffic Analysis Details - Last 30 Days Report The Traffic Analysis Details - Last 30 Days report is a drilldown report that shows measurements for an entity selected on the Traffic Analysis - Last 30 Days report. It can be filtered for a selected software service, server, or site. How to Access the Report To access this report, drill down from the Traffic Analysis - Last 30 Days report. For more information, see Traffic Analysis - Last 30 Days Report [p. 102]. 104

105 Chapter 10 Network Analysis Reports Report Contents and Usage The Traffic Analysis Details - Last 30 Days report does not show temporary data, but it helps you identify the dominant users or servers of particular software services from a 30-day perspective. By default, the time range for the report is Last 30 days and the resolution is set to 1 day. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Software Services by RTT Report The Software Services by RTT report is a drilldown report that shows details for the Server RTT and Client RTT metrics. It helps you identify software services that have the worst server RTT value and enables you to analyze the relationship among the RTT value, refused connections, and bandwidth usage for a particular software service. How to Access the Report To access this report, drill down from the Top Traffic Summary - Today report and click the Software Services by RTT link. For more information, see Top Traffic Summary - Today Report [p. 100]. Report Contents and Usage This report shows the condition of software services that were affected by a high local RTT value. If your AMDs are located near the servers (this is a standard AMD deployment) and the server response takes too long, you should investigate whether there is a problem with the server or with the interior routing. By default, the time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The maximum set of provided statistics includes all the metrics available on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Software Services View - Charts - Network Report The Software Services View - Charts - Network report displays performance charts that shows how the total bandwidth usage, network performance, or server loss rate changes over time. These charts are filtered for a selected software service or server. How to Access the Report To access this report, open the Network Status - Software Services View or Network Status - Servers View report and click a drilldown link in the Software service or Server name column. For more information, see Network Status - Software Services View Report [p. 98] and Network Status - Servers View Report [p. 99]. 105

106 Chapter 10 Network Analysis Reports Report Contents and Usage By default, the following charts are displayed: Total bandwidth usage Network performance End-to-end RTT Server loss rate Server realized bandwidth These charts enable you to identify when the total bandwidth usage was the highest or how the network performance and server loss rate change over time. The time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Software Services View - Charts - Performance Report The Software Services View - Charts - Performance report displays performance charts that show how the number of operations, application performance, and operation time change over time. These charts are filtered for a selected software service or server. How to Access the Report To access this report, open the Network Status - Servers View report and click a drilldown link in the Server name column. For more information, see Network Status - Servers View Report [p. 99]. Report Contents and Usage By default, the following charts are displayed: Operations/min Application performance Operation time These charts enable you to identify when the number of operations per minute was the highest or how the application performance and operation time change over time. The time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. 106

107 Software Services View - Charts - Availability Report The Software Services View - Charts - Availability report displays performance charts that shows how the application availability or the number of TCP errors changes over time. These charts are filtered for a selected software service or server. How to Access the Report To access this report, open the Network Status - Software Services View or Network Status - Servers View report, click a drilldown link in the Software service or Server name column and then click the Availability tab. For more information, see Network Status - Software Services View Report [p. 98] and Network Status - Servers View Report [p. 99]. Report Contents and Usage By default, the following charts are displayed: Availability TCP connection attempts TCP sessions with errors TCP errors These charts enable you to identify when the number of successful TCP connection attempts was lower than the number of failures or when the number of TCP errors was the highest. The time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Software Services View - Charts - Clients Report The Software Services View - Charts - Clients report displays performance charts for a selected number of users or affected clients filtered per software service or server. Use these charts to see how the number of clients change over time. How to Access the Report To access this report, open the Network Status - Software Services View or Network Status - Servers View report, click a drilldown link in the Software service or Server name column and then click the Clients tab. For more information, see Network Status - Software Services View Report [p. 98] and Network Status - Servers View Report [p. 99]. Report Contents and Usage By default, the following charts are displayed: Unique users Unique clients Chapter 10 Network Analysis Reports 107

108 Chapter 10 Network Analysis Reports Affected users (application) Affected users (availability) Affected users (network) These charts enable you to identify when and how many clients were affected by network, availability, or application performance problems. You can also analyze how the number of unique clients changes over time. The time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Network View - Hosts and Users Reports The Network View - Hosts and Network View - Users reports are sibling reports that list, for a selected server, all hosts and users and their statistics. Metrics presented on this report are specific to the AMD, but performance data can come from AMDs and Flow Collectors. How to Access the Reports The reports are accessed as drilldown reports from numerous higher-level reports, by clicking on values in the columns related to hosts or users (for example, Unique users). Report Contents and Usage In this report, you can identify hosts and users experiencing network performance problems or generating the most traffic. By default, the time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. The full set of all the available statistics is provided on the Software service, operation, and site data data view. For more information, see Software service, operation, and site data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown Reports On the Network View - Users report, click a link in the User name column to acces the following reports: User Activity - User Diagnostics report. For more information, see User Activity - User Diagnostics Report in the Data Center Real User Monitoring Central Analysis Server User Guide. Capture packets dialog box. For more information, see Starting a Packet Capture in the Data Center Real User Monitoring Smart Packet Capture User Guide. 108

109 Chapter 10 Network Analysis Reports On the Network View - Hosts report, click a server name or a server IP address to access the Network Status - Servers View report. For more information, see Network Status - Servers View Report [p. 99]. Network Overview - Links Report The Network Overview - Links report presents network statistics from the link perspective, which shows the link or interface on which the traffic was observed. How to Access the Report From the CAS top menu, choose Reports Network Analysis Network Overview - Sites and click the Links tab. Report Contents and Usage The concept of links applies to the following entities: LAN segments WAN ATM Virtual Path Identifiers (VPI) or Virtual Channel Identifiers (VCI) Frame Relay Other WAN technologies that do not break links into virtual subparts Interfaces as reported by NetFlow User Defined Links (UDLs) The Network Overview - Links report shows measurements from different monitoring devices, for example Flow Collectors, or AMDs. There are no separate reports for different source or link types. However, various monitoring devices provide slightly different information for different links. By default, the time range for the report is Today (the last calendar day for which data is available) and the resolution is set to 1 period. Metrics are grouped into three sections: Usage, Performance, and SNMP. The full set of all the available statistics is provided on the Network link data data view. For more information, see Network link data in the Data Center Real User Monitoring Central Analysis Server User Guide. Drilldown Reports You can access more detailed reports from the Name column: Network Status - Software Services View report. For more information, see Network Status - Software Services View Report [p. 98]. Software Services View - Charts - Network report. For more information, see Software Services View - Charts - Network Report [p. 105]. 109

110 Chapter 10 Network Analysis Reports 110

111 APPENDIX A Diagnostics and Troubleshooting Report-Related Issues Central Analysis Server automatically detects a range of exceptions (anomalies) and notifies the report users. Exception notifications are displayed as yellow (warning) or red (error) triangle icons in the upper-left corner of the report window. To see the notification message, position the cursor over the triangle icon. The Slow Operation Load Sequence report is empty for an operation which is part of an XML transaction. Why and how do I fix this? For XML and SOAP, Operation Elements data is identical to Operation Analysis data, so, to avoid unnecessarily keeping the duplicates in the database, a VDATA_FILTER_XMLSOAP filter is set to true by default. Keeping this filter set to true saves disk space but, because the XML and SOAP entries are filtered out, it makes reporting on the Operation Elements level (elements or headers) impossible. To change the value of VDATA_FILTER_XMLSOAP property in userpropertiesadmin, type in the Web browser's Address bar and press [Enter], change the filter's property value, and click Set value to accept the change. To access this screen, you need to have administrative privileges for the report server. The yellow triangle displays AMDs produce no performance data. What do I do? The message AMDs produce no performance data means that AMDs connected to the report server do not produce any new data. To resolve this issue, you have to investigate the configuration of the AMDs and determine why they do not produce the performance data. The yellow triangle displays An AMD produces data stamped with a time from the future. What do I do? The report server has a built-in protection from simple configuration mistakes. One of the related problems is when data is incorrectly time stamped by AMD. This happens when the AMD is running with the system clock incorrectly set and is not being synchronized with the report server. If you see this notification, check the system time on the report server and on the AMD. Ensure the time synchronization option is turned on. 111

112 Appendix A Diagnostics and Troubleshooting To check the time synchronization: 1. Launch the RUM Console. 2. Select the AMD, right-click it and choose Open Configuration. The AMD Configuration window appears. 3. Select Global General. Check the IP address of the server authorized to set the AMD time. Make sure it is the same as the report server IP address. 4. Check the report server time setting. Do this by reading the time that is displayed at the bottom of the reports. Ensure the report server has the time zone set correctly. Figure 14. Example of the Report Time Stamp The yellow triangle displays A daily maintenance task is in progress. Data processing suspended. What do I do? Once a day the report server has to perform a database maintenance and memory cleanup. During that time, the data processing has to be suspended and you will see delayed data on reports. The daily maintenance is usually performed as the first task after midnight and it takes up to half an hour in installations with a large database. It is normal and expected to see this warning just after midnight. But if you see the message during the day, it can be a symptom of incorrect system configuration (check the time settings on the server) or of system overload. The yellow triangle displays No contact with the primary AMD. What do I do? This message indicates that the report server has lost contact with at least one primary AMD. If an AMD is marked as primary and the report server cannot communicate with this AMD, even if the performance data can be downloaded from the other AMDs, the system will wait until the communication with the primary AMD is restored. The yellow triangle displays No contact with any of the AMDs. What do I do? This message indicates that the communication link cannot be established with any of the attached AMDs. Check the network settings on the report server or the configuration of AMDs. 112

113 Appendix A Diagnostics and Troubleshooting The yellow triangle displays Delay in data processing. What do I do? If the last processed data is significantly behind the current time due to slow data processing or idle periods that occurred in the past, the report server displays the triangle icon with the message Delay in data processing. If the server had a delay, but now it is catching up, this message will not appear anymore. To confirm that delay is decreasing, inspect server.log and search for messages similar to this: T REC :10: zdata_43f47e58_5_t is being processed. Sample begin ts = :25. Sample delay 17 min. If the delay becomes smaller, the server is catching up. If the delay values are growing, it can indicate a system overload. The yellow triangle displays The AMD has not yet generated performance data. What do I do? This message indicates that some data files have already been generated on some AMDs, but not on the others. This may not be an indication of a problem and, when you refresh the reports after 30 to 60 seconds, this message may disappear. If necessary, verify the time synchronization among all the AMDs. See The yellow triangle displays Delay in data processing. What do I do? [p. 113]. The yellow triangle displays Data processing is being performed in the debug mode. What do I do? Data processing can be manually suspended and controlled by so-called debug mode, which can be enabled using Control Panel. Open Control Panel by typing: in the Address field of the web browser and clicking Go, then select Controlled data processing from the Configuration Management section. The red exclamation mark displays Data loading is in progress. Reports may be incomplete. What do I do? This message indicates that the report server is currently starting up. Because of this the information presented on reports may be incomplete. Depending on the database size, the startup process may take up to several minutes. If the server restart was not done manually or was not planned, inspect server.log or contact Customer Support. The red exclamation mark displays Low memory. The real-time cache will only be updated. What do I do? This message indicates that the report server has no free memory to process new entities such as software services, servers, and URLs. This message will be cleared when some resources are freed, this usually happens at midnight during the scheduled database maintenance (see The yellow triangle displays A daily maintenance task is in progress. Data processing suspended. What do I do? [p. 112]). All the metric values presented on reports (except user/client counters) will show correct values. However, the predefined tabular reports may not show all the entities they are intended to show. All the charts and DMI reports show correct data. The mechanism of updating the real-time cache, as described above, is a protection that allows the report server to continue the operation instead of closing down due to lack of memory resources. 113

114 Appendix A Diagnostics and Troubleshooting The red exclamation mark displays The number of servers has reached the defined limit. What do I do? The report server has a built-in limit of the number of monitored servers. If the number of observed servers reaches a defined limit, the report server will not accept any new servers and will drop the collected data for those servers. The predefined value of the limit can be customized. However, the report server can automatically adjust the limit in low-resources situations. The red exclamation mark displays The number of clients has reached the defined limit. What do I do? The report server has a built-in limit of the number of monitored clients. If the number of registered clients (which also includes aggregated virtual clients such as Client from... ) reaches a defined limit, the report server will not accept any new clients and will drop the collected data for those clients. The predefined value of the limit can be customized. However, the report server can automatically adjust the limit in low-resources situations. The red exclamation mark displays The number of sites has reached the defined limit. What do I do? The report server has a built-in limit of the number of automatically created sites. If the number of observed automatic sites reaches a defined limit, the report server will not create any new automatic sites and such traffic will be allocated to All Other. The predefined value of the limit can be customized. However, the report server can automatically adjust the limit in low-resources situations. The Sites report for a selected application is empty. Why? If the Sites report for a selected application is filtered for a client tier, such as Synthetic or RUM sequence transactions, it will not show any data. To see statistics for sites, drill down from the Applications report as follows: 1. Click the application name on the Applications report. 2. Click the client tier name on the Tiers report for a selected application. For the Synthetic tier, you will see the Overview Application Status report; for the RUM sequence transactions tier, the Sequence Transactions Log report. 3. Depending on the type of report, click the Overview Site Status or the Sites tab. I see gaps on the chart reports. Why are the charts incomplete? Gaps in reports mean that the report server missed some data and was not able to get it into the database on time. Your reports may resemble the example below. Figure 15. Gaps in a Graphical Report There are several reasons why the graphical reports may have incomplete data: 114

115 Appendix A Diagnostics and Troubleshooting The AMD was not able to detect any traffic from the monitored network, so it was not able to produce any valid data for the report server. To confirm that this was the reason, connect to the AMD using an SSH client and check whether the files named zdata_xxxxx_x_x are located in the /var/spool/adlex/rtm directory. Similar symptoms can be observed if the AMD has been down for some time and data files were not produced for that time. If data files are present and the viewed chart displays only a fragment of the monitored traffic, for example, for a specific server or site, it may indicate that a part of traffic, which was indented to be monitored, is missing. In this situation, the data files are much smaller than usual for the corresponding period of the day. Similar situations, that is, gaps only on some reports, may occur in a multi-amd installation when some AMD s were down or disconnected from the network. In the case when only one AMD is connected to the report server, communication problems do not cause data gaps. If the report server cannot communicate with the AMD, it will wait until the communication is restored and then will process all the data from the past. When there are multiple AMDs connected to the report server and there is a break in communication with only some of them, the report server processes the data from the available AMDs, so in this case, gaps can appear on some reports. If it is a critical issue and your network (or its parts) require continuous monitoring and you cannot miss the data from some AMDs, you have to mark the AMDs as primary. In this case, the report server will wait until the communication with primary AMDs is restored, even if other AMDs are available. Gaps in charts on some reports in multi-amd installations may be caused by unsynchronized AMDs. The reason for that may be that if the report server sees a data file for a specific time period on one of the AMDs, it will wait only 30 seconds for data files covering the same period of time from other AMDs. The 30 seconds are the server's tolerance for time synchronization issues. To verify that this situation occurred, compare the clock readings from AMDs and then check the time synchronization settings (see The yellow triangle displays An AMD produces data stamped with a time from the future. What do I do? [p. 111]). It may happen that a part of data will be missing. This will result in a significant decrease of the aggregated data, used to render the chart bars. Note that this effect relates to metrics that are calculated as sums, for example, number of operations, number of errors, number of users, or bandwidth utilization. Charts showing the averages (RTT, loss rate, operation time) will not be affected. I see gaps on the log-term data chart reports. Why are the charts incomplete? The report server aggregates the data collected during the day into daily (and monthly) rollups. This is a scheduled process. If this process is not triggered, you will see gaps in the daily rollups. The most frequent reasons for missing rollups are: 115

116 Appendix A Diagnostics and Troubleshooting The report server was down in the night; report data generation starts at 12:10 AM local time and if the report server was down at that time, no aggregate data for long-term reports will be generated. The report server was overloaded and it took too much time for other crucial tasks; report data generation for long-term reports was canceled. You can always re-generate data for long-term reports. Open Control Panel by typing: in the Address field of the web browser and click Go, then select Regenerate Reports from the System Management section. I created a report that consists of several charts but it loads very slowly. How can I improve its performance? If you are using exactly the same set of dimensions and filters for every chart but would like to show different metrics on separate charts, there are two ways of improving such a report. In this example, it is assumed that you want a report that shows Client bytes, Server bytes, and Total bytes on separate charts for the HTTP analyzer. First, the simplest and recommended method, is to define one section that contains all these three metrics. Figure 16. Creating One Section with Three Metrics Open the Chart settings panel and from the single chart per list select Metric. If you are using metrics with different units, you can select the Metric unit option instead. For more information, see Displaying Multiple Charts in the Data Center Real User Monitoring Data Mining Interface (DMI) User Guide. The second method requires changes on the Subject Data and Result Display tabs. 1. For each report section (chart), create the same set of metrics. To do this, for each chart add metrics that are displayed on the other charts. Note that the order of metrics must be the same in every section. For example, each section must contain the Client bytes, Server bytes, and Total bytes metrics listed exactly in the same order. 2. Disable showing unnecessary metrics for each chart. 116

117 Appendix A Diagnostics and Troubleshooting Go to the Result Display tab and disable showing the redundant metrics. For example, for chart that is going to show only the Client bytes metric, disable showing the Server bytes and Total bytes metrics. Figure 17. Selecting Metrics to Display on a Chart Application performance and availability data is missing from the tabular reports. How can I fix this? The missing data manifests itself as zero or a hyphen. The most frequent reason for this situation is the incorrect setting of business hours and holidays. Inspect the business hours and holiday settings by choosing Settings Report Settings Business Hours. The following configuration screen shows the current settings. Figure 18. Business Hours Configuration Screen To collect performance data seven days per week, including non-business days and holidays, clear the Holidays check box and select the check boxes for weekend days. In addition, you can collect performance data in 24/7 mode, but be aware that this results in a higher database growth rate and a larger database. To enable collecting data all the time, open the Control Panel by opening the following page: In the Control Panel, click Advanced Properties Editor from the Configuration Management section. Set ONLY_BUSS_HOUR_REPORTING to OFF. 117

118 Appendix A Diagnostics and Troubleshooting To see whether your holiday definition is correct, click View Holidays. Figure 19. Defined Holidays Screen The list of holidays is hard-coded and the default set is for the USA. To select a set, click the Choose holiday definition list. To see the content of the selected set, click Preview. To store the newly selected set, click Save. 118