Foglight Experience Monitor 5.7.0
2015 Dell Inc. ALL RIGHTS RESERVED. This guide contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser s personal use without the written permission of Dell Inc. The information in this document is provided in connection with Dell products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Dell products. EXCEPT AS SET FORTH IN THE TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, DELL ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL DELL BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF DELL HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Dell makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Dell does not make any commitment to update the information contained in this document. If you have any questions regarding your potential use of this material, contact: Dell Inc. Attn: LEGAL Dept 5 Polaris Way Aliso Viejo, CA 92656 Refer to our web site (software.dell.com) for regional and international office information. Patents This product is protected by U.S. Patents # 6,928,471; 7,539,655; 7,565,336; and 7,941,385. For more information, go to http://software.dell.com/legal/patents.aspx. Trademarks Dell, the Dell logo, and Foglight, IntelliProfile, PerformaSure, and Tag and Follow are trademarks of Dell Inc. "Apache HTTP Server", Apache, "Apache Tomcat" and "Tomcat" are trademarks of the Apache Software Foundation. Google is a registered trademark of Google Inc. Chrome, Android, and Nexus are trademarks of Google Inc. Red Hat, JBoss, the JBoss logo, and Red Hat Enterprise Linux are registered trademarks of Red Hat, Inc. in the U.S. and other countries. CentOS is a trademark of Red Hat, Inc. in the U.S. and other countries. Microsoft,.NET, Active Directory, Internet Explorer, Hyper-V, SharePoint, SQL Server, Windows, Windows Vista and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. AIX, IBM, and WebSphere are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Sun, Oracle, Java, Oracle Solaris, and WebLogic are trademarks or registered trademarks of Oracle and/or its affiliates in the United States and other countries. SPARC is a registered trademark of SPARC International, Inc. in the United States and other countries. Products bearing the SPARC trademarks are based on an architecture developed by Oracle Corporation. OpenLDAP is a registered trademark of the OpenLDAP Foundation. HP is a registered trademark that belongs to Hewlett-Packard Development Company, L.P. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. MySQL is a registered trademark of MySQL AB in the United States, the European Union and other countries. Novell and edirectory are registered trademarks of Novell, Inc., in the United States and other countries. VMware, ESX, ESXi, vsphere, vcenter, vmotion, and vcloud Director are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. Sybase is a registered trademark of Sybase, Inc. The X Window System and UNIX are registered trademarks of The Open Group. Mozilla and Firefox are registered trademarks of the Mozilla Foundation. "Eclipse", "Eclipse Foundation Member", "EclipseCon", "Eclipse Summit", "Built on Eclipse", "Eclipse Ready" "Eclipse Incubation", and Eclipse Proposals" are trademarks of Eclipse Foundation, Inc. IOS is a registered trademark or trademark of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. Apple, ipad, iphone, Xcode, Mac OS, and Safari are trademarks of Apple Inc., registered in the U.S. and other countries. Ubuntu is a registered trademark of Canonical Ltd. Symantec and Veritas are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. YAST is a registered trademark of SUSE LLC in the United States and other countries. Citrix and XenDesktop are trademarks of Citrix Systems, Inc. and/or one or more of its subsidiaries, and may be registered in the United States Patent and Trademark Office and in other countries. AlertSite and DéjàClick are either trademarks or registered trademarks of Boca Internet Technologies, Inc. Samsung, Galaxy S, and Galaxy Note are registered trademarks of Samsung Electronics America, Inc. and/or its related entities. MOTOROLA is a registered trademarks of Motorola Trademark Holdings, LLC. The Trademark BlackBerry Bold is owned by Research In Motion Limited and is registered in the United States and may be pending or registered in other countries. Dell is not endorsed, sponsored, affiliated with or otherwise authorized by Research In Motion Limited. Ixia and the Ixia four-petal logo are registered trademarks or trademarks of Ixia. Legend CAUTION: A CAUTION icon indicates potential damage to hardware or loss of data if instructions are not followed. WARNING: A WARNING icon indicates a potential for property damage, personal injury, or death. IMPORTANT NOTE, NOTE, TIP, MOBILE, or VIDEO: An information icon indicates supporting information. Foglight Experience Monitor Updated - February 2015 Software Version - 5.7.0
Contents Installing and configuring......................................... 10 Pre-installation considerations........................................10 Required hardware and network information...........................10 Network taps................................................11 Installing Foglight Experience Monitor...................................15 Installing the appliance on the rack.................................15 Connecting the appliance........................................15 Making network connections......................................15 Setting the appliance clock.......................................16 Entering required network information...............................17 Configuring the appliance...........................................18 Using the web console and setup wizard..............................19 Entering a permanent license key...................................20 Configuring the Appliance Type....................................20 Designating Time Servers........................................22 Configuring protocols...........................................23 Detecting ports in use..........................................24 Configuring servers............................................25 Completing the Setup Wizard......................................27 Maintaining the appliance...........................................28 Advanced configuration options.......................................28 Appliance management.........................................29 Data monitoring..............................................29 Foglight Experience Monitor and Ixia Net Tool Optimizers...................29 Multi-appliance clusters.......................................... 31 About multi-appliance configurations...................................31 Setting up High Availability (HA) mode..................................32 Configuring Probes for High Availability mode...........................33 Deleting High Availability configurations..............................33 Aggregation of metrics and configuration................................34 Aggregating metrics............................................34 Configuration settings..........................................35 Storing report sets.............................................36 Switching from stand-alone to clusters..................................36 Creating a cluster from a stand-alone appliance.........................37 Configuring the appliance......................................... 38 Network settings................................................38 Foglight....................................................39 Mail......................................................39 Network Time Servers (NTP)......................................40 SNMP......................................................40 Other.....................................................40 3
User accounts...................................................41 User account management.......................................41 Global user account options.......................................46 Increasing security for user account management and access privileges..........48 Security settings on the appliance.....................................50 Configuring the web server to use SSL................................50 Disabling the use of SSL.........................................51 Report settings..................................................51 Email groups.................................................52 Adding email groups............................................52 Scheduled reports.............................................53 Metric categories..............................................54 Database metrics..............................................55 Baseline calculations...........................................56 Time frames.................................................57 Distribution metrics............................................58 Percentile metrics.............................................60 Foglight metrics..............................................60 User agent transformations.......................................62 Database configuration............................................66 Viewing the database page.......................................67 Viewing the database details page..................................67 Resetting a database...........................................68 Resetting the configuration.......................................69 Exporting a configuration........................................69 Importing a configuration........................................70 Setting database retention.......................................70 Configuring remote database access.................................72 Purging existing resources........................................72 Appliance integrity...............................................73 Viewing system information.......................................74 System information in a multi-appliance setup..........................74 Viewing and changing the license...................................75 Database repair log............................................76 Updating the appliance............................................76 Installing an upgrade...........................................77 Backing up and restoring data........................................78 Restoring a backup.............................................82 How backup works.............................................82 About the security of your appliance.................................84 Specifying monitored web traffic.................................... 85 Configuration settings.............................................85 Identifying protocols..............................................85 Manually adding protocols and ports.................................86 Automatically discovering ports....................................87 Removing protocols............................................89 Managing protocols on a Portal.....................................89 4
Managing monitored servers.........................................89 Manually adding a new server.....................................90 Automatically discovering servers...................................90 Removing servers..............................................91 Editing server details...........................................91 Configuring server options........................................93 Configuring SSL keys............................................97 Exporting SSL keys from IIS.......................................99 Managing servers on a Portal..................................... 102 Using filters to exclude traffic...................................... 103 Defining URL filters........................................... 103 Testing URL filters............................................ 104 Defining subnet filters......................................... 104 Defining subnets................................................ 105 Manually adding subnets........................................ 106 Automatically discovering subnets................................. 107 Checking for overlapping subnets.................................. 108 Configuring a default subnet..................................... 109 Removing subnet definitions..................................... 109 Managing subnets on a Portal..................................... 110 Identifying user sessions........................................... 110 How user sessions are identified................................... 111 Configuring user session identification............................... 112 Defining user session options..................................... 114 Configuring client IP tags....................................... 115 Augmenting user sessions log information............................ 116 Using logout patterns to detect completed sessions...................... 117 Monitoring instrumented web pages................................... 119 Instrumenting web pages....................................... 119 Configuring instrumentation options and metrics........................ 120 Transforming monitored URLs......................................124 Managing URLs................................................. 124 Encoded and decoded URLs...................................... 125 Setting URL encoding in the Resource List............................ 125 Setting URL encoding for user sessions metrics in the All Metrics View.......... 126 Filtering data in the Resource List................................. 126 Processing and aggregating URLs..................................... 126 Sending URLs to the Foglight Experience Viewer........................ 127 Transforming URLs example using variable rules........................ 128 Managing URL transformation rules................................... 129 Defining site rules............................................... 129 Declaring site rules........................................... 130 Automatically discovering site rules................................ 130 Editing and removing site rules................................... 131 Defining path rules.............................................. 131 Segment rules as path rule components.............................. 132 Segment rule actions.......................................... 134 5
Managing path rules........................................... 138 Editing and removing path rules................................... 140 Managing variable rules........................................... 140 Transforming query variables.................................... 141 Transforming form variables..................................... 142 Transforming session identification variables.......................... 142 Performing actions on variable names and values....................... 143 Using hints.................................................... 148 Defining page hints........................................... 149 Defining hit hints............................................. 150 Defining asynchronous page hints.................................. 150 Defining asynchronous hit hints................................... 151 Testing hints................................................ 151 Deleting hints............................................... 152 Using page definitions......................................... 152 Advanced URL options............................................ 156 Show ports in URLs option..................................... 157 Show parameters in URLs option................................. 157 Show HTTP request methods in URLs option......................... 157 POSTs with no content-type should be handled as XML option.............. 158 Proxy tunneling.............................................. 158 Do not strip the www prefix from URLs option...................... 158 Ignore 401 codes during NTLM authentication option.................... 159 Exclude redirections from metric calculations option................... 159 Use HTML parser for page recognition option......................... 160 Extensions................................................. 160 Response codes.............................................. 161 Managing applications............................................162 Integrating with CA SiteMinder...................................... 162 Communicating with SiteMinder................................... 163 Integrating with SafeNet Hardware Security Modules (HSMs)................... 165 Before you begin............................................. 165 Configuring Foglight Experience Monitor to use SafeNet HSM................ 166 Associating monitored SSL ports with HSM keys......................... 168 Managing application components.................................... 169 Defining an application component................................. 170 Editing an application component.................................. 173 Deleting an Application Component................................ 173 Defining an instrumented application component....................... 173 Editing an instrumented application component........................ 174 Deleting an instrumented application component....................... 175 Managing services definitions....................................... 175 Defining a service............................................ 175 Configuring services auto-discovery................................ 179 Configuring synthetic transactions................................. 181 Managing existing service definitions................................ 182 Using alternative methods to define a service.......................... 183 6
Monitoring Microsoft Office SharePoint Servers............................ 183 Configuring SharePoint monitoring................................. 184 Configuring how URLs display..................................... 186 SharePoint and SOAP.......................................... 186 Monitoring PeopleSoft applications.................................... 187 Monitoring PeopleSoft applications................................. 187 Configuring how URLs display..................................... 189 Monitoring Siebel applications....................................... 189 Monitoring Siebel applications.................................... 190 Configuring how URLs display..................................... 191 Monitoring SOAP applications....................................... 192 Defining a SOAP application...................................... 193 Modifying SOAP application definitions.............................. 195 Defining a SOAP transaction...................................... 195 Managing existing SOAP transaction definitions......................... 197 Mapping SOAP operations to a web service............................ 197 Adding and removing SOAP tags................................... 198 Configuring enterprise-wide service levels.............................. 199 Setting enterprise-based service levels.............................. 200 Foglight components and the appliance...............................201 Connecting to the Foglight Experience Viewer............................ 201 How Foglight Experience Monitor and Foglight Experience Viewer work together... 201 Connecting the appliance to your network............................ 202 Connecting Foglight Experience Viewer to a Foglight Experience Monitor........ 203 Configuring monitoring appliances................................. 203 Foglight Management Server........................................ 207 Ensuring product compatibility.................................... 208 Setting up the Cartridge for FxM................................... 208 Configuring the appliance for data communication...................... 208 Exporting metrics to Foglight..................................... 209 Synthetic transaction scripts........................................ 211 Using the console program........................................212 Accessing the console program...................................... 212 Logging in using the Dell Remote Access Controller (DRAC)................. 213 Logging in from Microsoft Windows................................. 213 Logging in from Linux and UNIX................................... 214 Exploring the main menu.......................................... 214 Network configuration............................................ 215 Configuring the control device.................................... 216 Configuring the auxiliary device settings............................. 217 Displaying information for monitor devices............................ 218 Advanced network settings...................................... 219 System date and time configuration.................................. 219 System date configuration....................................... 220 System time configuration....................................... 221 System timezone configuration................................... 221 7
Database management............................................ 222 Account management............................................ 223 Change password............................................. 223 Enable and disable SSH access.................................... 224 Enable and disable web access.................................... 224 Appliance update............................................... 224 Update from CD............................................. 225 Update from PKG file.......................................... 225 Update from USB flash drive..................................... 225 Advanced options............................................... 225 Configuring the appliance type................................... 226 Configuring Foglight Experience Viewer settings........................ 227 Reset firewall to allow HTTP..................................... 228 Add and remove network device................................... 228 Configuring the license......................................... 228 More advanced options......................................... 229 System restart................................................. 229 Troubleshooting................................................ 230 Sample output for NIC driver..................................... 231 Reporting system status........................................... 232 Logging out of the console program................................... 235 Troubleshooting the appliance......................................236 Common issues when installing the appliance............................. 236 Appliance connection failure through a web browser..................... 236 All reports are empty.......................................... 236 Verify the network configuration and connectivity....................... 237 Verify the network tap configuration................................ 241 Verify the server configuration................................... 244 Appliance runtime issues.......................................... 244 Foglight Experience Monitor reports................................ 244 Report sets are not updating..................................... 245 Confirming system health.......................................... 246 System performance.......................................... 247 System events............................................... 247 Database performance......................................... 248 Error rates................................................. 248 Using the appliance support tools.................................... 248 Install log.................................................. 249 Creating a support bundle....................................... 249 Monitor NIC activity test........................................ 249 TCP dump................................................. 250 Appendix: Third party software.....................................251 Using X-Forwarded-For............................................ 251 Enabling the insert X-Forwarded-For in the HTTP profile................... 251 irule..................................................... 252 Configuring the web server to extract the IP address from the HTTP header...... 252 8
Appendix: Dell PowerEdge system appliance............................254 Dell PowerEdge R300 and R310...................................... 254 Dell PowerEdge R610............................................. 255 Dell PowerEdge R710............................................. 256 Dell PowerEdge R720............................................. 256 Updating the Dell system firmware on the appliance........................ 257 Accessing the drivers and downloads................................ 257 Downloading the ISO images..................................... 258 Updating the firmware on the appliance............................. 259 Contacting Dell................................................. 260 Technical support resources........................................ 260 9
1 Installing and configuring Foglight Experience Monitor is a comprehensive appliance-based solution that empowers organizations to effectively manage, troubleshoot, and optimize all components of the service delivery chain under their control. With our turn-key, self-contained computer system, customers gain unprecedented visibility into the inner workings of their network infrastructure and the quality of the end user s experience. The appliance provides concise, accurate information in real-time about component performance, systematic failures, and a wealth of other information. This provides configuration instructions, conceptual information, and instructions on how to use the browser interface. This guide is intended for users who want to configure Foglight Experience Monitor using the browser interface. Foglight Experience Monitor appliances are shipped with the Foglight Experience Monitor Quick Installation Guide, which provides all the essential information required to physically install the appliance, connect it to your network, and configure it to collect network traffic. This section follows the steps outlined in the Quick Installation Guide, and provides additional information about installing and configuring the appliance, as well as references to topics covered elsewhere in the or in the User Guide. For more information, see these topics: Pre-installation considerations Installing Foglight Experience Monitor Configuring the appliance Maintaining the appliance Advanced configuration options Pre-installation considerations This section provides a quick overview of items and network information required during the installation of a Foglight Experience Monitor appliance. For more information, see these topics: Required hardware and network information Network taps Required hardware and network information The following hardware is required before installation: Monitor and Keyboard during the installation process, you need a VGA monitor and keyboard with a USB connector to perform the initial setup of the appliance. Network Cables the appliance requires network cables to connect its Control port to your network and its Monitoring ports to network taps. If you plan to use multiple Monitoring ports, the corresponding number of cables is required (for more information, see Multiple monitoring ports). The following network information is required before installation: 10
Appliance IP Address a unique IP address assigned to Foglight Experience Monitor. Subnet Mask the subnet mask that corresponds to the Foglight Experience Monitor IP address and the subnet. Gateway the IP address of your network s gateway. Primary DNS IP Address allows the appliance to perform DNS look-ups. Network Time Protocol (NTP) server the IP address of one or more NTP servers. This allows the appliance to synchronize its clock with an external NTP server. Ports the port numbers used on your network for HTTP and HTTPS network traffic. Figure 1. Monitor Devices menu The Monitor Devices menu in the console program shows which monitoring ports are in use. For more information about the console program, see Displaying information for monitor devices and Accessing the console program. Network taps A network tap provides monitoring access to the traffic flowing across a network. Different types of devices are often used as network taps, but only full-duplex network taps and smart taps with full-duplex enabled guarantee the 100% data capture that requires to provide complete and accurate data processing and reporting. A true full-duplex tap splits the ingress traffic (requests to the web servers) and egress traffic (responses from the web servers) into two streams. When those streams are fed into two separate monitoring NICs on an appliance, you get full data capture. IMPORTANT: Foglight Experience Monitor does NOT support monitoring web traffic using SPAN ports or aggregation network taps. These devices do not reliably mirror all traffic routed through them to the monitoring port, which results in inaccurate data in Foglight Experience Monitor. For more information, see Understanding packet drops with unsupported network taps. Configuring a network tap Installing a tap between two points on a network requires that the network cable connecting these points be replaced with a pair of cables both feeding into the tap. The tap passes through all traffic between the two points normally so there is no disruption to the normal traffic flow, and it also copies that traffic to its monitor port. Connecting the tap s monitor port to one of the appliance s monitoring ports provides the network traffic that the appliance needs to monitor your applications. Two tap providers include Datacom Systems (http:// www.datacomsystems.com) and NetOptics (http://www.netoptics.com). 11
It is important that you consider the type of ports you are connecting (for example, RJ45, fiber LC, fiber SC). Network taps have different connector options that should match the type of connections used on your network. IMPORTANT: Connecting the appliance to a port that is part of a VLAN results in incomplete traffic monitoring and inaccurate metrics. As a member of a VLAN, that port is only exposed to VLAN broadcast traffic, and cannot see traffic going to other servers. Consequently, this is not a valid deployment option. To connect the network tap to a network, disconnect the cables from switch/server/router A and B. Next, connect the cable from switch/server/router A into Network port A on the network tap, and the cable from switch/server/router B into Network port B on the network tap. Figure 2. Connecting a network tap to the network Network Ports Monitor Ports To switch, server or router NOTE: Use this configuration when the network tap does not have a power supply. To connect the network tap to the appliance, connect Monitor port A to Monitoring NIC 1 (eth2) on the Foglight Experience Monitor appliance. Next, connect Monitor port B to Monitoring NIC 2 (eth3) on the Foglight Experience Monitor appliance. Figure 3. Connecting the network tap to the appliance Network Ports Monitor Ports To monitoring NIC 1 To monitoring NIC 2 Multiple monitoring ports Foglight Experience Monitor currently supports Dell PowerEdge systems that have up to eight monitoring NIC ports. NOTE: The number of Ethernet ports available on your appliance varies, depending on the hardware type. For a list of supported hardware platforms and guidance in identifying the Ethernet ports, see Appendix: Dell PowerEdge system appliance. When using multiple monitoring ports, the appliance automatically reads each NIC port for incoming traffic. No additional configuration on the part of the administrator is required. The console program allows you to see which ports the appliance is using for monitoring and enables you to perform basic troubleshooting for each port. 12
Understanding packet drops with unsupported network taps SPAN ports and aggregation network taps are often used for small-scale monitoring purposes. However, they are not reliable enough to support a large-scale, enterprise-critical monitoring solution like. The following bullets explain why these devices are unsuitable for use with. SPAN ports SPAN ports (also known as diagnostic ports) can be found on most switches and routers. Network traffic flowing through a switch can be mirrored to the SPAN port, which in turn connects to a monitoring solution. Switch providers do not guarantee that 100% of the traffic reaches the SPAN port. Spikes in traffic through the switch can result in a significant number of dropped packets (from 5% to as much as 20% of all packets). Dropped packets translate to poor data quality in. Aggregation network taps Aggregation network taps take a full-duplex link and merge the ingress and egress streams into a one half-duplex stream. While they drop substantially fewer packets than SPAN ports (due to the use of buffering), aggregation taps are not guaranteed to capture 100% of the traffic routed through them. Aggregation taps generally drop less than 1% of packets, but a 1% drop rate (potentially higher in some instances) has a significant impact on the quality of the data in. For information on supported network taps, see Network taps. Monitoring individual web servers In the simplest case, where the traffic you want to monitor is handled by one or two web servers, connect the appliance to a network tap located in front of the individual web servers. The appliance monitors only this traffic. Monitoring web server farms To monitor all web servers in a server farm, connect the appliance to a network tap between the firewall and the load balancer, as shown in the following diagram. This allows an appliance to see all traffic before it is separated out by server destinations. Redundant paths to the server farm (or multiple server farms) may mean deploying additional network taps to obtain complete coverage. IMPORTANT: If you have a reverse proxy in your network, load balancer, or you may want to choose a different deployment location. For more information, see Understanding how load balancers and reverse proxies affect metrics. Figure 4. Connecting the appliance to a network tap 13
Configuring web servers in a server farm When the network tap is installed between the firewall and the load balancer, the traffic contains only the IP address of the load balancer rather than the web servers. While this behavior is important for security purposes, it means that Foglight Experience Monitor attributes the data for all web servers to the load balancer IP address. Usually, you want data attributed to individual web servers. To enable the appliance to track data by web server, you need to configure the web servers to insert an extra shared HTTP response header into the traffic. For example, you can create an HTTP response header called SERVER-ID. On each web server, you assign a unique, fictitious (for security purposes) IP address to this header. After the Foglight Experience Monitor web console is running, an FxM Administrator defines a server identifier with the same name as the response header, and the appliance automatically begins tracking hits by web server. For more information, see Configuring server options. Understanding how load balancers and reverse proxies affect metrics Some load balancers and all reverse proxies multiplex requests from multiple client connections onto one proxy network connection to the web server. So, while there could be thousands of connections coming into the website, the load balancer or proxy may maintain only a dozen connections to each web server. This network architecture will affect some of the metrics collected by Foglight Experience Monitor, depending on where the network tap is installed in relation to the load balancer or proxy. The affected metrics are Network Latency and Processing Time for hits and pages, which are also included in the calculations of End-to- End Time for hits and pages. For more information about how these metrics are calculated, see How standard metrics are calculated in the FxM Metric Reference Guide. When a network tap is installed between the client and the reverse proxy, Network Latency is an estimate of the time it takes for a request to travel from the end user s device to the point where the tap is located. This is the desired measurement. The Processing Time includes the time spent on the request by the load balancer or reverse proxy and the web server, which in many cases is acceptable, but may not be desirable if your stakeholders want the Processing Time to reflect the performance of the web server only. When a tap is between the load balancer or reverse proxy and the server farm, the Processing Time reflects the time spent by a web server without the overhead of the load balancer or proxy, but the Network Latency is a fraction of the actual Network Latency. In this scenario, Network Latency is based on the time it takes for a request to travel from the load balancer or reverse proxy to the tap. The following table summarizes the trade-off in metric calculations. Select the deployment option that suits your needs. Table 1. Deployment options and metric calculations Network Tap Installed Network Latency Processing Time Between the client and the load balancer or reverse proxy (recommended) Between the load balancer or reverse proxy and the server farm Estimate of time elapsed from the end user s device to the tap, which is the expected value. Estimate of time elapsed from the load balancer or reverse proxy to the tap, which is a fraction of the expected value. Time spent on a request by the load balancer or reverse proxy and a web server. Time spent on a request by a web server. Some customers install network taps in both locations and configure two separate installations of Foglight Experience Monitor to monitor the taps. The Foglight Experience Monitor instance monitoring the client side tap provides accurate Network Latency metrics, while the Foglight Experience Monitor instance monitoring the server farm side provides accurate Processing Times and volume metrics for each web server. IMPORTANT: If you install network taps on both sides of a load balancer or reverse proxy, do not connect them to the same installation of Foglight Experience Monitor. The metrics collected in this fashion are inaccurate. 14
Installing Foglight Experience Monitor This section provides directions for installing the hardware and for connecting the Foglight Experience Monitor to the network. For more information, see these topics: Installing the appliance on the rack Connecting the appliance Making network connections Setting the appliance clock Entering required network information Installing the appliance on the rack All Dell PowerEdge systems ship with a full set of documentation and rack-mounting hardware. For instructions on how to unpack and physically install the Foglight Experience hardware in the network rack, see the Dell Systems Rack Installation Guide or Dell Systems Rack Installation Instructions. IMPORTANT: If you are deploying a Foglight Experience Viewer along with the Foglight Experience Monitor, before continuing with this procedure, see Connecting to the Foglight Experience Viewer. Connecting the appliance When mounting the Foglight Experience Monitor appliance onto the server rack, all network hardware must be connected before beginning the logical configurations. NOTE: The number of Ethernet ports available on your appliance varies, depending on the hardware type. For a list of supported hardware platforms and help identifying the Ethernet ports, see Appendix: Dell PowerEdge system appliance. To connect the appliance: 1 Attach one end of the network cable to the correct monitoring port (eth2-eth5). NOTE: The fiber-capable NICs use an LC connector. If your network uses SC connections, use an SCto-LC adapter cable to make the proper connection. 2 Plug the other end of the cable into the network tap. 3 Attach another network cable to the control port (eth0) on the appliance. 4 Plug the other end of the cable into the appropriate port on your WAN. 5 Attach the power cable, USB keyboard, and VGA monitor to the appropriate ports on the rear panel of the appliance. 6 Power on the appliance. TIP: After the appliance has been deployed, if you encounter problems with appliance data collection, or communication issues between the Foglight Experience Monitor and other machines on the network, see Troubleshooting the appliance for help. Making network connections The console program allows you to perform the initial configuration of the appliance. When setting up the appliance for the first time, you must set the system clock, and configure network settings. 15
Logging into the console If you correctly attached all cables to the Foglight Experience Monitor (as outlined in Connecting the appliance), the console program login prompt appears on the connected VGA monitor after powering on the appliance. For more information about logging into the console, see Accessing the console program. Log in with user name settings setup and password setup. TIP: After this installation, you can modify the account password using the console program. See Account management. All account passwords can also be modified in the web console. See User accounts. After successfully logging in, the main menu appears. Figure 5. Welcome to Setup menu Setting the appliance clock These options allow you to change the appliance date, time, and time zone. To set the system clock: 1 From the console main menu, arrow down to System Time Configuration and press Enter. 2 To modify the date, arrow down to Change Date and press Enter. The System Date Configuration screen appears. 3 Using the calendar options, enter today s date and return to the System Date / Time System menu. 16
4 To modify the time, arrow down to Change Time and press Enter. The System Time Configuration screen appears. 5 Type the current time in the 24-hour hh:mm:ss format and return to the System Date / Time System menu. 6 To modify the timezone, arrow to the Change Timezone and press Enter. The YAST screen appears. Select the clock and time zone that you want the appliance to use when displaying data. You can leave the UTC setting unchanged at its default value. 7 Press OK to return to the main menu. Entering required network information From the console main menu, arrow down to Network Configuration and press Enter. Figure 6. Network Configuration menu Arrow down to a selection to view and set network options. Setting control device options To enter required network information: 1 From the Network Configuration menu, arrow down to the Control Device option. The control device is the NIC port that is used for external access to the appliance through a web browser or an SSH client. 2 Enter the following information: 17
IP Address you are assigning to the appliance Netmask used for the network segment in which the appliance is located Network Gateway IP address Hostname for the appliance (for example a fully qualified domain name such as agent4.dell.com) the IP address for the primary DNS server the IP address of the secondary DNS server 3 Press OK to return to the Network Configuration menu. 4 Arrow down to Save Settings and press Enter. 5 The Save Changes dialog appears. To save settings, tab to the Yes option and press Enter. The system saves all of the new network settings, and begins the verification process. Each of the network settings are tested and the results are displayed in the console program. If the network configuration test fails, return to the Network Configuration Menu and verify that the IP addresses you have entered are correct. When the network configuration test succeeds, your appliance is now properly connected to your network and configured for use. Logging out of the console program After passing the network verification test, you are returned to the console program main menu. Tab to the Exit option and press Enter to leave the console program, and initialize the appliance. Configuring the appliance The Setup Wizard takes you through the installation process. For more information, see these topics: Using the web console and setup wizard Entering a permanent license key Configuring the Appliance Type Designating Time Servers Configuring protocols Detecting ports in use Configuring servers Completing the Setup Wizard 18
Using the web console and setup wizard The web console provides a user interface through which users can examine the metrics collected by the system and create customized reports. It is accessed whenever you use a web browser to connect to the IP address or hostname assigned to the appliance. When you connect to the appliance for the first time, the Setup Wizard is initialized, and guides you through the steps necessary to get the appliance started. IMPORTANT: You must have cookies enabled in your web browser in order to log in to, and use the web console. To log in to the Wizard: 1 Open a web browser on any computer on the same network as the Foglight Experience Monitor. 2 Type the Foglight Experience Monitor IP address or host name in the browser s URL address box (For example, http://192.168.1.10, or fxm.company.com). The IP address and host name were respectively configured at step 1 and step 6 in the previous section, Making network connections on page 15. The web console login screen appears: 3 Type admin in the Login and Password boxes, then click Submit. After logging in, the Setup Wizard introduction page is displayed. The Setup Wizard introduction page displays a list of items you need in order to complete the Setup Wizard: a license key specify whether the appliance is a portal, probe, or stand-alone an IP address for an NTP server ports used for TCP traffic on your network 19
IP addresses of the servers you intend to monitor NOTE: For Multi-Appliance Users - If you are currently creating a portal as part of a multi-appliance cluster, you only need to set the NTP server. Every page of the Setup Wizard has four buttons along the bottom of the page. Table 2. Setup wizard generic buttons Cancel Back Next Finished Click this to terminate the Setup Wizard at any time; the Foglight Experience Monitor will not be configured, and then next time you log in the Setup Wizard will automatically restart. Click this at any time if you want to return to the previous page of the Setup Wizard to make changes. Moves you to the next page of the Setup Wizard; click this when you have completed the current page. Finishes the process. This button is only enabled on the last page of the Wizard. IMPORTANT: Avoid using your web browser s back and forward buttons when using the Setup Wizard. Instead, use the Back and Next buttons provided on each page of the Wizard. 4 Click Next to continue to the next Setup Wizard step. Entering a permanent license key If a permanent license key has already been installed on your appliance skip this step of the Setup Wizard and proceed directly to the next step. If your appliance has only a temporary key the Appliance License page of the Setup Wizard appears as shown in the following illustration. Figure 7. Appliance License page To obtain a permanent license key, you need to provide Dell Technical Support with the Host Id as shown on this page. Once you receive the license key paste it into the License Key field and click Next to continue. Configuring the Appliance Type The next Wizard Setup step is the Type page. 20
Figure 8. Type page The appliance must be configured either as a stand-alone machine, or as part of a multi-appliance cluster. Stand-alone appliances do not communicate with other appliances, whereas appliances that are part of a cluster make use of a portal to aggregate metrics from probes, and present a unified view of the cluster of appliances. For more information, see Multi-appliance clusters. You can configure your appliance for one of three distinct roles. First, it can function as a portal that provides a unified database and configuration repository for a cluster of appliances. Portals provide a unified view of the metrics collected by multiple probes, they do not monitor network traffic. Second, it can function as a monitoring probe in a multi-appliance cluster. Probes monitor network traffic based on the configuration specified on the portal, and transmit all of their collected metrics to the portal. Third, it can function as a stand-alone monitor. Stand-alone monitor network traffic and save collected metrics into a database on the same appliance. They do not communicate with other appliances. When you change the appliance type using this page, a pop up window appears informing you that the process may take a few minutes. IMPORTANT: To configure an appliance as a probe, the system must be able to establish a network connection to the portal. If an error message appears during the process the most likely cause is that the appliance could not contact the portal using the IP address provided. Configuring a Stand-alone If you are installing a single appliance on your site, or one of multiple appliances that are not going to share metric data, select Stand Alone Monitor, then click Next to continue configuring the appliance. Configuring a Portal The following procedure sets up the appliance to function as a Portal. To configure a Portal: 1 If you are installing an appliance as part of a multi-appliance cluster, and would like the current appliance to act as the portal, select Portal. 2 In the Data port box, type the TCP port number that the appliance will use for data communication with other appliances. 3 This can be any port number that is not in use on your network or by the appliance itself for other purposes. Be careful that you do not use a port number that will be blocked by your firewalls or other security devices on your network. The web console displays an error message when you choose a port that is already in use on the appliance. 21
4 In the Control port box, enter the TCP port number that all appliances in the cluster will use to communicate control and configuration information using MySQL protocol. The default setting is 3306. When possible use the default setting as it will avoid the need for port forwarding on the portal. In a multi-appliance environment, all control and configuration settings are stored in the MySQL database on the portal. You can use any port number that is not in use by the appliance or in use by the network for other purposes. 5 Select the Encrypt using SSL check box if you want the data communications between the portal and probes to be encrypted using SSL. NOTE: This setting does not affect control and configuration communication which is performed using unencrypted MySQL. 6 Select the Compress to minimize bandwidth check box if you want network communication between the probes and portals to use data compression. This setting decreases the volume of network transfers but adds some amount of overhead required to compress and decompress the data. 7 Click Next to move to the next step of the Wizard. When installing and configuring appliances as part of a multi-appliance environment, set up and configure the portal before configuring probes that connect to the portal. IMPORTANT: Portals and probes must use the same version of the software in order to communicate. Mixing versions is not supported. It is also not possible to mix 32-bit and 64-bit appliances in a multiple-appliance cluster. Configuring a Probe The following procedure sets up the appliance to function as a Probe. To configure a Probe: 1 If you are installing an appliance as part of a multi-appliance cluster, and would like the current appliance to act as a probe, choose Probe. 2 In the Portal IP box, type the IP address of the appliance that you configured to serve as the portal. 3 In the Control port box, type the TCP port number that the probe uses to retrieve control and configuration information from the MySQL database that is hosted on the portal. This port number must match the number that was previously configured on the portal. 4 Click Next to move to the next step of the Wizard. The server addition and discovery process found in the Setup Wizard is identical to configuration functionality that is used after installation. To further refine your server list, see Managing monitored servers. Designating Time Servers The Time Servers page allows you to identify one or more Network Time Protocol (NTP) servers. Setting NTP servers ensures accurate synchronization of the appliance s internal clock to the correct time. 22
Figure 9. Time Servers page NOTE: Multi-appliance deployments require that the portal and probes maintain consistent clocks, in order to accurately aggregate data and synchronize monitoring. Stand-alone installations of the appliance also benefit from accurate timestamps for data aggregation, and the reporting of historical data. Stand-alone configuration Type up to three NTP server IP addresses in the Time Server (#1, #2, #3) boxes on this page. Click Apply when you have finished configuring NTP server information. Multi-appliance configuration If configuring a portal, the Setup Wizard is complete at this point. Proceed to Completing the Setup Wizard. Configuring protocols The Protocols page of the Setup Wizard enables you to specify which protocols and ports you would like the appliance to monitor. The appliances provides detailed metrics for HTTP and a standard set of metrics for all other TCP-based protocols. Figure 10. Protocols page HTTP on ports 80 and 8080 and HTTPS on port 443 are automatically configured as default protocols. If your site uses other ports, modify the list of ports for each protocol on this page. If your site uses protocols, other than HTTP and HTTPS, click Add a Protocol to define the protocol and the ports it uses. After the protocols and port numbers are configured to match those used by your site, click the Next button to advance to the next step of the Setup Wizard. 23
Detecting ports in use To detect all of the ports that are currently being used, use the auto-discovery feature on this page. A list of ports in use is returned. The data that is returned for each port can help you determine if there are any deployment issues with how the appliance is monitoring network traffic. To detect ports in use: 1 Click Auto-Discover Ports. The appliance begins to analyze current network traffic and lists the ports that it discovers. The autodiscovery page refreshes frequently, and all ports appear within a few seconds. 2 To stop the auto-discovery process, click Stop at any point. The auto-discovery page displays the following data: Table 3. Information displayed on the auto-discovery page Data Ports Client Packets Server Packets New TCP Connections Active TCP Connections Definition Displays the port number. The number of client packets being transmitted from clients to the server. NOTE: If the packet count is zero and the Server Packet value is greater than zero, the network tap is not configured correctly. The number of packets being transmitted from servers to clients. NOTE: If the packet count is zero and the Client Packet value is greater than zero, the network tap is not configured correctly. The number of TCP connections that have been observed on the port. NOTE: If the number of connections equals zero or the number is small in relation to the number of active connections, the load balancer is maintaining a limited number of TCP connections to the web server and using the proxy server to send all requests over the TCP connections. The number of active TCP connections discovered in the traffic for the port. 3 Click the details link to open a details dialog box for this port. The HTTP-detected and SSL-detected indicators report whether HTTP or SSL traffic was detected for this port. All servers listening on this port are listed. 24
4 Click OK to close the window. 5 Select the check boxes that correspond with any ports you want the appliance to use to monitor traffic. 6 Click Add. The Protocols > Edit page appears where you can assign a name and type to the protocol definition, and then save it. 7 Click Next to proceed to the next step of the Setup Wizard. Configuring servers The Servers page allows you to identify which servers on your network are being monitored by the appliance. Figure 11. Servers page 25
Auto-discover servers Although you can manually enter the IP addresses of the servers you want to monitor, it is recommended that you use the auto-discovery feature to detect available server IP addresses in the traffic that the appliance is currently receiving. To automatically discover servers: 1 Click Auto-Discover Servers to begin the server detection process. The appliance begins to analyze current network traffic and looks for server IPs. The page updates every few seconds, expanding the list of servers as they are detected. The list of servers should be complete within one minute. TIP: If only one IP address is displayed during the auto-discovery process, you may have a situation where the physical server IPs are not present in the IP headers of the traffic the appliance is receiving. This is typically the case when a load balancer is in use. For this case, you need to perform additional configuration steps in the appliance. For more information, see Configuring a server identification tag. 2 Click Stop at any point to stop the process. All servers found appear in a check-list. The following information displays in the returned list of Servers. Table 4. Information displayed in the list of servers page Data Servers Client Packets Server Packets New TCP Connections Active TCP Connections Definition Displays the server number. The number of client packets being transmitted from clients to the server. NOTE: If the packet count is zero and the Server Packet value is greater than zero the network tap is not configured correctly. The number of packets being transmitted from the server to clients. NOTE: If the packet count is zero and the Client Packet value is greater than zero the network tap is not configured correctly. The number of TCP connections that have been observed on the port. NOTE: If the number of connections equals zero or the number is small in relation to the number of active connections, the load balancer is maintaining a limited number of TCP connections to the web server and using the proxy server to send all requests over the TCP connections. The number of active TCP connections discovered in the traffic for the port. 3 Click the details link to open a details dialog box for this server. 26
The HTTP-detected and SSL-detected indicators report whether HTTP or SSL traffic was detected for this port. Active ports for the server are listed. 4 Click OK to close the window. 5 Select all the server check boxes that you want the appliance to monitor. 6 Click Add. 7 You are returned to the main Servers page, and all newly added servers are now listed. IMPORTANT: You can configure a maximum of 300 server IPs. Once the appliance is running, use the System Health metrics to determine whether the number of servers you have configured is too much for a single appliance. If the Packet Drop Rate is above 1% then you may need to reduce the number of server you have configured and investigate alternative deployment configurations. For more information about servers, see Managing monitored servers. Completing the Setup Wizard The final page of the Setup Wizard summarizes the configuration choices made in previous steps, and asks that you confirm them. 27
Figure 12. Setup Wizard final page Review all the configuration settings. It you need to return to any point in the Setup Wizard to make modifications, click the Jump to Page links to move to specific pages, or use the Setup Wizard Back button to review pages in reverse chronological order. When you are satisfied with the configuration settings, click Finished to complete the Setup Wizard. NOTE: If you do not click Finished, the appliance does not begin collecting data, and the next time you log in to the web console, you are prompted to begin the Setup Wizard again. However, all information entered in the previous session with the Setup Wizard is retained, and you can simply click Next at each step if you are still content with the configuration settings that exist. After clicking Finished, you are taken to the home page. The appliance begins to monitor traffic and metrics begin to appear within 10 minutes. TIP: After the appliance is deployed, if you encounter data collection or communication problems, see Troubleshooting the appliance for help. The steps outlined in the Troubleshooting chapter help you to pinpoint the source of these problems. Maintaining the appliance As a best practice, it is recommended that you reboot each appliance at least once every 90 days. This operation can be performed through the console program (for details, see System restart). Rebooting the appliance allows the system to run the standard file system maintenance utilities (fsck and xfsrepair) during the boot-up sequence. This helps to avoid file system inconsistency errors that periodically arise in the normal course of operations. Failing to run these system maintenance utilities on a regular basis may result in the need for manual disk repairs that could involve booting the system from a CD. Advanced configuration options The appliance supports a variety of advanced configuration options that allow you to fine-tune how it monitors network traffic and generates reports. The following sections outline some of the advanced configuration options that are available. For more information, see these topics: Appliance management Data monitoring Foglight Experience Monitor and Ixia Net Tool Optimizers 28
Appliance management Alarm Email Notification you can configure groups of email addresses that will receive notifications when an alarm is triggered. For more information, see Email groups and Mail server. User Account Management you can create user accounts for each person you would like to have access to the appliance. Each account gives the user their own set of predefined reports and the ability to create additional customized reports. For more information, see User account management. Security Profile for organizations that required enhanced security for user account management, you can implement a security profile that separates user account management and access privileges between users of the web console and network or system administrators. For more information, see Increasing security for user account management and access privileges. Data monitoring Data Collection Policy configure the appliance to apply a set of rules (or transformations) to the URLs that it is monitoring. This ensures that the Pages and Hits categories contain the level of granularity appropriate for your site. For more information, see Transforming monitored URLs. Secure HTTP if any of your monitored servers use SSL, and you did not provide the keys to the appliance during the Setup Wizard, this step can be performed using the web console. For more information, see Configuring SSL keys. Server Service Levels the appliance provides the ability to define service level thresholds for the processing time consumed by each server. These thresholds can be modified to match the service levels desired by your organization. For more information, see Changing monitored servers service levels. SOAP the appliance can monitor SOAP activity that utilizes HTTP or HTTPS as transport protocols. A sequence of SOAP operations can be defined to form a transaction. For more information, see Monitoring SOAP applications and Configuring enterprise-wide service levels. Services a service, a sequence of web pages, can be defined and monitored by the appliance. For more information, see Managing services definitions. User Sessions appliance administrators can define how individual user sessions are identified by indicating end-user points of entry for web applications that are being monitored. For more information, see Identifying user sessions. Application Components web-based applications can be defined and monitored as a single entity by the appliance. For more information, see Managing application components. Installing and configuring and Managing applications explain all of these configuration options in detail. Foglight Experience Monitor and Ixia Net Tool Optimizers The Ixia Net Tool Optimizer (NTO) portfolio of smart taps provides aggregation, advanced filtering, load balancing, complex routing, and de-duplication of network traffic for monitoring devices like Foglight Experience Monitor. Some NTO devices also provide a feature known as hardware timestamping, that can help improve the overall accuracy of metrics reported by Foglight Experience Monitor. When the hardware timestamping advanced feature is enabled, NTO devices append a trailer containing a timestamp to each packet. The timestamping feature can use either a local time source or a Network Time Protocol (NTP) server. It is highly recommended that you use the NTP option with the same NTP server that is configured on your Foglight Experience Monitor system. Make sure to enable the hardware timestamping for each port on the Ixia device that is feeding traffic to Foglight Experience Monitor. 29
Consult your local Ixia sales representative to determine which NTO device would be best for your environment. For a description of how to enable the appliance to use hardware timestamps, see Configuring server options. 30
2 Multi-appliance clusters Some environments require the deployment of a group of appliances due to the volume of traffic to be monitored or because traffic cannot be collected from a single location on the network. This section explains how to setup, configure, and manage multi-appliance clusters for users who need to deploy them. NOTE: If you have only a single stand-alone appliance installed at your location then this section is not applicable. For more information, see these topics: About multi-appliance configurations Setting up High Availability (HA) mode Aggregation of metrics and configuration Switching from stand-alone to clusters About multi-appliance configurations An appliance can be configured to monitor up to 300 distinct servers, but depending on the volume of traffic for each server this could be too much load for a single appliance. In these cases, multiple probes are deployed that can each monitor a share of the total traffic. Additionally, there may exist organizations that use multiple data centers to serve up the same application. In either of these cases, a multi-appliance configuration is required. A multi-appliance cluster centers around the portal, which acts as the hub through which all other appliances in the cluster communicate. These other appliances, referred to as probes, are each configured to monitor a set of server IPs which is typically distinct from the server IPs configured for the other probes. Portals, probes and stand-alone appliances all utilize the same basic hardware platform. During the Setup Wizard, you can choose to configure an appliance to serve the role that has been assigned to it in your deployment plan. 31
Figure 13. Type page During initial configuration using the Setup Wizard, you are asked to indicate whether the appliance will act as a portal, probe or as a stand-alone. For the end user viewing metrics and reports, there is no difference between what they see when browsing the web console of either a probe, portal, or stand-alone. The user interface has the same look-and-feel and functions the same way. The data itself is different because a portal shows an aggregated view of the metrics collected on all of the probes reporting to it. By logging into the web console on a probe, the user sees only the metrics collected by that probe, not the aggregate metrics that the portal displays. End users should retrieve reports from the portal in a multi-appliance deployment. NOTE: User accounts and report sets are not synchronized in a multiple appliance cluster. If a particular user needs to log in to all appliances, you must create a separate user account for this user on each of these appliances. Setting up High Availability (HA) mode If you are setting up the appliance as a portal, you need to determine if you are going to setup monitoring probes that either report to the portal using high availability mode or as a regular probe. HA mode enables you to designate pairs of probes that provide redundancy in the event of a catastrophic failure of one of the systems. Both probes should be configured to monitor identical streams of traffic coming to your site. One probe, which is designated as the primary probe, is the preferred source of metrics by the portal. The other probe, which is designated as the secondary probe, is utilized by the portal in the event that the primary probe fails to communicate with the portal, no matter the reason. When the primary probe returns online, the portal switches back to that probe as the preferred source of metrics for the pair. However, if the primary server restarts, the appliance portal will accept data from this server. You need to configure your primary and secondary probes to monitor the same set of servers and ports to ensure that the metrics they are collecting are identical. While operating in HA mode, both the primary and secondary probes are collecting traffic and generating metrics, which are transmitted to the portal. However, the portal only accepts one data feed, preferably from the primary. Again, if the primary probe fails to communicate, the portal will utilize the data feed from the secondary probe. This capability allows you to protect yourself against a catastrophic failure in one of the probes. For more information, see these topics: Configuring Probes for High Availability mode Deleting High Availability configurations 32
Configuring Probes for High Availability mode To configure probes for High Availability mode: 1 Navigate to Configure > Appliance > High Availability. By default, any existing configurations are displayed. 2 Click the Add Pair link to create a new configuration. 3 From the Primary list, select a server IP that you want to act as the Primary probe. 4 From the Secondary list, select a server IP that you want to act as the Secondary probe. 5 Click OK. NOTE: After creating a configuration, the probe IPs that you selected from the Primary and Secondary lists no longer displays in the list. Deleting High Availability configurations To delete a High Availability configuration: 1 From the High Availability page, select the check box beside the configuration that you want to delete. 33
2 Click Delete to remove the configuration. Aggregation of metrics and configuration Multi-appliance configurations are, for the most part, transparent to the end user. When viewing and building report sets, end users are never exposed to the concept of portals or probes. Any exceptions are outlined in Storing report sets. Appliance administrators should have an understanding of how multiple appliances share metric data and configuration settings, and in what ways the multi-appliance concept is hidden from end users. For more information, see these topics: Aggregating metrics Configuration settings Storing report sets Aggregating metrics In a multi-appliance cluster, all metrics collected by probes are pushed to the portal at the end of each fiveminute interval. The portal then aggregates the data in real-time and stores it in its database. This architecture guarantees that there is always a single set of consolidated data that represents all of the traffic that was collected by the probes in the cluster. Figure 14. Aggregating metrics Metrics are aggregated in a multi-appliance cluster using the following procedure: 1 Probes analyze network traffic and generate metrics. 2 Every five minutes, all probes push their metrics and alarms up to the portal. 3 On the portal, a process called the consolidator responds and verifies that each of the probes have communicated with the portal as expected. If one or more of the probes, have not reported in, the portal delays consolidating the data for an additional five minutes. This safeguard is built-in to ensure that temporary interruptions of communications do not affect the aggregated data saved on the portal. 4 When the metrics from all the probes have been collected, the metrics are merged by the consolidator and stored in the local database on the portal. A subset of this merged, or unified, data is also then exported to Foglight by the Cartridge for FxM. IMPORTANT: If the portal is off-line for any period of time, the data collected by probes during that time may be lost. Gaps can appear in reports generated in the portal's web console. 34
Configuration settings End users who browse the web console of the portal see a unified set of data representing the traffic collected by all of the probes in the cluster. This notion of the central data portal for end users does not always apply to the configuration settings for the portal and probes in the cluster. While most of the configuration settings are shared by all appliances in the cluster, there are some settings that are specific to each probe: The servers that are monitored (configured on the Configure > Monitoring >Servers page). The ports used by protocols including HTTP and HTTPS protocols (configured on the Configure > Monitoring > Protocols page). All network settings (configured on the Configure > Appliance > Network Settings page). All user account settings (configured on the Configure > Appliance > User Accounts page). The backup configuration settings (configured on the Configure > Appliance > Backup & Restore page). All security settings (configured on the Configure > Appliance > Security page). The Foglight Experience Viewer settings (configured on the Configure > Appliance > Network Settings page). For all other configuration settings, administrators can connect to any appliance, including the portal, to make changes to those settings. NOTE: Auto-discovery functions that are present in a number of areas in the configuration are not available on the portal since these require a live feed of network traffic to work. All configuration settings, whether shared or probe-specific, are stored in a MySQL database on the portal that is accessed by the probes using the MySQL protocol. Figure 15. Configuration settings sharing No matter where configuration settings are made, they are shared across the cluster. As shown in the diagram above, probes poll the configuration database on the portal every five minutes to receive any changes that have been made. This means that there could be up to a 10 minutes delay before configuration changes are reflected in the collected metrics. If the portal goes offline, the probes utilize a local cached copy of the configuration database that reflects the last known state of the centralized configuration database on the portal. The probes can even be rebooted when a portal is down, and continue to use their own respective local copies of the configuration. 35
Storing report sets Unlike most configuration settings, report sets are not aggregated at a central location, they are stored on the appliance on which they were created or modified by the appliance user. This means that a user who creates a report set locally on a particular probe does not see it if they log in to another probe or the portal. Generally, this should not be a concern since users should always be directed to use the portal to access metrics collected by the cluster. In the event that a user unknowingly creates report sets on a probe, and is reluctant to manually rebuild them all on the portal, you can use report packages to migrate the user s report sets. To migrate report sets: 1 Log in to the probe where you want to move the report sets. 2 Navigate to the Monitoring > Customize Report Sets page. 3 Click Distribute All Report Sets, located at the bottom of the page. 4 From the list of users, select the check box beside the users to whom you want to distribute your reports. 5 Click Done. 6 From the Customize Report Sets, click Create Report Set Package. The Create Report Set Package window appears, displaying a package file link. 7 Right-click the link, and save the file to your system. 8 Log in to the portal. 9 Go to Customize Report Sets page. 10 Click Reset Report Sets. 11 In the Reset Report Sets window, select the use a report package option, and locate the file you saved in step 7. 12 Click Submit. For more information about report sets, see the Distributing report sets and Resetting and copying a report set in the Foglight Experience Monitor User Guide. Switching from stand-alone to clusters You may need to switch from a stand-alone machine to a probe as part of a cluster. There are two likely reasons why you might need to make this change: You are using a single stand-alone appliance. Due to growth in your server farm and web traffic, you wish to upgrade to a multi-appliance cluster. You are using a multi-appliance cluster, and also have a few stand-alone appliances deployed on your network. You would like to take an existing stand-alone appliance, and promote it into the multiappliance cluster. The following section (Creating a cluster from a stand-alone appliance) describes the steps you must take to ensure the best possible outcome for your existing metric data and configuration settings. 36
Creating a cluster from a stand-alone appliance When converting a stand-alone appliance deployment into a multi-appliance deployment, you can promote the stand-alone to a portal. Metrics are retained during this process but the configuration is not retained. If you want to retain the configuration, follow these steps: 1 Back up your configuration settings (see Backing up and restoring data). 2 Change the appliance type from stand-alone to portal (see Configuring the Appliance Type). 3 Restore the stand-alone configuration settings on the newly created portal (see Restoring a backup). Once your portal is completely established with the original configuration settings, and after you have installed the two new appliances as probes, the configuration settings propagate throughout the cluster. Merging a stand-alone appliance with an existing cluster If you are transitioning a stand-alone appliance into an existing cluster, you will lose all of its configuration settings. In most cases, this consequence is irrelevant since the stand-alone appliance s transition into the cluster implies the destination cluster s settings and data collection policies take precedence over what currently exists on the stand-alone appliance. The stand-alone appliance s metrics are preserved, and are available through the web console. However, they do not reflect the configuration settings of the cluster and will likely have some inconsistencies. For this reason, it is recommended that you purge metrics at this time. See Database configuration. After you change the appliance type to probe, and link it with the existing portal, all cluster-wide configuration settings are accessed from the central database. See Configuring the Appliance Type. Some appliance settings are not global, and therefore are not shared from the central portal s MySQL database. These settings (ports and servers) are unique to each probe, and need to be configured. For more information, see Specifying monitored web traffic. 37
3 Configuring the appliance The following chapter covers configuration options that affect the appliance itself, as well as how it functions as a part of your network environment. For more information, see these topics: Network settings User accounts Security settings on the appliance Report settings Database configuration Appliance integrity Updating the appliance Backing up and restoring data Network settings You can access the Foglight Experience Monitor network configuration page by clicking Configure > Appliance > Network Settings. After you are finishing configuring the settings, click Apply. Figure 16. Network Settings page For more information, see these topics: 38
Foglight Mail Network Time Servers (NTP) SNMP Other Foglight Use the Foglight section of the Network Settings page to establish how Foglight Experience Monitor communicates with the Foglight Management Server and Foglight Experience Viewer. Foglight server IP Users of the Foglight Management Server browser interface have the ability to navigate into the Foglight Experience Monitor web console to view reports containing metrics and information that is not available in the Foglight Management Server browser interface. Those users will be prompted for a login name and password when they enter the Foglight Experience Monitor web console, unless they have enabled single sign-on with FxM. To enable the single sign-on with FxM feature, enter the IP address of the Foglight Management Server in the Foglight Server IP box on this page. If there is more than one Foglight Management Server that is receiving metrics from this appliance, each IP address should be entered in this box, separated by a comma. Foglight server port In the Foglight Server Port box, type the port that the Foglight Management Server uses for access to its browser interface. The default port value is 8080. Foglight Experience Viewer server IP In the FxV Server IP box, type the IP address of the Foglight Experience Viewer Server that you want to link to this appliance. Foglight Experience Viewer server port In the FxV Server Port box, type the port that the Foglight Experience Viewer Server uses. This value should always be set to default HTTP port value (80 or 7630). IMPORTANT: To ensure a secure communication with the FxV Server, you must also log in to the FxV browser interface, navigate to Configure > Superuser Tasks > Server Configuration, and set SSL Redirection Enabled to Yes. This setting affects all the web console traffic, and the FxM drilldown links are automatically redirected to the HTTPS port (by default, 443 or 7643). Mail Use the Mail section of the Network Settings page to establish how Foglight Experience Monitor sends email notifications and distributes reports. 39
Mail server In the Mail Server IP box, type the IP address of an SMTP mail server that the appliance can use to send email notifications. Email notifications can be assigned as actions that occur when an alarm is triggered. For more information about email notifications, see Sending email to a group in the Foglight Experience Monitor User Guide. Reports can be configured for distribution to a group of email address. For more information about report distribution, see Distributing report sets in the Foglight Experience Monitor User Guide. Email field In the Email from field box, type the full email address of the sender that the appliance uses to send email notifications, for example fxmalarm@dell.com. When users receive email notifications as a result of an alarm, the from field appearing in the email message is the string you enter in this box. For more information, see User account management. Network Time Servers (NTP) In the NTP Server boxes, enter the IP addresses of up to three Network Time Protocol (NTP) servers. This ensures that the system clock on the appliance accurately reflects the current time. This is especially important for multi-appliance clusters. For more information on NTP including a list of publicly accessible time servers, see http://www.ntp.org. SNMP SNMP provides the ability to send traps (notifications) to advise an administrator whenever an alarm is triggered. A trap is a network message that is used to report alerts about a managed subsystem. In the SNMP Server boxes, enter the IP addresses of up to three SNMP (Simple Network Management Protocol - version 2c) servers on your network. After you have configured at least one SNMP server you will be able to send SNMP traps whenever an alarm is triggered. For more information, see Issuing SNMP traps in the Foglight Experience Monitor User Guide. You must also enter the community identifier that the system uses to send SNMP traps. The two links, at the bottom of the page, give you access to the SNMP (Version 2) Structure of Management Information file and the Management Information Base (MIB) file. Other This section contains additional network configuration settings. Syslog server In the Syslog server IP box, type the IP of a server running syslog that receives all of the configuration log messages that are generated on the appliance whenever a change is made to configuration. This facility allows you to monitor configuration changes that are made on the appliance to detect possible violations of your site's security or administrative policies. Traceroute protocol Traceroutes can be assigned as actions that occur when an alarm is triggered. For more information, see Perform traceroutes in the Foglight Experience Monitor User Guide. 40
You can choose either UDP (User Datagram Protocol the default) or ICMP (Internet Message Protocol). To determine which protocol your network supports for traceroutes, contact your network administrator. User accounts The User Accounts page enables you to manage user accounts and user account options. You can access this page by clicking Configure > Appliance > User Accounts. The appliance provides personal user accounts for each individual who needs access to it through the web console, terminal, remote database, or secure shell (SSH). These accounts specify the login name and password for each user, as well as the granted privileges. Each user account also has its own package of report sets that users may customize to fit their requirements. For Foglight users who want the ability to view reports in the appliance, you must set up a user account that matches the login name used in Foglight. To complete the integration process, you must also setup the Foglight server IP and port on the Configure > Appliance > Network Settings page. For more information, see Foglight Management Server. For more information, see these topics: User account management Global user account options Increasing security for user account management and access privileges User account management In order to connect to the Foglight Experience Monitor console to view Report Sets, create alarms, and view metric data, users require accounts. These are managed on the User Accounts page, accessed by clicking Configure > Appliance > User Accounts. Each user account offers a private workspace in which the user can manage and customize a personalized group of report sets and snapshots. Figure 17. User Accounts page The SSH Access and DB Access columns provide a quick way to determine whether any users have SSH or database access enabled, in which case SSH port (22) or SQL database port (3306) are currently open on the appliance. 41
Defining login names It is strongly recommended that you create individual user accounts rather than reusing the default admin account. If more than one user logs into the admin account, conflicts may arise. NOTE: If a Foglight user wants to view reports on the appliance, you need to create a Foglight Experience Monitor user account with the same the login name as the Foglight user account. For more information, see User accounts. A valid login name must adhere to the following guidelines: Begins with a letter or underscore Contains only letters, digits, underscores, dots, hyphens, or dollar signs Is unique Is different than the default Linux user accounts that are pre-installed on your appliance. Table 5. Default Linux user accounts agent bin console daemon dbuser haldaemon install ldap lost+found man messagebus nobody ntp root snort sshd suse-ncc toor uccp vconfadmin vconfuser www wwwrun Adding user accounts To create a new user account: 1 On the User Accounts page, in the Users section, click Add a User. The User Accounts: Add page is displayed. 2 In the Login box, type the login name. For more information, see Defining login names. 3 In the Password and Confirm Password boxes, type the password assigned to the user. 42
TIP: A message appears to the right of the Password box, providing information about the strength level of the password you enter. The password creation rules you must follow depend on whether the Enforce strong passwords option is enabled on the main User Accounts page. For a description of the strong and standard policy, see Global user account options. 4 Select an option from the Type list. Table 6. User account types Type Administrative Power User Secured Power User General Guest Access Level Unlimited access. Assign this role only to users who are responsible configuring, maintaining, and updating the appliance, the database, the appliance software, and user accounts. Only Administrative users can have terminal and SSH access enabled. Power Users have access to all features except those related to configuring, maintaining, and updating the appliance, the database, the appliance software, and user accounts. In the web console, the Configure > Appliance menu (which contains the restricted tasks) is removed. Similar to the Power User role except that Secured Power Users cannot run autodiscovery functions that can expose sensitive customer data. This includes session identifier auto-discovery, login variable auto-discovery, and variable rule auto-discovery. Assign this role to users who need to do the following tasks only: Configure alarms Customize the report sets in their accounts View existing report sets View metrics, resources, and alarm definitions, depending on the user s profile settings (see Global user account options) NOTE: General users cannot distribute reports sets to other users. Assign this role to users who need to do the following tasks only: View existing report sets View metrics, resources, and alarm definitions, depending on the user s profile settings (see Global user account options) 5 To allow this user to log in to the web console, select the web console access enabled check box. 6 To allow this user to establish an SQL connection with the appliance database, select the Remote database access enabled check box. 7 To allow this user to log in using a physical terminal (or KVM device) attached to the appliance, select the Terminal access enabled check box. 8 To allow this user to establish an SSH session with the appliance, select the Secure shell (SSH) access enabled check box. NOTE: This option is enabled only if the user has a strong password and terminal access enabled. 9 To allow this user to view User Session metrics (that is, metrics gathered about users who are accessing applications that the appliance is monitoring), select the Allow access to User Session metrics check box. NOTE: Selecting this option allows the user to see the User Session Log in the web console. 10 If the user requires a Section 508-compliant user interface, select the Use Section 508 compliant settings check box. 43
11 Select one of the following options: a b NOTE: Section 508 refers to the amendment of the American Rehabilitation Act, which calls for usability of software user interfaces for those who are visually impaired. When enabled, use of the web console does not require a mouse. Use default report sets copy from user If you selected option b, select a user from the list from which you would like to use the report sets. 12 Click OK to save the new user profile. Exporting user accounts When adding new appliances to your network environment, you can export your user accounts from an existing appliance and then import them into the new appliance. For more information, see Importing user accounts. To export the accounts: 1 Click Configure > Appliance > User Accounts. 2 In the User Accounts page, click Export Users. 3 Right-click the Configuration File link and save the ZIP file to a local drive. Importing user accounts When adding new appliances to your network environment, you can export your user accounts from an existing appliance and then import them into the new appliance. For more information, see Exporting user accounts. To import the accounts: 1 Click Configure > Appliance > User Accounts. 2 In the User Accounts page, click Import Users. 3 Click Browse. Navigate to and select the ZIP file you exported from the other appliance. 4 Click Import. Removing user accounts Existing user accounts can only be removed by a user whose own account is of type Administrative. To delete user accounts: 1 Locate their name on the User Accounts page. 2 Select the check box beside their name. 3 Click Delete. Editing user account settings Existing user accounts can only be edited by a user whose own account is of type Administrative. 44
To edit user account settings: 1 On the User Accounts page, click the Edit link beside the account name, in the Settings column. The User Accounts > Edit page is displayed. 2 In the Login box, type the login name. For more information, see Defining login names. 3 Select an option from the Type list. For more information, see User account management. 4 To allow this user to log in to the web console, select the web console access enabled check box. 5 To allow this user to establish an SQL connection with the appliance database, select the Remote database access enabled check box. 6 To allow this user to log in using a physical terminal (or KVM device) attached to the appliance, select the Terminal access enabled check box. 7 To allow this user to establish an SSH session with the appliance, select the Secure shell (SSH) access enabled check box. NOTE: This option is enabled only if the user has a strong password and terminal access enabled. 8 To allow this user to view User Session metrics for the user account (metrics gathered about users who are accessing applications that the appliance is monitoring), select the Allow access to User Session metrics check box. NOTE: Selecting this option allows the user to see the User Session Log in the web console. 9 If the user requires a Section 508-compliant user interface, select the Use Section 508 compliant settings check box. NOTE: Section 508 refers to the amendment of the American Rehabilitation Act, which calls for usability of software user interfaces for those who are visually impaired. When enabled, use of the web console does not require a mouse. 10 Click OK to save the edits made to the user s profile. Unlocking user accounts A user account becomes locked if the user fails to login six consecutive times. The lockout duration is 30 minutes. Only a user with administrative privileges can unlock the account for you before the lockout period ends. 45
To unlock the user s account: 1 On the User Accounts page, click Edit beside the account that has been locked, in the Settings column. The User Accounts > Edit page is displayed. 2 Click Unlock Account. A confirmation message appears at the top of the page: The account has been unlocked. 3 Click OK to save the edits made to the user s profile. Editing user account passwords Existing user account passwords can only be edited by a user whose own account is of type Administrative. To edit user account passwords: 1 On the User Accounts page, click the Edit link beside the account name, in the Password column. The User Accounts > Change Password page is displayed. 2 In the Password and Confirm Password boxes, type the new password assigned to this user account. 3 Click OK. IMPORTANT: The password creation rules you must follow depend on whether the Enforce strong passwords option is enabled on the main User Accounts page. For a description of the strong and standard policy, see Global user account options. Global user account options The User Accounts page > User Account Options section provides several global settings that affect all web console users: Restrict guest account navigation (see Restricting guest accounts). Enforce strong passwords (see Configuring strong passwords). Expire passwords after X days (see Configuring the password expiration). Expire web console, terminal, database and SSH session after X minutes of inactivity (see Configuring the session expiration). Minimum password length (see Configuring the minimum password length). 46
Restricting guest accounts Section User account management presents the three main categories of appliance users (Administrative, General, and Guest) and their corresponding system types. Guest accounts are typically created for users whose only interest is viewing report sets that have been created by others. By default, the Restrict guest account navigation option on the User Accounts page is disabled. This means the user s menu options expand to include the viewing of metrics, resource lists, and the alarm log. Figure 18. System Report Sets page When Guest type users are not restricted, they can also view resources, metrics, and the alarm log. If the Restrict guest account navigation option is enabled, any user whose profile is of type Guest is only able to view report sets. Other commands normally accessed via the menus are not available. Figure 19. Current System Status page As is the case with General type users, Guest users cannot create report sets; however, while General users are able to create alarm profiles, Guest users still do not have this feature available to them. Configuring strong passwords You can enforce the use of strong passwords for all user accounts. On the User Accounts page, the Enforce strong passwords option allows you to require strong passwords whenever new accounts are created or existing users modify their passwords. If you modify this option after some user accounts have been created with passwords that are not considered strong, those users are still be able to log in to the system. The default password only requires that passwords consist of five characters. The strong password policy requires that passwords do not contain the user account name, and must contain characters from at least three of the four following character types: upper case, English alphabetic characters: A-Z lower case, English alphabetic characters: a-z 47
base 10 numbers: 0-9 non-alphanumeric characters: i.e., $, #, or % Configuring the password expiration The password expiration policy causes user passwords to expire after a defined period of time. An administrator can configure the password expiration length (number of days). If a user attempts to log in to the web console using an expired password, a warning message appears; the user is authenticated and prompted to change the password immediately. If a user attempts to log in through the attached terminal using an expired password, the following message appears: Your password has expired. Choose a new password. Authentication token manipulation error. If a user attempts to log in through SSH using an expired password, he is not authenticated and, therefore, cannot change the password that way. The system displays the following message: Warning password expired. You must change your password now and login again! Changing password for <user>. In either situation, users must log in through the web console to change an expired password or have an administrator change the password for them. NOTE: Administrators can always change the password of a user whose password has expired. By default, the Expire passwords after X days option on the User Accounts page is disabled. Configuring the session expiration A session expiration policy can be configured for the web console, SSH, and terminal sessions in order to force sessions to terminate after a defined period of time. An administrator can configure this expiration period. In the web console, when a session expires, users are redirected to the login page, where they must re-enter their login name and password. Terminal sessions will terminate and the user will be presented with a login prompt. SSH sessions and database are dropped. By default, the Expire web console, terminal, database and SSH session after X minutes of inactivity option on the User Accounts page is disabled. Configuring the minimum password length Administrators can define a policy to enforce a minimum length for passwords for all user accounts. By default, the Minimum password length option is set to 5. Increasing security for user account management and access privileges Some organizations require enhanced security for user account management. In these environments, you can activate the appliance s security profile, called lockdown mode. This optional mode separates user account management and access privileges between users of the web console and network or system administrators. 48
Changes implemented by lockdown mode Lockdown mode implements the following changes to the appliance software and the web console. User account changes: Splits user account management. The user accounts setup and support are now special administrative accounts that cannot be managed through the web console. Restricts access to the console program to the setup account only. Restricts SSH access to the support account only. Adds Change Setup Account Password and Change Support Account Password options to the account management page for the setup account. Console program changes: Removes the option to grant web console access to the currently logged in user by hiding the Enable/Disable Web Access option. Removes the ability to access the shell by hiding the Remove Shell Access From Console and Access Shell options. Modifies the Enable/Disable SSH option so that it is only available for the support account. Web console changes: Excludes the special administrative accounts from the user account management pages. Removes the options to enable database, terminal, and SSH access for all accounts. Directory changes: Saves support bundles, Dell E-Support Tool reports, and Raid Status and System Health screen outputs to the home/support directory. Activating lockdown mode You activate lockdown mode by running a script from the command line. IMPORTANT: Once activated, this mode cannot be deactivated. To activate lockdown mode: 1 Before running the script, ensure that the existing user accounts are in the following state: The setup user account exists. The account must have console program access enabled and web console access disabled. The support user account exists. The account must have SSH access enabled. All other user accounts have only web console access enabled (no access to the console program). 2 Log into the console program as the setup user. 3 Select Advanced Options, then More Advanced Options. 4 Select Access Shell. 5 On the command line, run: /usr/local/ecrit/lockdown.sh 6 When prompted for confirmation, select Yes to continue. Lockdown mode is now enabled. 7 Run a new backup. For more information, see Backing up and restoring data. NOTE: If you attempt to restore a backup created before lockdown mode was enabled, it causes issues. To avoid confusion, you may want to discard the pre-lockdown backups. 49
Accessing the console program remotely After activating lockdown mode, the setup account no longer has SSH enabled, and the support account no longer has access to the console program. Therefore, the process for remotely accessing the appliance and its console program changes. Now when using SSH, you must log in as the support user, and then at the command line, switch to the setup account to launch the console program. To access the console program remotely: 1 Use SSH to access the appliance. For instructions, see either Logging in from Microsoft Windows or Logging in from Linux and UNIX. 2 At the login prompt, log in as the support user. A command prompt appears. When logged in as support, there is no access to directories and an extremely limited ability to run commands. 3 On the command line, type: su setup 4 Log in as the setup user. The lockdown version of the console program starts. For more information, see Changes implemented by lockdown mode. Security settings on the appliance You can manage security settings for the appliance on the Security page accessed by clicking Configure > Appliance > Security in the menu. Figure 20. Security page From this page you can configure the appliance web server to use Secure Sockets Layer (SSL). NOTE: If your organization requires enhanced security for user account management, see Increasing security for user account management and access privileges. For more information, see these topics: Configuring the web server to use SSL Disabling the use of SSL Configuring the web server to use SSL This section outlines how to configure the appliance s web server to use Secure HTTP. To configure the appliance to monitor secure HTTP traffic with the web servers it is monitoring, see Configuring SSL keys. 50
To configure the web server to use SSL: 1 In the SSL Certificate Key File box, click Browse and locate the key file. 2 In the SSL Certificate File box, click Browse and locate the certificate file. 3 Once both files have been located, click Enable SSL to switch the appliance s web server to use Secure Socket Layer. IMPORTANT: From this point on, in order to access the web console, you must use HTTPS in your web browser address box instead of HTTP. Disabling the use of SSL Use the Disable SSL button in the Configure Web Server or SSL section to reset the appliance s web server to HTTP. Report settings You can configure report settings using the pages found on the Configure > Reporting menu. You can customize how metrics display in Report Sets by configuring the pages contained in the Report menu. Each report, whether it consists of a chart or table, reports on specific metrics collected by the appliance. Table 7. Pages available for configuring report settings Page Categories Distribution Metrics Email Groups Foglight Metrics Scheduled Reports Time Frames User Agents Options Details For more information, see Metric categories. For more information, see Distribution metrics. For more information, see Email groups. For more information, see Foglight metrics. For more information, see Scheduled reports. For more information, see Time frames. For more information, see User accounts. For more information, see Baseline calculations, Percentile metrics, and Configuring enterprise-wide service levels. For detailed information, see these topics: Email groups Adding email groups Scheduled reports Metric categories Database metrics Baseline calculations Time frames Distribution metrics Percentile metrics Foglight metrics User agent transformations 51
Email groups An email group is a collection of email addresses for people who may or may not have user accounts. Email groups are used for sending alarm notifications (see section Sending email to a group in the Foglight Experience Monitor User Guide) and routing scheduled reports (see Scheduled reports). The Email Groups page, accessed by clicking Configure > Reporting > Email Groups, allows you to manage email groups in your system. Figure 21. Email Groups page Adding email groups To add a new email group: 1 On the Email Groups page, click Add an Email Group. The Email Groups > Edit page is displayed. 2 In the Email Group Definition section, in the Name box, enter a self-explanatory name for the email group. 3 In the Email Addresses section, enter an email address in the text box, then click Add to include it in this email group. 4 Repeat Step 3 for all email addresses to be included in this email group. 5 Click OK to save the email group settings. Editing email groups On the Email Groups > Edit page, you can edit the name and membership list of an email group. IMPORTANT: Links between an alarm and an email group are maintained when you modify the email group name. 52
To edit an email group: 1 On the Email Groups page, click a group s corresponding Edit link. The Email Groups > Edit page is displayed. 2 To modify the user group name, update the group definition in the Name box. 3 To modify the membership of this user group, add or remove email addresses from the list, as necessary. To add an email address, enter the address in the Email Address text box and click Add. To remove an email address, select its corresponding check box and click Delete. 4 When you finished making changes, click OK to save the new group settings. Removing email groups When viewing entries on the Email Groups page, you can delete a group by selecting its corresponding check box, then clicking Delete. Scheduled reports A scheduled report is a report defined to be automatically generated and distributed to select users, via email. This capability allows you to email Adobe Portable Document Format (PDF) files containing selected report sets to a list of defined recipients. Users who receive reports do not need user accounts or login credentials since they are receiving only PDF files containing the information you want them to see. Managing scheduled reports The Scheduled Reports page, accessed by clicking Configure > Reporting > Scheduled Reports, allows you manage scheduled reports in your system. Figure 22. Scheduled Reports page You can schedule a new report for distribution by clicking the Add Scheduled Report link. You can edit an existing scheduled report by clicking its corresponding Edit link. For more information, see Scheduling a report for distribution. You can delete an existing scheduled report by selecting its corresponding check box and clicking Delete. You can test an existing scheduled report by clicking its corresponding Test link. A report is immediately sent out to the defined recipients. The Test in progress message is displayed at the top of the page while the test is being performed. Scheduling a report for distribution On the Scheduled Reports > Edit page, you can edit the distribution settings for a scheduled report. 53
Figure 23. Scheduled Reports: Edit page To edit a scheduled report: 1 From the User Account list, select the user account containing the report you wish to distribute. 2 From the Report Set list, select the report set that should be distributed to the recipients. The list of report sets shown is obtained from the current set of reports for the user account specified. 3 From the Recipients list, select the email group whose members should receive the report distribution. 4 Select the Enable check box to enable the report generation. 5 Define the schedule for report generation. You can select whether the report is generated hourly, on specified days of the week, or monthly. 6 Click OK to save the scheduled report settings. Metric categories By deselecting categories and clearing the appropriate check boxes, you are disabling the collection of data for certain metric categories. The settings for this page are global and therefore affect all appliances that are part of a multiple appliance cluster. To select the metric categories that the appliance will gather metrics for, navigate to Configure > Reporting > Categories. By default, most of the metric categories are enabled for collection. Ordinarily, you do not need to disable any categories but there may be cases where a certain category is not particularly useful for your applications. In this case, disabling the category may improve overall system performance and responsiveness, especially if the category is creating a large amount of entries in the database. Categories that do not appear in this list cannot be disabled. You can turn off particular categories that you do not find useful. The following list of categories can contribute most to rapid database expansion: Application Component by City Hit Page Path Service by City Service Step by City 54
These six categories are limited to 100,000 entries that can be loaded into the database for a five minute interval. This is a safety mechanism designed to prevent a database from reaching a threshold capacity that can sometimes occur due to errors in configuration. Records beyond the 100,000 limit are discarded. Figure 24. Categories page Database metrics You can view metrics for each category s database using the System Database category. For more information, see System database in the Foglight Experience Monitor Metric Reference Guide. 55
Baseline calculations The appliance can calculate baseline metrics that provide a historical perspective for a specific metric of a given resource. Figure 25. Example of baseline calculation Users can choose to include a baseline in reports by clicking Show Baseline. Baseline calculation is configured on the Baselines page, which is accessed by clicking Configure > Reporting > Options in the main menu. Figure 26. Baseline calculation options Baseline calculations can be based on: hourly data daily data weekly data monthly data To change a baseline calculation, enter an amount of time in the appropriate box, indicating the number of preceding units of time the baseline calculation includes. For weekly and monthly baselines, the calculation uses all of the preceding time periods. For example, if you have configured four months in the Monthly baseline calculation text box, a monthly baseline displayed in May would include the aggregation of data from January, February, March and April. Similarly, for weekly baselines the baseline calculation includes the preceding number of weeks specified in the Weekly baseline calculation text box on this page. Hourly and daily baselines work differently in that the time period is considered significant. For example, an hourly baseline calculation for 10:00 utilizes only the metrics for the 10:00 hours found in the preceding 168 hours, assuming you have retained the default setting of 168 hours. Similarly, daily baselines calculations includes metrics only from the same day. For example, the calculation for a Friday baseline includes only the metrics for each Friday found in the preceding 32 days, again, assuming you have retained the default setting of 32 days. After modifying baseline calculation values, click Apply to retain the changes. The next time a report set is viewed, the new baseline calculation appears. 56
Time frames Each user account has its own package of report sets that end users can configure to customize the report for their needs. For more information, see the Foglight Experience Monitor User Guide. Selecting time frames The Edit Report Set page allows end users to display the metric data for specific time frames. Figure 27. Edit Report Set: Metric Chart page If the time periods included with the appliance do not offer the window of time needed to analyze metrics, you can define your own. For more information, see Configuring time frames for reports. Configuring time frames for reports The Time Frames page, accessed by clicking Configure > Reporting > Time Frames, lists all time frames that are available when building reports sets and using the Metric View. Figure 28. Time Frames page You can add a new custom time frame by clicking the Add a Time Frame link. You can edit an existing time frame by clicking its corresponding Edit link. For more information, see Editing a time frame. 57
You can delete a time frame by selecting its corresponding check box and clicking Delete. Editing a time frame The Time Frames > Edit page allows you to define the time frame settings. Figure 29. Time Frames: Edit page In the Label box, enter a name for the time frame. This label appears in the Time Period lists on the Metric View page and when editing report sets. Ensure the label is unique and descriptive. Select one of the Source options that best defines your custom time frame. The first option allows you to define the time frame by number of periods. For example, a custom Last 2 Weeks time frame could be defined as the Last 2 periods of Weekly data. The second option allows you to build a custom time frame by unit of time, followed by the granularity of presentation. For example, a custom Last Full Week by Hour time frame could be defined as the Last full Week showing Hourly data. When you are finished configuring the information, click OK to save the definition, and return to the Time Frames page. Distribution metrics The Distribution Metrics configuration page, accessed by clicking Configure > Reporting > Distribution Metrics allows you to add and modify a distribution configuration for specific distribution metrics. Distribution metrics are used to display the exact breakdown (or frequency) of individual data points for metrics, such as End-to-End Time and Processing Time. 58
Editing distributions To configure a distribution you need to choose a name, a metric, and define the units and buckets. In the Name box, enter a self-explanatory name for the distribution configuration. From the Metric list, select the distribution metric. You can only choose from the list of existing distribution metrics. IMPORTANT: Each distribution metric has a default configuration except for the Access Speed Distribution which cannot be customized. When a distribution metric is edited, the changes apply to all metrics across all categories that use the distribution configuration. You can also create additional distribution configurations and use them when configuring Application Components, Services, SOAP Applications and SOAP Transactions. For each bucket, enter the start and end time in milliseconds. For example, the first bucket might start at 0 and end at 25. All data points between 0 and 25 milliseconds are counted in this bucket. Depending upon the level of granularity in which you want to view data, you can define the number of buckets and the range of each bucket to suit your needs. Buckets display the total count of individual data points that fall within the lower and upper ranges defined for the bucket. Click the Add a Bucket to create additional buckets or to remove the last bucket click Remove Last Bucket. Figure 30. Distributions: Edit page IMPORTANT: Each distribution configuration contains a version number. If a distribution configuration is modified, metrics for that distribution collected prior to the change can no longer be displayed in the web console. If you attempt to view a distribution metric, previously stored in the database in a time frame prior to the last configuration change, a warning message is displayed. IMPORTANT: Also note that modifying a distribution configuration causes the agent to be restarted, resulting in a five or ten-minute gap in metric collection that is visible in reports displaying five-minute interval data. NOTE: The maximum number of buckets that can be configured for a distribution is 100. Viewing distributions on related pages After a distribution is configured, you can view metrics from the related page. For example, the Metric View for Enterprise displays several distributions. The Command Initial Response Time Distribution displays three columns: unit (msecs), count and percentage. The unit column displays the bucket range, the count column displays the number of downloads for each bucket and the percentage column displays the percentage for each bucket of the total number of downloads. 59
Figure 31. Viewing distributions Percentile metrics The Percentiles section, accessed by clicking Configure > Reporting > Options, allows you to change the percentage of events that are included in percentile metric calculations. Figure 32. Viewing percentile metrics Percentile metrics are used to verify the quality of service for a specific event that is related to the end-user experience. Some examples of percentile metrics are Page End-to-End Time Percentile and Page Processing Time Percentile. Percentile metrics indicate the quality of service by reporting the maximum time required for an event type to occur, where the time reported represents the upper threshold for n% of all occurrences of the event over a given time frame. For example, if the Page End-to-End Time Percentile metric has a value of four seconds, this means n% of the pages downloaded for the monitored site were received by end users in less than four seconds. By default, the percentile level is set to 95%. This means the metrics for the aforementioned events report the time required for an event to complete for the 95th percentile. The system approximates the time required for n% of all events to occur by using the mean, standard deviation, and the standard normal variable. See Metric definitions in the Foglight Experience Monitor Metric Reference Guide for information about how these metrics are calculated. Changing the percentile level To change the percentile level, enter a value in the box on the Percentile Metrics configuration page, then click Apply. All percentile metrics are calculated using the new percentile level. Foglight metrics The Foglight Metrics configuration page, accessed by clicking Configure > Reporting > Foglight Metrics, allows you to select a subset of location data that is important to your configuration and export to the Cartridge for FxM in Foglight. 60
Figure 33. Foglight Metrics page Select from the five metric categories: city, region, country, ISP, and subnet information that you want to become available in Foglight. For each metric category, any associated application and service information is also exported to Foglight. For example, by selecting the city Reno, NV (US) the following metrics are exported to Foglight: City for Reno Application by City for Reno Service by City for Reno To configure Foglight metrics: 1 Select the metric category that you want to configure and click its Add link. For example to add a city click the Add City link. 61
2 Select a resource from the list or use the search box to look for a specific city. The resource is added to the City metric list. 3 Continue to add resources to each list that you want exported to Foglight. 4 Click Apply. User agent transformations The User Agents page, accessed by clicking Configure > Reporting > User Agents, allows you to create, modify, or delete rules that determine how user agent HTTP headers are transformed before being recorded in the User Agents metric category. The HTTP headers are sent by the browser to inform the web server about the identity of the browser. Whenever the system encounters a user agent string that does not match any of the defined rules, the original user agent string is recorded. By default, the page displays a number of preconfigured transformation rules that can be removed or modified by an Administrator. Figure 34. User Agents page The User Agents metric category gives a breakdown of user agents that accessed the monitored site from a variety of perspectives. These metrics can show you which user agent had the longest page processing time. 62
Figure 35. Example of Metric Analysis page The list of user agents displayed are retrieved from the HTTP headers of incoming session traffic. The appliance uses transformation rules that take the user agent string, and change it into a form that is more easily understood by end users. For example: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0) would be transformed into: Explorer 7.0 Adding a user agent transformation rule To add a user agent transformation rule: 1 Navigate to the Configure > Reporting > User Agents page and click Add User Agent Rule. The User Agents > Edit page is displayed. 63
2 In the Find box, type a string or regular expression that will be used to match against the user-agent HTTP header. Alternatively, click Select User Agent to select from a list of user agents that the appliance has encountered so far. This list above is populated with both transformed and non-transformed user agent strings. a b c d e Search for a specific user agent by typing in a query using the Search text box. If you need to search by Perl regular expression, check the Interpret as a regular expression option. Click Go. Select a User Agent from the returned list. Close the Resource List. 3 In the Display box, type a string that the system will use as the resource name for user agents matching the Find string. 4 Select the Interpret find string as a regular expression check box if you want the regular expression implemented in Perl. If you select this option, matching will be performed in a case-sensitive manner. Perl regular expressions can be made case-insensitive by adding the following characters at the beginning of the string (?!). If you do not select this option matching is performed in a case-insensitive manner. The system will match the Find string against any part of the user agent string. 5 Select the Extract version following the find string check box if you would like the user agent version number to appear with the agent name in the web console. 64
NOTE: By default, this check box is not selected. When this option is selected the match is based on the digits that appear after the search string. When there are no digits after the search string, within three characters, the system assumes that there is no version. For example, if a transformation rule s Find value is mozilla, and its Display value is Mozilla, Mozilla 4.0 and Mozilla 5.0 will appear as separate entries in the user agents list. 6 Select Applies only to major version numbers check box if the transformation rule you are creating should only be applied to user agent versions that fall within the defined range entered in the from and to boxes. 7 Click the Test link to test the Find string against the current list of User Agents. NOTE: Results displayed may include some strings that have already been transformed by existing User Agent rules. 8 Click OK to save the user agent transformation rule. The new user agent transformation rule takes effect immediately. Editing or deleting user agent transformation rules All existing transformation rules are listed on the User Agents page. To edit a user agent transformation rule, click its corresponding Edit link to display the User Agents > Edit page, where you can modify the string the rule matches against incoming user agent strings, the display name in the web console, or how user agent version numbers are handled. To remove an individual transformation rule, select its corresponding check box, then click Delete. To reset the default set of transformation rules, click Reset User Agent Rules (located at the bottom of the page). The current set of rules is dropped and then the default rule set is initialized. Using multiple transformation rules with different user agent version ranges The Has Version and the Applies only to major version numbers check boxes can be configured differently in multiple user agent transformation rules to differentiate user agent versions ranges whose performance you wish to evaluate separately or differently. For example, the appliance s default handling of the Mozilla web browser as user agent is as follows: Rule A: Find: mozilla Display: Netscape Has Version: yes Applies only to major version numbers: from 0 to 4 Rule B: Find: mozilla Display: Mozilla Has Version: yes Applies only to major version numbers: n/a If mozilla is found in the user agent, it is displayed as Mozilla, along with its version number unless is it found as version 0 through 4, in which case, it is displayed as Netscape along with its version number. Use multiple overlapping transformation rules to distinguish or consolidate user agent types and versions as required in order to effectively evaluate your web applications. 65
NOTE: The user agent transformation rules are applied in the order that they appear on the Configure > Reporting > User Agents page. If you want a user agent transformation rule, that you add, to have priority over others, you need to use the up and down arrows to move the position of the rule in the list. Database configuration The Database page, accessed by clicking Configure > Appliance > Database, provides you with a number of administrative tasks for managing the appliance s database. The links and controls available on this page enable you to perform tasks such as: Viewing the database page Viewing the database details page Resetting a database Resetting the configuration Exporting a configuration Importing a configuration Setting database retention Configuring remote database access Purging existing resources Figure 36. Database page For a complete list of database types and their description, see the Foglight Experience Monitor SQL Reference Guide. 66
Viewing the database page The database page contains a list of databases, and for each database, details about metrics, storage, and its data retention periods. For each database you can view information about its size and usage. The Size value for each database refers to the hard disk space used by the database. The Usage (%) value refers to the percentage of space on the database partition that the database is occupying. The Size and the Usage columns display a total value for all of the databases included on the Database page. These values display the overall space that the databases are using and do not include values for any configuration databases. The Database Partition Usage metric, viewed on the Metric Analysis > System Health > Database Partition Usage page displays a total value for the database partition that includes configuration database values. If you compare the total percentage on the Database page with the metric value on the Database Partition Usage page, the totals will not match. For information about database totals that include configuration databases, see the Database Partition Usage metric in the Foglight Experience Monitor Metric Reference Guide. NOTE: The total values on the Database page will not match the total amount of the System Health - Database Partition Usage metric. Figure 37. Database total information In addition you can click the Details link which allows you to drill down to detailed information for a specific database. For information about viewing database details, see Viewing the database details page. You can also reset an individual database or all databases on the list. For information about resetting a database, see Resetting a database. Viewing the database details page You can view detailed information about a database by clicking the Details link that appears beside any of the databases on the Database page. The Details page displays information about categories and time periods. The values that are displayed on this page are stored in the System Database category. For more information about System Database category, see the Foglight Experience Monitor SQL Reference Guide. Figure 38. Database: Details page The Categories section displays all of the metrics that are contained in the database. For example, there are several application categories (application component by city, and application component by region) that 67
provide metrics from a different perspectives associated with the Application database. Each category displays information about its size, which is the hard disk space used by the category, and the percentage of the entire database that each category is occupying. The percentage column (%), when added, should total a 100% and should reflect the total for the same data stored in the System Database category. The Time Periods section displays the time periods associated with category data. Time periods reflect the breakdown of all the data for all the categories in the database. For example, "Hourly" is going to include hourly data for the entire complement of Application Component category such as Application Component, Application Component by City, Application Component by Country, Application Component by Server, Application Component by Subnet and Application Component by Region. Each time period displays information about its size, which is the hard disk space used by the time period, and the percentage of the entire database that each time period is occupying. The percentage column (%), when added, should total a 100% and should reflect the total for the same data stored in the System Database category. For information about resetting a category or time period, see Resetting categories and time periods. Resetting a database Sometimes it is necessary to delete data from a database. Resetting a database, deletes all of the data in the database and then reinitialize it. IMPORTANT: Use this capability with extreme caution as all metrics collected up to this point are discarded. The process of resetting the metric database may take several minutes depending on the size of the database. Attempts to generate reports during this time may generate error messages in the web console. IMPORTANT: when you perform a reset the numbers displayed on the Database and Details pages do not immediately reflect the change. It can take up to 15 minutes for the metrics to change as a result of the reset. To reset a database: 1 Select the Reset box for the corresponding database. 2 Click Reset button. A confirmation window appears. 3 Click OK to proceed with resetting the database. To reset the configuration you must click OK to the warning message. To reset all databases: 1 Select the Check All link located at the bottom of the page. 2 Click Reset button. A confirmation window appears. 3 Click OK to proceed with resetting the database. Resetting categories and time periods IMPORTANT: The following options should be used with extreme caution; they can modify or delete all configuration settings and metrics in your system. At times, it may be advantageous to delete metric categories and time periods for each database. Using this capability, you can clear the entire categories or time period list or remove a single resource. Categories are configured on the Configure > Reporting > Categories page. Categories that are not selected on this page do not collect metrics during monitoring. Although, these categories will display on the Categories section of the Database > Details page, with a zero value since no metric data has been collected. For more 68
about Metric Categories, see Metric categories. When resetting categories for a database, you are resetting only the appliance that you are logged into regardless if this appliance is part of a cluster of appliances. To reset categories and time periods: 1 Select the Reset box for the corresponding category or time period. 2 Click Reset button. A confirmation window appears. 3 Click OK to proceed with resetting the selected item. To reset the configuration you must click OK to the warning message. Resetting the configuration If you are moving an appliance to a different location or want to monitor an entirely different set of applications, you may want to reinitialize the configuration database before proceeding. To reset the configuration: 1 Click the Reset Configuration link at the bottom of the Database page. A confirmation window appears. 2 Click OK to proceed with the reinitialization. Exporting a configuration If you want to transfer an existing database configuration from one appliance into another appliance, you can use the Export Configuration option. Clicking Export Configuration starts the process whereby all of the database configuration settings are saved to a ZIP file. Save the ZIP file to a location where it can be accessed by the other appliance. To export a database configuration: 1 Click the Export Configuration link at the bottom of the Database page to open the Database > Export Configuration page. 2 To include into the exported configuration any SSL key that you may have defined or loaded into your appliance, specify a pass phrase in the Passphrase box; otherwise leave it blank. IMPORTANT: When you import a configuration file exported from the same appliance, any existing SSL keys are restored whether a passphrase was used or not during the exporting process. When you import a configuration file exported from a different machine, any SSL keys are restored only if a passphrase was used during the exporting process. 69
3 Click Submit to save the contents of the ZIP file to a location and complete the export process. Importing a configuration For various reasons you might want to use an existing database configuration from another appliance. To import the database configuration, the appliance from which you want to import the file must have access to the location where the ZIP file was saved. To import a database configuration: 1 Click the Import Configuration link at the bottom of the Database page to open the Database > Import Configuration page. 2 To navigate to the ZIP file location, click Browse. 3 If the configuration file was exported with a passphrase and you want to restore any SSL keys included with this configuration, specify the pass phrase in the Passphrase box; otherwise leave it blank. IMPORTANT: When you import a configuration file exported from the same appliance, any existing SSL keys are restored whether a passphrase was used or not during the exporting process. When you import a configuration file exported from a different machine, any SSL keys are restored only if a passphrase was used during the exporting process. 4 To import the contents of the ZIP file to the appliance s database, click Import. Setting database retention The appliance incorporates self-correcting mechanisms to ensure that the amount of disk space consumed by each metric database does not consume all of the available space on the database partition. The database retention page allows administrators to customize the target retention time for each time period. To view existing database retention settings, click the Retention link at the bottom of the Database page. There are six types of retention time periods that are configurable by the administrator. The discrete retention type is composed of separate distinct data points made up of the following categories: Alarms, User Sessions, 70
SOAP Faults, and HTTP Faults. The 5-minute interval, hourly, day, week, month periods store data based on target time set by the administrator. For more information, see the information about Target in the following table. Each time period type displays the following information. Table 8. Information displayed for each time period Column Target Minimum Target Maximum Current Target Description Indicates the amount of data (in days) that is stored when the system fills the database partition. Once filled, the system removes older data to make room for newer data. This value reflect the amount of data stored, under optimal conditions, when the database partition is not filled to capacity. The number of days that data for the time period is currently being retained. The number of days that the system attempts to retain data for each time period. Foglight Experience Monitor Administrators can adjust the value upward or downward if a particular time period is considered more important than another. The system attempts to retain the amount of data suggested by the target, but may not always be able to achieve that goal depending on the amount of data that is being collected for each time period. Click OK to set the target retention. When using the retention periods, data that is older than the current retention interval is purged over time it is not purged immediately. For most sites, these retention intervals can be achieved without filling up the database partition. However, when disk space consumption on this partition exceeds 88% capacity, the system purges data for all these categories. This results in the following retention periods. Table 9. Retention periods Time Period Discrete Real-Time Hour Day Week Month Retention 24 hours 24 hours 30 hours 75 days 8 weeks 8 months If the disk space consumption on this partition exceeds 88% capacity, the data in the Discrete time period is rotated until disk consumption is less than 88% or the Discrete period contains its minimum of 24 hours worth of data. In cases where the disk partition remains in excess of 88% capacity, and the Discrete time period holds less than 24 hours worth of data, the data in the Real-Time time period is rotated until it reaches 24 hours worth of data. If the partition remains in excess of 88% capacity, and the Real-Time time period holds less than 24 hours worth of data, hourly data is rotated to 30 hours worth of data. This process continues using the day, week and month time periods or until such time as the database partition is reduced to under 88% capacity. To reduce the load on the database you can configure the following items: On the Configure > URLs > Advanced Options page you can choose to ignore response codes that may be filling up your HTTP Fault log. On the Configure > Monitoring > User Sessions page, you can also configure the system not to record every user session. For more information, see Identifying user sessions. On the Configure > Reporting > Categories page, reduce the amount of data being stored in the database by turning off particular categories that you do not find useful. For more information, see Metric categories. 71
NOTE: The appliance records in the database a maximum of 2,000 hits for each User Session. This limit is necessary to avoid flooding the database with records. This limit is not configurable. If you require to have access to data that exceeds the maximum limit, it is recommended to use the Foglight Experience Viewer in conjunction with the Foglight Experience Monitor. Configuring remote database access You can enable remote access to the database using port 3306. To configure remote database access: 1 Click the Remote Access link at the bottom of the Database page. The Database > Remote Access page appears. 2 Select the Enable remote SQL database access check box to enable remote access to the metric databases. This option must be used in conjunction with the password you specify in order to query the database remotely. This capability allows you to use your own querying and reporting tools (such as Crystal Reports) to use the metrics collected by your appliance. 3 Type a password in the Password field. 4 Click OK. IMPORTANT: The password entered must be a strong password (must be at least 8 characters in length, and a combination of letters and numbers). Purging existing resources At times, it may be advantageous to purge the resource list for specific categories. The resource list refers to the list of names for a category that you can access via the Analysis > Resource List menu item. Configuration changes that you have made might result in a significant number of obsolete names cluttering up the resource list for certain categories. Using this capability, you can clear the resource list for an entire category, or remove a single resource that you no longer want to appear in the web console. This action does not discard any metrics from the database. As new metrics are collected, the resource list repopulates with names reflecting the new metrics. For these resources, the full set of historical data that was collected before the purge is still available in the web console. However, obsolete names that are not generating any new metrics will never again be available in the web console. Consequently, if this historical data is important, this capability should not be used. For more information, see Transforming monitored URLs. To purge a list of resources or an individual resource, click Purge Resource Names link at the bottom of the Database page, to displays the Purge Resources page. 72
Figure 39. Database: Purge resource Names To purge a single resource from a category: 1 From the Category list, select the metric category whose resource you would like to purge. 2 Click Choose Resource to display a Resource List window. 3 Locate and select the resource you would like to purge. 4 Click OK. A window with a warning message appears. 5 Click OK to begin purging the chosen resource. A window with a warning message appears. To purge all resources from a category: 1 From the Category list, select the metric category whose resource you would like to purge. 2 Select the All option. 3 Click OK. 4 Click OK to begin purging the resource list. It may take up to 15 minutes to purge the entire list depending on its size. To purge all resources matching a regular expression: 1 From the Category list, select the metric category whose resources you would like to purge. 2 Select the Regular Expression option and type a regular expression matching the names of the resources that you want to delete. 3 Click OK to begin purging the resources. Appliance integrity In order to monitor traffic and collect data, the appliance needs to be up to date, backed up regularly, and if required, communicating through secure channels. The following sections outline how to maintain the integrity of your appliance. For more information, see these topics: Viewing system information System information in a multi-appliance setup Viewing and changing the license Database repair log 73
Viewing system information The System Information page can be accessed by clicking Help > System Information. The System Information page presents information about the appliance that you are currently connected to via the web console. You can use it to do the following tasks: Confirm the version and build information Confirm the appliance hardware specifications Change the appliance license System information in a multi-appliance setup If you have a multiple-appliance cluster, the System Information page on each probe displays the status of communications with the portal. Figure 40. Portal status On the portal, this page displays the last communication time for each probe in the cluster. Figure 41. Last communication time Reassigning probe settings This page allows you to reassign configuration settings for an existing probe to a different IP address. 74
To reassign a probe s settings to a different probe: 1 From the System Information page, in the Probe section, click Move next to the probe that you want to reconfigure. The Move Probe page opens. 2 In the IP Address box, ensure this is the IP address of the existing probe. 3 In the New IP Address box, type the IP address of the appliance that you want to serve as the portal. A message appears. 4 Click OK. Removing Probe configuration settings On occasion you might want to remove the configuration settings for a probe that no longer exists or has been assigned a different IP address. To remove probe configuration settings: 1 From the System Information page, in the Probe section, click the Remove link next to the probe that you want to reconfigure. 2 Click OK to the message to remove the configuration settings for the probe. Viewing and changing the license Your appliance should arrive with a permanent license key previously configured, but there may be circumstances in which only a temporary 90-day license key was installed. If your appliance has a temporary license key, you need to obtain a permanent license key and configure using the steps detailed below. IMPORTANT: If a temporary key is allowed to expire, the appliance agent stops collecting data until a valid key is provided. Previously collected data is retained. License key changes can be performed by clicking Change the license for the appliance on the main System Information page. This displays the System Information Appliance License page. Figure 42. System Information: Appliance License page The License Type is displayed. If your appliance is still using a temporary license key, the number of Days Remaining is shown. If the License Key box contains a key, then your appliance is already configured with a license. 75
Each appliance generates a Host ID which is based on its unique hardware characteristics. You need to provide this Host ID in order to obtain a new license key from Dell Technical Support. NOTE: Contact your Dell account manager for more information about acquiring a license. Database repair log The Database Repair Log displays the database repair process and appears immediately after an upgrade when the appliance is restarting.the database repair process is an expected part of the upgrade and typically only lasts for a few minutes, although if there are corrupt tables that need to be repaired the process can take up to one hour to complete the repair process. Updating the appliance Periodically, Dell releases a new version of the Foglight Experience Monitor software. You can access the Upgrade page by navigating to Help > Upgrade. You should exercise caution when upgrading an appliance, since the upgrade may include changes that require an adjustment period on the part of your users (for example, significant changes to the web console). The upgrade process can take anywhere from 15 minutes to 8 hours depending on the complexity of the changes required to upgrade the configuration and metric databases. During this time, the web console is not available. 76
IMPORTANT: It is essential that you do not manually reboot the appliance during the upgrade process. If you do so, the system may become irretrievably corrupted. Consequently, it is highly recommend that you conduct a full backup before applying any upgrade. NOTE: When an upgrade is underway, the Foglight Experience Monitor console goes offline for the duration and cannot be used. Users should not reboot the appliance during this time. The update process will restart processes automatically and may or may not reboot the machine. Check the login page of the web console to confirm when the appliance is available. For more information, see Installing an upgrade. Installing an upgrade Download upgrades from the Dell Support Portal and install them using the following procedure. This method of upgrading is recommended. To install an upgrade: 1 Click Browse in the Upgrade Appliance section of the Upgrade page and locate the package on your local hard drive. 2 Click Upload to upload the package to the appliance. The uploading process is complete when the Install button appears. 3 Click Install to start the update process during which the package is decompressed, decrypted, and then installed into the proper locations on the appliance. 77
IMPORTANT: The Log page is displayed until the upgrade installation process requires all system applications to stop. Consequently, the web console will not be available and the system may be offline for a period of 30 minutes to 3 hours. Do not reboot the appliance during the upgrade. When the console program is available during the database upgrade process, you can log in to see the progress of the upgrade. When the web console login page becomes available, the upgrade is complete. NOTE: The upgrade file must have a.pkg extension, otherwise you will receive an error message. Backing up and restoring data Backups include all databases and configuration settings that are needed to recreate an identical system on a new appliance. Settings such as the appliance's IP address and gateway IP are not included in the backup and must be reconfigured manually using the console program. The appliance s backup and restore functions can utilize either File Transfer Protocol (FTP) or Secure Shell (SSH) as a network protocol to transfer files across the network to and from a backup server that you have designated. For environments with stringent security requirements, it is recommended that you utilize SSH as the backup protocol. Backups can be scheduled to occur on a daily or weekly basis or they can be initiated manually. You can also manually initiate a backup of the configuration files and databases which is useful for quickly transferring configuration from one appliance to another. All backups can be configured, scheduled, and initiated on the Backup & Restore page, accessed by clicking Configure > Appliance > Backup & Restore. IMPORTANT: Most system processes are halted during the backup, but the appliance continues to collect metrics. Metrics collected during the backup are queued and then loaded into the database once the backup has completed and all of the system processes have restarted. Locating a server to use for backup To ensure that the backups for your appliances are secure and reliable, it is essential that you use a backup server with at least 20 GB (gigabytes) of available disk space. Your backup server should also have a fast disk I/ O subsystem that is capable of writing large files and has a transfer rate that is able to keep up with the demands placed on it. 78
For best performance, use a backup server that is not being heavily loaded by other unrelated applications. If your backup server is heavily loaded with other activity, it is possible that backups from your appliances could experience timeouts which would cause them to fail. It is also possible that backups can take many hours to complete. In general, a backup can take approximately two to three hours but this amount of time can increase if the backup server is heavily loaded and responding slowly. During backups, the appliance's web console is unavailable. To ensure backup availability in the event of a system failure, it is highly recommended that you schedule a daily or weekly backup. It is recommended to avoid scheduling backups at midnight due to the fact that the appliance conducts regular database maintenance functions at midnight every day. This could increase the amount of time that the backup requires. In addition, if you have multiple appliances using the same backup server, it is important that you do not schedule the same backup time for more than one appliance. Doing so can cause your backup server to become overloaded, triggering timeouts which will result in incomplete backups. The backup server can be running any operating system as long as it supports an FTP or SSH server. Configuring backups using FTP Foglight Experience Monitor supports two protocols for backups: FTP and SSL. The following steps explain how to set up a backup server to use FTP. To configure backups using FTP: 1 Ensure that your backup server is running an FTP server that can be accessed across the network from the appliance. For information on how to configure Windows 2003 to provide an FTP server that you can use for backups, see: http://support.microsoft.com/kb/323384. NOTE: Make sure that you do not configure the FTP server solely for anonymous access, and that you do not disable write access. 2 Log in to the web console on the appliance and navigate to Configure > Appliance > Backups & Restore. 3 In the Server box, enter the IP address of the FTP server that you located in Step 1. NOTE: You must enter an IP address not a domain name. 4 In the Directory box, enter the path you have created in the user account on the FTP server that will hold the backups. If you want to store backups at the user account s home directory, you can leave this blank. The appliance refers to this path when creating, as well as restoring backups. 5 Select the FTP button to specify that you want to use FTP as the backup protocol. 6 In the User Name and Password text boxes, enter the account name and password for the user account on the FTP server that will be used for backups. 7 Select the type of frequency that you want to use: None no frequency selected. Daily specify a time, by the hour and minutes, at which the backup process will begin. Weekly specify the day, the hour and minutes at which the backup process will begin. 8 Click Apply to save the settings you have entered. When successfully applied a message appears: The settings have been successfully applied. 9 Click Verify Settings to ensure the appliance can establish a connection with the FTP server. For more information, see The verification process. A window is displayed, providing status messages regarding the success or failure of the verification process. 79
Configuring backups using SSL This procedure describes how to set up backups using SSL as the communication protocol. This method provides a more secure and reliable approach than FTP. To configure backups using SSL: 1 Ensure that your backup server is running an SSH server and can be accessed across the network from the appliance. NOTE: A popular open-source SSH server that works on Windows platforms is cygwin which you can download here: http://www.cygwin.com/ 2 Create a backup user account on the SSH server. For example, name it support. 3 Create a subdirectory that will hold the backup files. For example, name it /home/support/incoming. 4 Log in to the web console on the appliance and navigate to Configure > Appliance > Backup & Restore. 5 In the Server box, enter the IP address of the SSH server that you located in Step 1. 6 In the Directory box, enter the path you have created in the user account on the SSH server that holds the backups (Step 3). The appliance refers to this path when creating, as well as restoring backups. 7 Select the SSL option to specify that you want to use SSL as the backup protocol. 8 Click Public key to download the secbackup_key.pub key to your local drive. 9 Copy the public key file into the user s home directory on the SSH server. If you are using cygwin under Windows and the user name is support, then the directory should look like: c:\cygwin\home\support Log in to the SSH server as the user name support, and type the following at the command line: # mkdir.ssh # chmod 700 /home/support/.ssh # cat secbackup_key.pub >> /home/support/.ssh/authorized_keys2 # chmod 600 /home/support/.ssh/authorized_keys2 IMPORTANT: Permission settings are required. 10 In the User Name box, type the user account name on the SSH server that will be used for backups (Step 2). TIP: You do not need to enter a password. 11 In the Frequency section, specify one of the following: None no frequency selected Daily specify a time, by the hour and minutes, at which the backup process will begin Weekly specify the day, the hour and minutes at which the backup process will begin 12 To save the settings that you have entered, click Apply. The next steps involve copying the appliance s public key to the SSH server so that authentication can be performed automatically by the backup process. 13 To ensure the appliance can establish a connection with the SSH server, click Verify Settings. A window displays, providing status messages regarding the success or failure of the verification process. 80
The verification process The verification process transfers a small file to the backup server and then retrieves it. If the file retrieved matches the file that was transferred, then the backup server is verified. A rough speed calculation is also performed. Download and upload times recorded during the verification process are displayed in a window. These timings vary each time you run the verification process because of differences in network utilization and CPU and disk utilization on the appliance. Generally, you should expect to see timings greater than one MBps. If your timings are lower, backups could take a very long time to complete. Investigate the quality of the network path between the appliance and the backup server and also the performance of the backup server itself. You may need to secure a backup server with higher performance or relocate the backup server so that the backups have a shorter and faster path across the network. If the verification fails, you need to check out the items in the following list of possible causes: The backup server is on a part of the network that is not accessible from the application. The user credentials that you entered are incorrect. The user account does not have write privileges. The directory you entered does not exist. If you are using SSH, the public key that you provided for the SSH server is not correct. If you are using SSH, the appliance public key was not added to the authorization list of keys for the SSH user account that you are using. Manually starting a complete backup The complete backup includes data from the metric database and the configuration database. 1 Ensure that the backup server information has been configured and verified. For more information, see Locating a server to use for backup. 2 Click Start a complete backup now. The Backup & Restore > Restore Now page is displayed, indicating the appliance data is being sent to the backup server. At some point during the backup, you will lose connection to the web console as the system begins to shut down processes. 3 You can log back in to the appliance after the web console has been restarted. IMPORTANT: A backup can take as long as five hours if the database is large. The amount of time required by the backup process is dependent on how much data needs to be written to the backup server and how often you conduct backups. For more information, see these topics: Restoring a backup How backup works About the security of your appliance 81
Restoring a backup Restoring a backup is normally only necessary when a catastrophic hardware failure requires replacement of your appliance. Follow the steps below to restore a backup. IMPORTANT: A backup can only be restored on an appliance that has the same version number as the appliance where the backup was created. To restore a backup: 1 Verify the backup server settings are correct. For more information, see Backing up and restoring data. 2 Ensure that there are no users currently logged into the web console. 3 On the Backup & Restore page, click Restore a backup. The Backup & Restore > Restore page is displayed, and a system message states: Building the list of available backups. Please wait. 4 Wait for the appliance to finish scanning the directory specified on the Backup & Restore page. Once complete, two lists are displayed: the first containing a list of backups made from the appliance to which you are currently connected, and the second containing a list of backups made from other appliances. 5 For the backup you want to restore, click the Restore link that corresponds to the backup you want to restore. As the restore process proceeds, the web console halts causing you to lose your connection. Do not attempt to reboot the appliance during the restore process. Depending on the size of the backup, it may take several hours for the restore to complete. 6 After the appliance has restarted, you can log in. IMPORTANT: If you are restoring a backup from a different appliance, any SSL keys that you have configured for the monitored servers is no longer usable by the appliance. For security reasons, SSL keys are encrypted with an algorithm that uses characteristics of the hardware in the appliance itself. Therefore, the SSL keys cannot be decrypted on the new appliance because they were already encrypted on a different machine. You need to reload all SSL keys through the user interface. If you are restoring a backup to a machine with a different IP address all settings, such as the list of monitored servers and protocols, are carried over. How backup works Backups begin at their scheduled time or whenever you initiate one manually as described in Manually starting a complete backup. At the beginning of a backup all non-essential system tasks are shut down. This includes the Apache web server and all database processes. This is required in order to ensure that no changes are made to any of the databases while a backup is underway. The appliance continues monitoring traffic and collecting metrics during the backup process. Data files containing new metrics are queued up and are loaded into the database when the backup has completed. 82
When the first backup for an appliance is initiated, the system creates a unique directory for the appliance on the backup server. The file name of this directory is based on the MAC address of the control port (eth0) of the appliance. For example, if the MAC address of eth0 is 00:30 is named 0030482DF610_backup. This directory only contains backups from a single appliance. Within this top-level directory, additional directories are created containing a timestamp for the date and time the backup was created. For example, the directory 0030482DF610_backup/20080107164500Z is created for a backup that was initiated on Jan. 7, 2008 at 16:45 UTC. Within that directory, multiple subdirectories are created for individual databases and configuration files. You should never move or modify any of these files contained in the directories. Doing so causes the backup to become unusable. Backup directories contain three types of files: TGZ, MD5, and XML. The TGZ files are compressed tarballs of database or configuration files. The MD5 files are MD5 hashes of the corresponding TGZ files. These are used to verify that the TGZ files are not corrupt. You can verify the integrity of your backup by computing the MD5 hash of each file and comparing that to the MD5 key files that are created by the backup process. The XML file, backupinfo.xml, gets created in each backup directory. This file contains information about each component of the backup. It is used by the system for conducting subsequent backups and for restoring. NOTE: Some of the directories referenced in this file could be from previous backups due to the incremental nature of the backup process (described below). What items are backed up Backups include all configuration settings and metrics with the exception of the following: Five minute interval data User Sessions Alarms HTTP Faults SOAP Faults Backup space and frequency As outlined in the previous section, backups can be configured to run on a daily or weekly basis on the Configure > Appliance > Backup & Restore page. A full backup can require disk space equaling 20% of the total size of the Foglight Experience Monitor database partition. To find the current size of the database partition, click Help > System Information and look for Database Partition Size. An incremental backup can require up to 20% of the disk space consumed on the database partition since the last backup. To monitor your database partition usage over time, use the Configure > Appliance > Database page. For example, let's consider a database partition size of 3.6 TB. The initial backup of this database requires approximately 720 GB (about 20% of the partition size). If the system collects about 50 GB of new data a week, you also need an additional 10 GB for each subsequent incremental backup. Therefore, the disk space required to hold the initial backup and ten weeks of incremental weekly backups is 820 GB, calculated as follows: 720 GB + (10 GB * 10) = 820 GB The disk space requirements continue to increase over time until these backups are cleared off the backup server. At that point, the initial backup again requires 720 GB and the same estimates for incremental backups apply. It is recommended that you back up your appliance on a regular basis whether it be daily or weekly. It is also recommended that approximately once a month, you archive the contents of the directory server off to tape, DVD, or CD, on your backup. Doing this keeps your FTP server storage requirements to a minimum and allows for offsite archival of your appliance data. 83
About the security of your appliance For a list of security features provided by the Foglight Experience Monitor appliance, and for information about evaluating these features in connection with the federal information security standards recommended by NIST (National Institute of Standards and Technology) and promulgated under the FISMA (Federal Information Security Management Act), see the Foglight Experience Monitor Security and Compliance Field Guide. 84
Specifying monitored web traffic 4 This section describes configuration settings that allow you to tailor the metrics collected by the system. Topics such as identifying ports and portals and subnets are covered here. For more information, see these topics: Configuration settings Identifying protocols Managing monitored servers Using filters to exclude traffic Defining subnets Identifying user sessions Monitoring instrumented web pages Configuration settings During the Setup Wizard process you specified the ports and servers that the appliance should monitor. (For more information, see Configuring the appliance). This chapter discusses in much greater detail the configuration settings that allow you to tailor the metrics collected by the system to fit your requirements. Using the configuration pages found on the Configuring > Monitoring menu, you can adjust configuration settings that influence what metrics are collected. The diagram below depicts the order in which these settings are applied to distinguish the traffic that the appliance will monitor from the traffic that is discarded. Figure 43. Traffic monitoring diagram Identifying protocols The Protocols page, accessed from the Monitoring menu, allows you to configure the system to monitor applications using Transmission Control Protocol (TCP). TCP is a common transport layer protocol used both for Internet and Intranet applications. With Foglight Experience Monitor you can gather performance and volume metrics for any TCP-based application. Metrics such as IP packet counts, and TCP connection and response time metrics are available. For more information about metric definitions, see the Foglight Experience Monitor Metric Reference Guide. HTTP and HTTPS (HTTP over SSL) are reliable application protocols that use TCP to transport messages between client and server. Given their widespread usage, these are configured as default protocols when the appliance's 85
configuration database is initialized. If you would like to monitor other TCP-based protocols (such as FTP and SMTP), you must configure those protocols using the procedure defined below. When configuring protocols and ports on the Protocols page use the following guidelines: Each protocol must have a unique name. There can be only one protocol using type HTTP and one using type HTTPS. All other protocols must have type set to Other. A port number can only appear in one protocol definition. Figure 44. Protocols page For more information, see these topics: Manually adding protocols and ports Automatically discovering ports Removing protocols Managing protocols on a Portal Manually adding protocols and ports To add a protocol and port manually: 1 From the Protocols page, click Add a Protocol. 2 In the Protocols section, type the name of the protocol. 3 From the Type list select one of the protocol types: HTTP, HTTPS, or Other. TIP: Use the Other type if you are configuring a protocol other than HTTP or HTTPS. 4 In the Ports section, type the port number that corresponds with the specified protocol and then click Add. The port number is added to the list of ports. 86
5 Click OK to add the configuration to the Protocols page. Automatically discovering ports If you require assistance determining which ports are in use with monitored traffic, you can use the Auto- Discover Ports option and have the appliance automatically discover ports that are in use. To automatically discover ports: 1 On the Protocols page, click Auto-Discover Ports. The appliance begins to analyze current network traffic and lists all the ports that it discovers. This page updates every few seconds displaying the list of ports that have been discovered in the traffic that the appliance is currently monitoring. You may need to allow the discovery process to run for several minutes depending on the amount of activity your applications are currently receiving. 2 To stop the auto-discovery process, click Stop at any point, and use the list of ports that has been detected up to that point. The auto-discovery page displays the following data. Table 10. Information displayed in the auto-discovery page Data Ports Client Packets Server Packets New TCP Connections Active TCP Connections Definition Displays the port number. The number of client packets being transmitted from clients to the server. NOTE: If the packet count is zero, and the Server Packet value is greater than zero the network tap is not configured correctly. The number of packets being transmitted from servers to client. NOTE: If the packet count is zero, and the Client Packet value is greater than zero the network tap is not configured correctly. The number of new TCP connections observed on the port. NOTE: If the number of connections equals zero or the number is small in relation to the number of active connections, the load balancer is maintaining a limited number of TCP connections to the web server and using the proxy server to send all requests over the TCP connections. The number of active TCP connections discovered in the traffic for the port. 87
3 To open a details dialog box for this port, click details. The HTTP-detected and SSL-detected indicators report whether HTTP or SSL traffic was detected for this port. All active ports on the server are listed. 4 Select the check boxes that correspond with any ports you want the appliance to use to monitor traffic. 5 Click Add. 88
The Protocols > Edit page appears where you can assign a name and a type to the protocol definition, and then save it. Removing protocols On the main Protocols page, you can remove a protocol that is no longer being used by the TCP protocol on your network. You can delete a Protocol by selecting its corresponding check box, then click Delete. Managing protocols on a Portal When you access the Protocol page on a portal, a section for each probe displays the protocols and ports being monitored by each probe. Figure 45. Protocols page To configure protocols and ports for a probe you must use the web console of the probe. Click Configure protocols on the probe to launch the web console for the probe so you can modify the configuration. For more information about configuring protocols, see Identifying protocols. Managing monitored servers The Servers page lists the web servers monitored by the appliance to which you are currently connected. The Servers page enables you to manage the server list through the addition of new servers, and removal of 89
unwanted servers. You can configure the server settings to monitor secure HTTP traffic for any server (for more information, see Configuring SSL keys) as well as enable a Server IP Identification Tag, if any servers are operating behind a load balancer. Figure 46. Servers page For more information, see these topics: Manually adding a new server Automatically discovering servers Removing servers Editing server details Configuring server options Configuring SSL keys Exporting SSL keys from IIS Managing servers on a Portal Manually adding a new server From the Servers page, type an IP address in the text box, and then click Add. You can also enter a range of addresses (for example, 192.168.1.1-10). Automatically discovering servers If you want the appliance to find web servers for you, you can use auto-discover and it will analyze network traffic and list all the servers it discovers that have traffic for ports that are configured on the Protocols page. To automatically discover servers: 1 Click Auto-Discover Servers to begin the server detection process. The web console displays list of detected servers, updating every few seconds. Since the list s compilation is based on current network traffic, the servers you are interested in monitoring should appear within the first few minutes of the auto-discovery process. If necessary, you can allow the process to continue for up to 30 minutes. 2 To stop the auto-discovery process, click Stop at any point and use the server list that has been compiled up to that point. The auto-discovery page displays the following data. All servers found appear in a check-list. 90
3 To open a details dialog box for this server, click details. The HTTP-detected and SSL-detected indicators report whether HTTP or SSL traffic was detected for this port. All active ports on the server are listed. 4 To close the detail dialog box, click OK. 5 Check all the check boxes that correspond with any servers that you want the appliance to monitor. 6 Click Add. You are returned to the main Servers page, and all newly added servers are now listed. Removing servers On the main Servers page, you can delete a server entry by selecting its corresponding check box, then clicking Delete. Check All can be used to select all entries for deletion, and Clear All is used to clear any entries that were selected for deletion. Editing server details You can change the display name and command processing time. Changing server display names For every server listed on the Servers page, there exists a server IP address (under the Servers column), and a server name (under the Display Name column). The display name is used anywhere the server is mentioned in the web console. 91
Figure 47. Server display name By default, the server IP address is assigned as the server. To change the display name: 1 From the settings column, click the server s corresponding Edit link. This displays the Servers > Edit page for that server. 2 In the Display Name box, type the name you want the server to display in the web console. 3 In the Command Processing Time Service Level Threshold box, type the desired service level for command processing times on this server. For more information, see Changing monitored servers service levels. 4 You can also have the appliance retrieve the server domain name by clicking Lookup DNS name. The DNS name is presented in a window, which you can accept by clicking Use DNS name. A DNS lookup may or may not yield a server name, depending on how the server is configured. To use the lookup feature, the appliance needs to be aware of the DNS servers on your network. See Network settings for more information. 5 To accept the name, and return to the Servers page, click OK. Changing monitored servers service levels Every server listed on the Servers page has a Command Processing Time metric value associated with it. You can alter the threshold that determines the value of this metric following the steps defined below. For more information about this metric, see Command Processing Time in the Foglight Experience Monitor Metric Reference Guide. 92
To set service level thresholds: 1 Click the corresponding Edit link in the Settings column for the server for which you want to set the service level threshold. This displays the Servers > Edit page for that server. 2 In the Command Processing Time Service Level Threshold box, type the desired service level for command processing times on this server. For example, if you decide that the time to process commands should not exceed one second, enter 1000 (units are in milliseconds). The Command Processing Time Service Level metric shows you the percentage of commands whose processing time was under one second. 3 To commit the change for this server and return to the main Servers page, click OK. 4 Repeat this procedure for every server whose default threshold requires modification. Configuring server options The Server Options page provides the ability to configure a server identification tag and also to specify how the appliance filters the traffic that it is receiving. To configure server options: 1 On the Servers page, click Server Options. 2 In the Server IP Identification Tag box, type the name of HTTP header tag that the appliance will use to identify servers. 3 In the Traffic Filtering list, select a filtering type. Choose one of following options to control how the appliance filters incoming traffic. 93
Table 11. Filtering types Option Use PCAP 1 Filter Use internal filter Use minimal filtering No filtering Definition Recommended. This option uses the PCAP filter for both TCP ports and IP addresses. It causes the system to apply the list of configured server IPs in a PCAP filter that it uses for each monitoring port. This option is the most efficient mode of operation and it ensures that unwanted traffic does not appear in the metrics. This is the recommended setting because it incurs the lowest overhead on the system. If you select this option, a limit of 300 configured server IPs is enforced. NOTE: If this option is not in the list, you have already configured 300 servers to use the PCAP filter. 300 servers is the maximum number supported by PCAP. You need to select one of the other filter options. This option does not use a PCAP filter. The agent filters both the TCP ports and the IP addresses internally. It causes the system to apply the list of configured server IPs within the analysis layer of the FxM agent. This option is less efficient than the Use PCAP filter option, but may be needed at sites where more than 300 server IPs need to be monitored. This option uses the PCAP filter for TCP ports; IP addresses are not filtered. This causes the system to accept any and all server IPs that appear in the monitored traffic. This option is not as efficient as the first two options, but may be needed at sites where the list of servers that need to be monitored is dynamic and, therefore, cannot be configured in advance. NOTE: In Foglight Experience Monitor versions 5.6.2 (or earlier), this option was known as No filtering. When upgrading your system to version 5.6.3 (or later), the old No filtering option automatically becomes Use minimal filtering. The new No filtering option is now defined as specified below. This option dictates that the traffic is not filtered either via PCAP or by the agent internally. Network traffic for any and all TCP ports and IP addresses is captured and analyzed by the system. This option is the least efficient of the four options, but is required at sites where the IEEE 802.1ad specification (also referred to as 802.1QinQ) is employed. The disadvantage to this option is that unwanted traffic may be monitored by the system and can appear in the metrics. This option also causes the greatest amount of overhead on the system. 1 PCAP (Packet CAPture) consists of an application programming interface (API) for capturing network traffic. UNIX -like systems implement PCAP in the libpcap library; Windows uses a port of libpcap known as WinPcap. 4 In the Maximum Frame Size box type the value for the largest jumbo frame size expected to be present in the monitored traffic. This setting must be at least 9,038 bytes but no more than 64,000 bytes. If the setting is too low, some traffic may be missed. Use caution when increasing the size, as this has an impact on system memory consumption. Generally, Dell Technical Support can advise you when this setting needs to be increased. 5 To enable the hardware timestamping functionality, select the Hardware Timestamping check box. For details about this feature, see Foglight Experience Monitor and Ixia Net Tool Optimizers. Configuring a server identification tag The Server Options page provides the ability to configure a server identification tag and also to specify how the appliance filters the traffic that it is receiving. 94
If your site uses a load balancer, the server IP addresses may not be available in the network traffic that the appliance is monitoring. Consider the following conditions: your web servers are situated behind a load balancer the load balancer is configured to mask server IPs you want the appliance to collect data from any or all of the individual servers concealed behind the load balancer the appliance will be deployed anywhere in front of the load balancer A load balancer that uses a virtual IP is typically configured to strip out server IP information in the IP layer of any communication it receives from its server farm. In this context, the physical server IPs remain hidden, and the appliance is unable to distinguish between unique servers. This means the appliance will not be able to break out metrics for each individual server. This issue can be resolved by configuring each web server to insert an HTTP header tag that contains the server s unique IP address (for example, header name: SERVER-ID, header value: 192.168.1.89 ). If you configure the appliance to recognize the tag as the Server IP Identification Tag (in this case, SERVER- ID), it can identify unique servers behind the load balancer. Actual server IP addresses can then be viewed in the web console (in this case, 192.168.1.89) instead of the load balancer s virtual IP. These real server IPs appear when you auto-discover servers. For more information, see Automatically discovering servers. NOTE: See your web server documentation to find out how to have an HTTP header tag, containing the real server IPs, inserted into outgoing traffic. Configuring a server identification tag in Microsoft This example shows you how to configure an HTTP header that will contain each server's physical IP address in Microsoft Internet Information Services (IIS). This header will allow Foglight Experience Monitor to track metrics by server as described above. To configure a HTTP header in IIS: 1 On the web server, navigate to Start > Programs > Administrative Tools > Internet Information Server (IIS) Manager. 2 In the left pane, expand the Web Server node and then expand the Web Sites node. 3 Select a Web Site from the list. 4 Right-click the Web Site and from the menu select Properties. 95
5 Select the HTTP Headers tab. 6 Click Add. The Add/Edit Custom HTTP Header dialog box displays. 7 Type a unique value for the Custom header name. For example, SERVER-IP. 8 Type a unique value for the Custom header value. For example, 192.168.1.1 (a standard formatted IP address). 9 Click OK. IMPORTANT: Foglight Experience Monitor expects a valid IP address in this field. This IP address may be a fictitious address so as to protect the identity of your physical servers. This is recommended if the application you are monitoring is publicly available on the Internet. 10 Configure each Web Server using steps 3-9. IMPORTANT: The address of the load balancer and the IP address that you configured in the Custom header value must appear in the list of configured servers on the Configure > Monitoring > Servers page in order for the appliance to collect metrics for the server. By default, when the appliance is set up for the first time using the Setup Wizard, it monitors HTTP traffic for each server listed on the Servers page. If any of your servers utilize secure HTTP (HTTPS), the appliance must be provided with each server s private keys in order to decrypt the secure network traffic they process. These keys are typically stored in encrypted format with an associated password. In order for the appliance to decode and analyze your HTTPS traffic, you need to provide both the key file and the associated password. The key file and password can be uploaded to the appliance and stored locally in files that are hidden and encrypted. The secure key file needs to be in the PEM (Privacy Enhanced Mail), DER (Distinguished Encoding Rules) or PKCS12 format. For more information about SSL certificates and keys, see Exporting a Certificate to the Appliance:. These files typically resemble the following sample: -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED 96
DEK-Info: DES-EDE3-CBC,F85757828BFF54C8 QlHfWyCnqpe5ag4LgNiRQaqrcTC5bBV3yT35tRbB0WKB3VsaDcHuhjlyz3ohQmgpHoZtWcCxCRm8DOROlBP EBhmRgUSYxByyt/ Y8OvL9Ei2YFdRjBapsJjEjpTEl6AXNGHKjHbmyCHs5O1LvnwEv13b51Q0RHRpRZX1yNVL34cz/ efmcfvlhgqlychfb0qrzlcnfvw6vrxwfkkm5jbw67wmkjqeda+vmmxskprdxfvyud/ HoB+gnMP0ecnSX7k4xZrLkIwk4QcVaJST2BPiDq0DhBdUfXRurJd9AQuAECAw6SgumqIRY/ CrH31w5dxqXzs2UY0lqODpU3tVqZW8+OxX34ojsBPeH0zmeOxnmQ8IebRyex8MTHhEVnpIU4DDNdKlbE0/ seoyngjrxfh2ldcrxlktmgrovg1vl216vgigbpfyzgobdiexkej24tw4do0irhyqiozvzvlr2kcm9l64nru hwgzlagpdlagflkexatqvp1nnetxor0ime4vbga1xlzdz/ath4u/ oe63hedj4b9c3mg8wapnx5ljaa7dvkskpwkzrp5qxcbun3r084j+q7kwmkiok0yt8rlqaplm7sayahokljp y0ehpfo4szpelhku39v4smba1llakcbxtzfyq22clckbtekckrc9yfoq8kpq9ssd2/ VCJRfLGhMRrVzSheOXeGDA/oKq/ ZZLO27r68odZaXQw1e9dR7oKStqNOG8M3WAsY3LEH++DAubqXDJ6g00aHC2eiVr9ZFfRUpKWRqybHns+ukA cg436ujok2rmyhwybhv8p2cdqf1ow9qqzo8qg== -----END RSA PRIVATE KEY----- Configuring SSL keys Foglight Experience Monitor supports the following Secure Sockets Layer (SSL) and Transport Layer Security (TLS) cryptographic protocols: SSLv2 SSLv3.0 TLSv1.0 TLSv1.1 TLSv1.2 NOTE: Due to the fact that TLSv1.3 only supports key exchanges based on ephemeral keys (unique keys generated for each TLS session), external devices (including Foglight Experience Monitor) are unable to decrypt website traffic using this protocol. When a server and the appliance communicate over SSL/TLS, a session first needs to be established (often referred to as SSL handshake). After a session has been opened, secure data can then be exchanged. This twostep process makes use of two different encryption algorithm types: key exchange algorithms, and bulk encryption algorithms. There are various implementations of each algorithm type, and the server s private key you are using must be supported by the appliance. Table 12. Key exchange algorithms Key Exchange Algorithms RSA Diffie-Hellman (DH) Kerberos5 Fortezza Support yes no no no The following bulk encryption algorithms are supported: Camellia RC4 RC2 IDEA DES 97
3DES AES FRT (Fortezza) To configure SSL Keys: 1 In the Secure HTTP column, click the Configure link that corresponds with the server whose private keys are to be uploaded to the appliance. The Servers > Configure Secure HTTP page for that server is displayed, listing any secure ports currently configured for the server. 2 Click Add SSL Key to display a window in which you can locate the file containing the private key. 3 Click Browse to open a dialog and locate the key. 4 Once you have confirmed its location, and the file path is listed in the Secure Key box enter the private key password in the Password box. 5 Select the port from the list of HTTPS ports in the Port box. These ports must have been previously configured on the Protocols page. 6 Click OK to upload the key, thereby enabling HTTPS monitoring for the specified port on this server. Each time you click OK, the private keys are uploaded to the appliance. Ensure you have provided private keys for all secure ports for each server that uses HTTPS. 98
IMPORTANT: The appliance encrypts the private key files and passwords that are uploaded to it and saves them in hidden files. This, along with the extensive hardening and security measures taken on the appliance itself, guarantees that your secure keys and passwords cannot be compromised. When exporting the private key from your web server, make sure to select Yes, Export the private key. Depending on the web server, the private key is not always included in an export by default.when uploading an SSL key for a server, if you encounter the message The key file you supplied is invalid. Possible reasons are: 1) incorrect password, 2) the file does not contain a private key, 3) the file was exported with strong encryption, or 4) the file is in an unrecognized format it indicates the key is considered invalid. IMPORTANT: Due to the nature of SSL, it is impossible to determine precisely why your key may be invalid, but there are two things you can do to diagnose this problem:1. Ensure your key is contained in a file that follows one of the PKCS, PEM, or DER formats.2. When exporting the private key from your web server, make sure to select Include private key. Depending on the web server, the private key is not always included in an export by default. For more information, see Exporting SSL keys from IIS. Exporting SSL keys from IIS The following example shows how to export an SSL certificate from Internet Information Services (IIS) using Microsoft Management Console. Loading Certificates into Microsoft Management Console: 1 On the taskbar, click Start and then click Run. 2 Type mmc to open the Microsoft Management Console. 3 From the File menu of the Microsoft Management Console, click Add/Remove Snap-in and then click Add. 4 From the Available Standalone Snap-in list, select Certificates and then click Add. 5 From the Certificate Snap-in dialog box, select Computer account. 6 Click Next. The Select Computer dialog box appear. 7 Select Local computer. 8 Click Finish. 9 Click OK in the Add/Remove Snap-in dialog box. The certificates folder is loaded into the console. 99
Exporting a Certificate to the Appliance: 1 Expand the certificates list and navigate to Certificate > Personal > Certificates. The right pane displays a list of certificates. 2 Right-click a certificate and select Select All Tasks > Export. The Certificate Export Wizard displays. 3 Click Next. 4 Select Yes, export the private key and click Next. 100
5 Ensure that Include all certificates in the certification path if possible and Enable strong protection are not selected. 6 Click Next. 7 In the Password dialog box enter and confirm a password. 8 Click Next. 9 Enter the File name to Export. 10 Click Next. 101
11 Review the export settings and click Finish to export the certificate. Managing servers on a Portal When you access the Servers page on a portal, a section for each probe displays the servers it has been configured to monitor. You cannot configure the list of monitored servers on a portal. Instead, use the web console for the probe whose server list needs to be changed. Configuring servers on a Probe On a portal, from the Servers page, click Configure servers on the probe to launch the web console for the probe so you can modify the configuration. Figure 48. Servers page 102
Using filters to exclude traffic Filters consist of regular expressions that are used to exclude specific traffic from being recorded in Foglight Experience Monitor or Foglight Experience Viewer. You can define two types of filters: URL Filters exclude traffic by URL, User Agent, HTTP URLs, or HTTPS URLs. These type of filters allow you to exclude traffic from Foglight Experience Monitor, Foglight Experience Viewer, or both. Subnet Filters exclude traffic by specifying a subnet. For more information, see these topics: Defining URL filters Testing URL filters Defining subnet filters Defining URL filters To define URL filters: 1 Navigate to Configure > Monitoring > Filters. The Filters page displays. 2 Click Add a URL Filter. 3 The Filter dialog box displays. 4 In the Filter box, enter the regular expression that will be matched against the incoming URL, or user agent string. 5 From the Type list, select one of the following options: URL excludes all URLs matching the filter. User Agent excludes crawler and synthetically generated traffic. HTTP URL excludes all HTTP URLs matching the filter. 103
HTTPS URL excludes all HTTPS URLs matching the filter. 6 To exclude specific traffic from appearing in the appliance s metrics lists, select the Exclude traffic matching this filter from analysis check box. 7 To exclude matching URLs from being transmitted to the Foglight Experience Viewer, select the Exclude traffic matching this filter from hits sent to Foglight Experience Viewer check box. 8 Click OK. You are returned to the Filters page. NOTE: If the regular expression is not valid, an error message is displayed. Testing URL filters After adding a URL filter you can verify that any data matching the filter criteria will be excluded from the traffic by clicking the Test link to the right of the filter. The Filter Test window displays a list of all URLs or user agents that match the filter you have entered and will be excluded from the traffic sent to analysis or playback. Figure 49. Filter test page Defining subnet filters You can define subnet filters in order to exclude traffic originating from a specific subnet of IP addresses. To define subnet filters: 1 Navigate to Configure > Monitoring > Filters. The Filters page is displayed. 104
2 In the Add a Subnet Filter box, enter the regular expression that should be matched against the incoming the subnet mask. Click Add. The filter is added to the Subnet Filters list. Defining subnets Subnets provide an independent network segment that consists of all devices for an IP address that have the same prefix. Using subnet masks, networks are divided into subnests based on security, performance and physical requirements. The Subnets page lists the IP address and subnet mask pairs that the appliance monitors. Each subnet can be further defined by a geographical location which consists of a city, a region and a country location. Configuring a subnet by location provides you with more control over incoming traffic. When using the geographical location, traffic is categorized by location based on a subnet s address. 105
Figure 50. Subnets page For more information, see these topics: Manually adding subnets Automatically discovering subnets Checking for overlapping subnets Configuring a default subnet Removing subnet definitions Managing subnets on a Portal Manually adding subnets To manually add subnets: 1 Navigate to Configure > Monitoring > Subnets page and click Add a Subnet. The Subnets > Edit page is displayed. 2 In the Name box, type a name you would like to appear in the metrics displayed in the web console for this subnet. The system provides a default display name for the subnet when it is first added based on the IP and subnet mask. Selecting a Geographical Location 3 In the Geographical location box, click the Select. 106
The Resource List appears. 4 In the Type list, select city, region or country to define the subnet. 5 Narrow the list to a more specific set of pages by entering a search string in the Search box. 6 If you need to search by Perl regular expression, check the Interpret as a regular expression option. You can use an asterisk (*) as a wild card to define a segment of the string that matches any combination of characters. 7 Click Go. 8 Select a location from the list. You are returned to the Subnets > Edit page. Selecting Masks 9 In the Subnet Masks section, type an IP address and subnet mask, separated by a / in the text box (for example, 192.168.1.0/24). 10 Click the Add button. 11 Click OK. Automatically discovering subnets The auto-discovery process examines monitored network traffic and constructs possible subnets that may be in use on your network. It is not possible for the appliance to determine the exact subnets that are in use because that information is not available in the traffic. Instead, the appliance uses the client IP addresses extracted from the IP headers in the traffic or from the x-forwarded-for HTTP header tag, and constructs a class C subnet that might be in use on your network. You may need to edit the mask portion of the subnet to match your actual subnets. The page updates every few seconds expanding the list of subnets as they are discovered. 1 On the Subnets page, click Auto-Discover Subnets link if you would like the appliance to suggest possible subnets that you want to configure. 2 Click Stop at any time to stop the auto-discover process and use the subnet list that has been compiled up to that point. All subnets that have been detected are listed (see the following illustration). 107
The following information displays in the returned list of Subnets. Table 13. Information displayed for subnets Data Subnets Client Packets Server Packets New TCP Connections Active TCP Connections Definition Displays the subnet number. The number of client packets being transmitted from clients to the server. NOTE: If the packet count is zero and the Server Packet value is greater than zero the network tap is not configured correctly. The number of packets being transmitted from the server to clients. NOTE: If the packet count is zero and the Client Packet value is greater than zero the network tap is not configured correctly. The number of TCP connections that have been observed on the server. NOTE: If the number of connections equals zero or the number is small in relation to the number of active connections, the load balancer is maintaining a limited number of TCP connections to the web server and using the proxy server to send all requests over the TCP connections. The number of active TCP connections discovered in the traffic for the subnet. 3 Select the check boxes for the subnets that you want the appliance to monitor. 4 Click Add. You are returned to the main Subnets page, and all newly added subnets are now listed. Checking for overlapping subnets Subnet definitions may be defined so that they overlap, meaning that a particular IP address will match more than one subnet definition. Trying to assess which subnets definitions are overlapping can be a difficult task to accomplish. Using the subnet mask, the system can determine whether subnet definitions are including the same IP address. Clicking Overlap Check at the bottom of the Subnet page launches the Subnet > Overlap Check page, which lists pairs of subnets and their subnet masks. Using the subnet mask, you can identify which part of the IP address is shared by each of the subnets. For example, in the following screen shot, AV Training shares the same IP prefix as Aliso Viejo. 108
Figure 51. Subnets: Overlap Check page Configuring a default subnet The default subnet allows users to configure a default subnet that will capture traffic for all IP addresses that do not match any of the configured subnets that appear on the Subnet page. Figure 52. Subnets: Default Subnets page To configure a default subnet: 1 From the Subnet page, click Default Subnet located at the bottom of the page. The Subnets > Default Subnet page appears. 2 In the Name field, type a name for the default subnet. 3 Select the Enabled check box. 4 Click OK. Removing subnet definitions You can delete a subnet entry by selecting its corresponding check box, then clicking Delete. 109
Check All can be used to select all entries for deletion, and Clear All is used to clear any entries that were selected for deletion. Managing subnets on a Portal When you access the Subnets page in a multiple-appliance configuration, the configuration list is identical on all appliances, including the portal. You can, however add subnet and mask pairs on the Subnets page on the portal. You cannot use the auto-discovery functionality, since the portal does not monitor network traffic. Identifying user sessions The User Sessions page, accessed by clicking Configure > Monitoring > User Sessions, allows you to define how the appliance should identify user sessions, and what login variables are in use on your site. Figure 53. User Sessions page To see user sessions that have been captured by the appliance, navigate to the User Sessions Log which can be accessed by clicking Analysis > User Sessions Log. 110
Figure 54. Example of user session log The log displays a list of user sessions that have been recorded by the system and is sorted by the time the user session ended. In the Identifier column either the login name or the IP address associated with the user session is displayed. Other information such as the name of an alarm triggered by the user session, the originating subnet or city, the duration of the session and the number of pages downloaded by the user is displayed. For more information, see these topics: How user sessions are identified Configuring user session identification Defining user session options Configuring client IP tags Augmenting user sessions log information Using logout patterns to detect completed sessions How user sessions are identified The most important configuration setting with regard to user sessions is the choice of a session identification variable. IMPORTANT: Without this setting properly configured, your appliance cannot accurately record user sessions. Session identification variables Session identification variables are the entities that are used by the applications you are monitoring to distinguish one user session from another. Typically, these are cookie variables but they can also be embedded in the URL either in the path, parameter or as a query variable. In some cases, form variables (POST parameters) can also be used to identify user sessions. 111
IMPORTANT: If you do not define any session identification variables, the system attempts to sessionize based on the user's client IP. It is strongly recommended that you do not rely on this functionality since it is frequently inaccurate in today's complex network environments due to the use of proxies, multiplexors and other devices. If you are not sure what to define as a session identification variable, speak to your application developers and ask them what they are using to track user sessions. Configuring user session identification Use the following steps to configure user sessions. To configure the appliance to accurately capture user sessions: 1 Provide session identification variables. See Managing user session identification variables. 2 Set the sessionizing mode. See Changing the sessionizing mode. 3 Configure Client IP Tags. See Configuring client IP tags. 4 Configure Login Variables. See Augmenting user sessions log information. 5 Configure Logout Patterns. See Using logout patterns to detect completed sessions. Managing user session identification variables The first step in configuring the system to track user sessions is to define the session identification variables that your application uses. A consultation with your application developers helps you determine what you should configure. If they are unavailable, a number of freely available tools, such as Mozilla Web Developer Add-on, can also help you determine what you need to configure. Adding a user session variable In the Session Identification Variables section, enter a variable used to identify user sessions, then click Add. The list of user session variables is updated to include your addition. Figure 55. Session identification variables The Session Identification Variables section lists all user session variables added to the appliance. NOTE: All defined user session identification variables are listed on this part of the User Sessions page unless your applications also include session identifiers in the path portion of URLs. In this case, define a path segment as the session identifier. Since they are represented by a location in a URL, as opposed to a set variable name, they are not listed here. Instead, they are found within the Path Rules definitions on the URL page. 112
TIP: If any of your applications append extra information to its user session variables, and this extra information is not relevant within the context of tracking user sessions (for example, an originating server name is appended to the user session variable) you can use URL transformations to remove this extraneous information before it is recorded by the appliance. For more information, see Transforming session identification variables. Prioritizing user session variables Use the and to establish the priority order with which the system evaluates user session identification variables. The system uses this in order to locate the session identification variable that it uses for a specific session. Using session identification variables auto-discover Rather than entering session identification variable names manually, use the Auto-Discover Variables feature to find variables used to identify user sessions. To auto-discover session identification variables: 1 On the User Sessions page, in the Session Identification Variable section, click Auto-Discover Variables. The appliance begins to detect session identification variables currently in use on your network. The Auto-Discovery page updates every few seconds, adding any newly detected variables to the list. 2 To cancel the search, click Stop. All session identification variables discovered up to that point are displayed in a list with accompanying check boxes. Each variable discovered provides the following information. Table 14. Auto-discover variable Data Definition Displays the name of the variable that are passed in the URL, POST data, Session Identification cookies, HTTP headers and Request headers. Variable You can select its check box and then click Add to configure it as a session identification variable. A high variation means that there are many different values that appear for the variable. Variables that are used for session identification are expected Variation to have a high variation ranking because they are different for every unique user session. Instances The number of times that the variable was found in the traffic. # Unique Values The number of unique values of the variable found in the traffic. Displays up to three of the most utilized values for the variable. Each Sample Values variable is no more than ten characters in length. 3 Select the check boxes that correspond to any Session Identification Variables you would like to use to build new variable rules. 4 Click Add. The User Sessions page is displayed, and the variables are added to the Session Identification Variables list. To prioritize the list, see Prioritizing user session variables. Removing User Session Variables Select the check boxes that correspond to the variables you would like to remove, then click the Delete button. 113
Defining user session options Access the User Sessions page by clicking Configure > Monitoring > User Sessions. The Options section enables you to configure sessionizing, capture and timeout settings. Figure 56. User session options Changing the sessionizing mode The appliance can use one of two sessionizing modes in order to detect and capture user sessions. By default, the appliance is configured to detect user sessions by individual hit. Sessionizing by hit If the appliance is configured to use hits to detect user sessions, every incoming hit is examined for user session variables. This means that a session can be easily identified, but the application needs to provide the same identifier in every hit for the duration of the session. Sessionizing by TCP connection When the appliance is configured to sessionize by TCP connection, it identifies a user session by detecting the session identification variable in the first few hits, then applies that determination to the remainder of the TCP connection. For some external applications such as Siebel, PeopleSoft and when monitoring SharePoint, this mode is useful because it can track a user session across changes in the session identification variable that can sometimes occur during a TCP connection. If your application behaves this way and you have not selected this mode, then the user sessions displayed in the User Sessions Log are fragmented whenever this occurs. Choosing the appropriate sessionizing method For the majority of applications, users can choose to sessionize by hit. This is especially important when monitoring Internet applications where end users are likely to be accessing the site through multiplexed TCP connections. With TCP multiplexing, the traffic from multiple end users uses the same TCP connection. When this occurs, the TCP connection sessionizing mode cannot properly distinguish user sessions. For some internal applications, like Siebel and PeopleSoft where TCP multiplexing is not in use, it may be more appropriate to sessionize by TCP connection. In general, you should try the sessionize by hit mode first and determine whether the user sessions collected by the system appears complete. If you see multiple user sessions that appear to be for the same user and are sequential in time, then that may be an indication that sessionizing by TCP connection may be more appropriate for your application. Defining how user sessions are recorded By default, all user sessions are recorded in the database. For performance reasons, it may be desirable not to capture every user session by clearing the Capture all User Sessions check box in the Options section of the User Sessions page. NOTE: If you clear this setting then only those user sessions that trigger an alarm are recorded. 114
Defining session timeout periods When the appliance has started recording a user session, it can use timeout periods in conjunction with logout patterns to determine when a user session terminates. The appliance has two timeout configuration settings that affect when a user session is considered terminated. User session timeout periods In the User Session timeout period box, you can adjust the amount of time a user session is inactive before the system considers the session to be terminated and writes the associated record to the database. Generally, you should only need to adjust this setting if your users tend to have long periods of time in which they leave an application inactive. By increasing the timeout period, you can cause user sessions that would have been regarded as distinct to be combined into one record. This setting should reflect the session timeout period that your application enforces. TCP timeout periods In the TCP timeout period box, you can adjust the timeout period that the system applies to inactive TCP connections before closing them. You may need to adjust this if your site makes use of proxies that keep TCP connections open without transmitting any data for extended periods of time. This setting should reflect the TCP timeout period that your application enforces. Configuring client IP tags By configuring the Client IP tags in the User Session page, you can define the tags of HTTP headers that contain IP addresses of client machines accessing your monitored applications. In typical network environments, client traffic is routed through a proxy server that replaces the originating client IP addresses with the IP address of the proxy server. The following HTTP headers are industry standards for identifying the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer, and display as the default Client IP Tags in the appliance: x-forwarded-for client-ip For more information, see Configuring client IP tags and Using X-Forwarded-For. If your site is using different HTTP headers than those listed above, you may notice that the client IP addresses appearing in the User Sessions Log are the same. To ensure that the correct originating IP addresses appear in the appliance web console, you will need to determine which HTTP header (if any) you need to configure in the Client IP Tags list. TIP: You can use either a browser plugin or Foglight Experience Monitor to determine the values in your HTTP headers. If there are no HTTP headers containing the originating IP address, contact your network administrator to determine if this feature can be enabled in your HTTP proxy or load balancer. This list is in priority order (if you have more than one header containing the Client IP) so make sure that the order reflects the configuration of your network. Use the and arrows to reorder the items in the list. Tags that are not in use on your site may be removed from the list although this is not required. Adding a client IP tag In the Client IP Tags section, enter the name of the tag that matches HTTP header containing the IP address of the client machine, then click Add. The list of logout patterns updates to include your addition. 115
Figure 57. Client IP Tags section Removing client IP tags Select the check boxes that corresponds to the Client IP Tag you would like to remove, then click the Delete button. Augmenting user sessions log information You can add information in the User Session Log by using the login names as identifiers. Using login names as identifiers Many applications check the credentials of users logging into the application by displaying a page that prompts the user to enter an account name and password. When the user enters this information, the login name is typically transmitted to the application server in a form (or POST) parameter. The appliance has the ability to extract these fields from the traffic and associate the login name for the user with the user session. This greatly facilitates the use of the User Sessions Log and makes it much easier to identify the user associated with a session. In the Login Variables section, you can define the names of variables that are used to contain login names in your application. While login variables are typically contained in form variables, the system can also extract them from cookies or query variables embedded in a URL. NOTE: This setting does not influence how user sessions are identified; it only affects how they are labeled in the User Sessions Log. With this configuration setting, you are able to see the login names of your users in the Identifier column of the User Sessions Log and in the All Metrics View report. You can also define an alarm based on the value of the login name that triggers whenever it matches a pattern that you define. IMPORTANT: Login names only appear in the User Sessions Log if the user actually enters their login name into a page during their session. Some applications, like Siebel, cache the user s credentials so they are not required to enter a login name to access the application every time. In these cases, the IP address appears in the log instead of the login name. If the application presents a login page before assigning a cookie that you are using as a session identifier, then the login name entered is not associated with the user session. Figure 58. Example of login names used as identifiers Adding a login variable In the Login Variables section, enter a variable that will be used to identify a user login, then click Add. The list of login variables is updated to include your addition. 116
Using auto-discover login variables Instead of entering variable names manually, you can choose the auto-discovery links to have the system generate a list of variables that are used to log in to the monitored application in the network traffic. For information about NTML authentication, see Extracting login names from NTLM. To identify login variables using auto-discover: 1 On the User Sessions page, in the Login Variables section, click Auto-Discover Login Variables. 2 To cancel the search, click Stop at any point. The list contains three columns: The Login column displays the name of the variable. The Instances column displays the number of times that the variable occurs in the traffic. The Sample Values column displays the three most utilized values for the variable. Each variable is no more than ten characters in length. NOTE: If you added a login variable called Authorization, login names using NTLM authentication protocol display as Authorization. 3 Select the check box beside the variable that you want to add and click Add. The variable name is added to the Login Variables list. Extracting login names from NTLM NTLM is a suite of authentication and session security protocols used in various Microsoft network protocol implementations. Originally used for authentication and negotiation of secure DCE/RPC, NTLM is also used throughout Microsoft's systems as an integrated single sign-on mechanism. It is recognized as part of the Integrated Windows Authentication stack for HTTP authentication; however, it is also used in Microsoft implementations of SMTP, POP3, and IMAP (all part of Exchange). Foglight Experience Monitor has the ability to decode NTLM authentication exchanges between clients and web servers, and extract the login name that users enter when logging into an NTLM-enabled system. To enable this functionality, you need only to add Authorization to the list of Login Variables. No other action is required. NOTE: Authorization will also show up when you choose the Auto-Discovery Login Variables function if NTLM exchanges are found in the monitored traffic. Removing login variables Select the check boxes that correspond to the login variables you would like to remove, then click the Delete button. Using logout patterns to detect completed sessions By default, the appliance considers a user session active until the timeout period has expired. (See Defining session timeout periods for more information about timeout periods.) Defining logout patterns helps the appliance determine if and when a user has actively logged out of their application. Being aware of a user that actively logs out, compared to waiting for a timeout period to be reached, means the user session is written to the User Sessions Log more quickly than it would be otherwise. This can assist with the troubleshooting of user issues in real time. 117
In the Logout Patterns section, you can enter a regular expression (e.g.,.*logout.*) that matched against the URL that the user is accessing. If the user accesses a page that matches any of the defined logout patterns (in this example, https://company.com/app/sessions?logout=true), the user session is considered terminated and the record is written to the database. Adding logout patterns In the Logout Patterns section, enter a regular expression meant to match a logout pattern in a URL, then click Add. The list of logout patterns updates to include your addition. Figure 59. List of logout patterns Removing logout patterns Select the check boxes that correspond to the expressions you would like to remove, then click the Delete button. Defining session identifiers embedded in URLs Some applications embed session identifiers in the path of a URL. For example the following URL: http://yoursite/yourpage.asp/234agt438c3k/index.html uses 234agt438c3k as the session identifier. If your applications identify user sessions by this method, you can define a path rule that the system uses for session identification. For general information on how transformations are performed on the path portion of URLs, see Defining path rules. For specific information, see Managing variable rules. User sessions and proxy servers Many network environments make use of Proxy servers that service the requests from clients by forwarding these requests to other servers. A client connects to the proxy server, requesting a web page, and the proxy server provides the page by connecting to the specified server and requesting the page on behalf of the client. Typically, the proxy server alters the client's request by removing the client's IP address and replacing it with the IP address of the proxy server itself so that it can receive the response from the application server before forwarding it on to the client. If this type of proxy server is in use on your network then you may see that the user sessions log displays the same IP address for all user sessions that are displayed as shown in the following illustration. Figure 60. Example of user session and proxy servers NOTE: The User Sessions Log displays identical IP addresses for different user sessions. 118
The X-Forwarded-For (XFF) HTTP header is a de facto standard for identifying the originating IP address of a client connecting to a web server through an HTTP proxy. XFF headers are supported by most proxy servers, notably Squid, Apache mod_proxy, F5 Big-IP, Blue Coat ProxySG, Cisco Cache Engine, Finjan's Vital Security, and NetApp NetCache. By enabling XFF headers in your proxy server, you allow the appliance to extract the user's IP from that header and display it in the user sessions log. See the documentation for the proxy server you have to determine how to enable XFF headers. No action is required in the appliance as it automatically searches for the XFF header by default. Monitoring instrumented web pages When a customer implements a Content Delivery Network (CDN), such as Akamai, Foglight Experience Monitor cannot automatically measure how long the CDN servers take to respond to a request for content. Therefore, the reported page load time becomes the time the in-house web server spends on the initial request for main.html before passing it on to a CDN server. This results in a partial page response. To get a more accurate page load time, customers can instrument their web pages so that Foglight Experience Monitor can track the time spent on CDN servers. This section describes how to instrument web pages and how to configure Foglight Experience Monitor to monitor the collected data. For more information on how the page load time is calculated in a CDN environment, see Instrumentation metrics in the Foglight Experience Monitor Metric Reference Guide. For more information, see these topics: Instrumenting web pages Configuring instrumentation options and metrics Instrumenting web pages To get started with instrumentation, use the dell-fxm.js JavaScript file provided on the FxM web console see the Instrumentation Script (dell-fxm.js) link at the bottom of the Instrumentation page, to track page load and other event times. For each page that you want to track using instrumentation metrics, you must include the JavaScript code as close to the top of the page as possible. For example: <!DOCTYPE HTML PUBLIC "~//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> <meta http-equiv="pragma" content="no-cache"> <title>dell - Your Smart Systems Management Company</title> <link rel="stylesheet" type="text/css" media="screen" href="styles.css" /> <script src="scripts/dell-fxm.js" language="javascript"></script> </head> <body onload="javascript:on_load();"> The JavaScript instrumentation code generates HTTP requests to the originating web site using the following format: http:dell.com/ rx/?&ets=load:305&uid=633943724234234198827&pn=http://dell.com/home.asp NOTE: By default, 404 Page not Found errors are transmitted as the response for these HTTP requests. It is recommended that customers set up their web servers to specifically return a 200 OK response, to avoid distorting other Foglight Experience Monitor metrics with false 404 errors. The following table describes the HTTP request fields. 119
Table 15. HTTP request fields Field Name uid pn ets Field Description Optional. Unique session ID for the page, either automatically created or overridden. Required. The page name. This field dictates the name that appears in the resource list for the Instrumented Page category. It can be either window.location() or any custom value. Required. Elapsed times serialized. Each element in this object is the name of the measure and its elapsed time; for example, slowjs(measure name):12(elapsed time). By default, ets is set to record load and unload times, but it can be set to different JavaScript events. For more information about these events, see http://www.w3schools.com/jsref/jsref_events.asp. The administrator needs to configure the Foglight Experience Monitor to recognize these HTTP requests as instrumentation hits containing metrics that are then funneled into the Instrumented Page category of metrics in the Foglight Experience Monitor metric database. For more information, see Configuring instrumentation options and metrics. If you need to define metrics other than the default metrics provided with the instrumentation script, your pages must include the appropriate instrumentation calls to record these metric values. For example, if your pages need to do some calculations at a certain point using JavaScript, you can place some instrumentation calls before and after the calculations, using dell-fxm.js, as presented in the following example. <script type="text/javascript"> window.fxm.start('calctime'); </script> <script type="text/javascript" src="calculate.js"></script> <script type="text/javascript"> window.fxm.stop('cacltime'); </script> The window.fxm.start() call starts the timer and the window.fxm.stop() call stops it. With the code presented in this example, you can see hits like these coming into the site: http:dell.com/ rx/?&ets=calctime:20,load:305&uid=633943724234234198827&pn=http://dell.com/home.asp By defining an instrumentation metric, you instruct the Foglight Experience Monitor system to begin tracking this metric in the Instrumented Page category. For more details about defining instrumentation metrics, see Adding instrumentation metrics. Configuring instrumentation options and metrics The Instrumentation page, accessed by clicking Configure > Monitoring > Instrumentation, allows you to configure your system to monitor pages that are instrumented for performance metrics. 120
Figure 61. Instrumentation page Setting instrumentation options The instrumentation options control what happens while Foglight Experience Monitor monitors instrumented web pages. You can set the following options: Transmit instrumentation hits to FxV While this option is enabled, information about hits is sent to Foglight Experience Viewer. Exclude instrumentation hits from other category metrics While this option is enabled, HTTP traffic generated by the JavaScript instrumentation module is excluded from all metric categories except the Instrumented Page category. Generally, this is the preferred behavior because it ensures that instrumentation traffic does not inflate the metrics for other categories. Specifying instrumentation URLs An instrumentation URL is a URL on the customer s web site that receives the instrumentation hits from the instrumentation code. By default, the dell-fxm.js script uses the /rx URL. You should add this URL to this list. If you modified the script to point to a different URL, include the modified URL in this list. You can perform the following tasks: To add a new URL, type the URL in the Add Instrumentation URL box and click Add. To delete an instrumentation URL, select its corresponding check box and click Delete. To view a real-time display of incoming hits from the instrumented pages deployed on the customer web site, click the Display Instrumentation Hits link. This auto-discovery mechanism checks the specified instrumentation URLs and displays incoming hits. 121
Figure 62. Instrumentation: Display Instrumentation Hits page To stop the auto-discovery operation, click Stop. The URL list is updated to display a details link for each instrumentation hit found in the monitored traffic. Click details to display additional details about an instrumentation hit. Reviewing the instrumentation file To review the dell-fxm.js JavaScript file provided with the FxM appliance, click the Instrumentation Script (dell-fxm.js) link at the bottom of the Instrumentation page. You have the option to open or to save the script file. Adding instrumentation metrics The Instrumentation Metrics section of the Instrumentation page allows you to manage the metrics that are tracked using the instrumentation. The Foglight Experience Monitor script collects some metrics by default, including Load Time and Unload Time. To review the default instrumentation metrics, see Viewing a complete list of instrumentation metrics. You can define up to 20 standard metrics (metrics with a mean, maximum, and minimum) and 20 counter metrics (simple counters). NOTE: The Load Time and Unload Time metrics are provided by the system as default metrics. You do not need to add them to your list of instrumented metrics. To add an instrumentation metric: 1 Click Add Instrumentation Metric. The Instrumentation: Instrumentation Metric page appears. 122
2 Type the metric name in the Name box. This is the metric name to be displayed in the web console (for example, Load Time). 3 In the Tag box, type in the tag used in the ets (elapsed times serialized) field of the instrumentation hits (for example, load). For details about this field, see Instrumenting web pages. 4 Choose the metric type by selecting one of the available option buttons: Standard metric or Count metric. 5 Click OK to save the metric. The Instrumentation page appears, with the new metric name displayed. 6 If you need to delete a metric, select its corresponding check box and click Delete. Viewing a complete list of instrumentation metrics After you finish defining metrics, you can review the metrics in the Metrics Analysis page. To review the list of instrumentation metrics: 1 Navigate to Analysis > Metric Analysis. 2 Find the Content section and expand Instrumented Page. 3 Click a metric to display its details. 123
Transforming monitored URLs 5 Configuring how the Foglight Experience Monitor handles URLs found in the monitored traffic is a critical component of a successful installation. URLs are the basic building blocks for several categories of metrics that provide insight into the performance of the applications that you are monitoring. This section describes how to configure URLs found in the monitored traffic. For more information, see these topics: Managing URLs Processing and aggregating URLs Managing URL transformation rules Defining site rules Defining path rules Managing variable rules Using hints Advanced URL options Managing URLs In this Resource List, all URLs monitored for the Hit category are displayed as they are captured. Within the web console, you can define transformation rules that determine how URLs discovered in your network traffic are converted into the URLs that appear in the user interface and are stored in the metrics database. IMPORTANT: Managing the list of URLs that the appliance collects is one of the most important administrative tasks that you will perform. Failure to implement an effective rule set for your site can result in the database being flooded with a large amount of URLs that make it difficult for you to find the metrics you want. In addition, if your site uses aliases, the metrics for a single web page can be split among multiple URLs appearing in the database unless you have defined the proper rules. The benefits of a sound data collection policy makes itself apparent when you view a metric analysis page for the Page and Hit categories. 124
Figure 63. Metric Analysis: Page - Page Stop Rate page The URLs shown above are stored in the metric database in the Page category. They represent the URLs that the appliance has analyzed in the network traffic that it has collected. For more information, see these topics: Encoded and decoded URLs Setting URL encoding in the Resource List Setting URL encoding for user sessions metrics in the All Metrics View Filtering data in the Resource List Encoded and decoded URLs Encoded URLs represent a sequence of characters from a particular character set that are paired with a whole number. Encoded characters that appear in the URLs can be decoded to reveal translated characters that are easier to understand. For example %20, when translated, appears as blank spaces. This URL is an example of an encoded URL. GET team.prod.quest.corp/sites/is-appl-dq/lists/time%20tacker/ EditForm.aspx?ID=2025&Source=http%3A%2F%2Fteam%2Eprod%2Equest%2Ecorp%2Fsites%2Fis%2 Dappl%2Ddq%2FLists%2FTime%2520Tacker%2FEntries%2520Sorted%2520by%2520Week%2Easpx%3F View%3D%7BEAE4AE96%2D1819%2D411F%2D8283%2DF7FDAAC861D2%7D%26FilterField1%3DResource %255Fx0020%255FNam%26FilterValue1%3DSandy%2520Dugan [HTTP] This is the decoded URL: GET team.prod.quest.corp/sites/is-appl-dq/lists/time Tacker/ EditForm.aspx?ID=2025&Source=http://team.prod.quest.corp/sites/is_appl_dq/Lists/ Time Tacker/Entries Sorted by Week.aspx?View={EAE4AE96_1819_411F_8283_F7FDAAC861D2}&FilterField1=Resource_x0020_N am&filtervalue1=sandy Dugan [HTTP] Setting URL encoding in the Resource List The Resource List enables you to select how you want to view URLs. You can switch between encoded and decoded URLs, allowing you to view URLs in the state that is most meaningful to you. 125
NOTE: The original URLs are always used when you are performing a query against data. The encoding and decoding options enables you to view choose how you want to view the URLs in the interface. To set URL Encoding or Decoding: 1 On any Resource List window, in the Search box, type a search string for information that you are interested in viewing. 2 Select Show URLs with original character encoding to encode the URLs. 3 Click Go. 4 Optionally, click Show URLs with decoded characters to decode the URLs that were returned in step 3. 5 Click Go. Setting URL encoding for user sessions metrics in the All Metrics View The All Metrics View page for User Sessions enables you to select how you want to view URLs. You can switch between encoded and decoded URLs, allowing you to view URLs in the state that is most meaningful to you. To set URL Encoding or Decoding: 1 Navigate to Analysis > User Sessions Log. 2 From the Identifier column, select a user session. The All Metrics View > User Session page displays. 3 Select one of the following options: 4 Click Go. Show URLs with original character encoding to encode URLs that were returned in step 2. Show URLs with decoded characters to decode the URLs that were returned in step 2. 5 Scroll to the User Session Pages and User Session Hits sections to view URLs related to that user session. Filtering data in the Resource List Use the following procedure to filter the Resource List. To filter data in the resource list: 1 The Resource List window displays all pages that have been monitored. 2 If desired, narrow the list to a more specific set of pages by typing a search string in the Search box. 3 Select the Interpret as a regular expression check box if you want to search using a Perl regular expression. 4 Click Go. This search is performed against the list of URLs using a Perl regular expression. Processing and aggregating URLs As the appliance encounters URLs in the stream of network traffic, it applies a set of transformations in order to arrive at a final URL. To examine this process a basic understanding of URL structure is necessary. 126
Consider the following example: http://www.mysite.com:80/travel/mexico.asp;cat=beaches?uid=12345 URLs are logically organized into five sections. Table 16. URL groups Site Port 80 Path Parameter Query The appliance processes each URL by splitting it into these sections, then applies optional transformation rules to each of them. The sections are then recombined, resulting in the final URL that the appliance stores in its database, and displays to users in the web console. Figure 64. URLs mapping www.mysite.com /travel/mexico.asp cat=beaches uid=12345 The appliance processes URLs retrieved from monitored network traffic. The extent to which the resultant URL differs from the original URL depends on the transformation rules defined. For more information, see these topics: Sending URLs to the Foglight Experience Viewer Transforming URLs example using variable rules Sending URLs to the Foglight Experience Viewer If you are using the Foglight Experience Monitor to transmit network traffic to a Foglight Experience Viewer, you can configure Foglight Experience Monitor to apply URL transformations on hits that are transmitted to the Foglight Experience Viewer. (See How Foglight Experience Monitor and Foglight Experience Viewer work together for more information on how Foglight Experience Monitor integrates with Foglight Experience Viewer.) Normally, when integrated with a Foglight Experience Viewer, the Foglight Experience Monitor is meant to act as a Foglight Experience Viewer s access point to the network, through which raw data is transferred, then written to a Foglight Experience Viewer s own database. Analysis can then be performed by a Foglight Experience Viewer users on unaltered data, and is independent of how the Foglight Experience Monitor transforms and records data for its own users. 127
Although transmitting raw URLs is typically most useful for a Foglight Experience Viewer users, there are situations where sending a transformed URL is preferred. When creating a Variable Rule or Path Rule in the Foglight Experience Monitor, you always have the option of having the rule apply to URLs that are transmitted to Foglight Experience Viewer. For information about Foglight Experience Viewer discards, see the Server by Port metric in the Foglight Experience Monitor Metric Reference Guide. For more information on Path Rules, see Defining path rules. For more information on Variable Rules, see Managing variable rules. Transforming URLs example using variable rules URLs that appear in monitored network traffic can be transformed into the URLs that appear in the Foglight Experience Monitor database by applying Variable Rules and configuration settings. The following examples explain how to transform URLs. Table 17. URL transformation example URL Configuration The URL as it appears in the traffic. The URL as it appears in Foglight Experience Monitor with no variable rules and the Show parameters in URLs option disabled. The URL as it appears in Foglight Experience Monitor after enabling the Show parameters in URLs option. The URL as it appears after defining a Variable Rule for the uid query variable. The URL as it appears after defining a Variable Rule for the action form variable. URL www.mysite.com/travel/ mexico.asp;cat=beaches?uid=12345 www.mysite.com/travel/mexico.asp;cat=xx?uid=xx www.mysite.com/travel/mexico.asp;cat=beaches?uid=xx www.mysite.com/travel/ mexico.asp;cat=beaches?uid=12345 www.mysite.com/travel/ mexico.asp;cat=beaches?uid=12345[action=search] As parameters, query variable or form variables are added to the appliance, the amount of URLs that appear in the Page and Hit database increases. In the following example, the parameter variable cat (category) is used to transform the following URL on the appliance: www.mysite.com/travel/mexico.asp;cat. Five different categories are displayed using the cat variable: www.mysite.com/travel/mexico.asp;cat=beaches?uid=xx www.mysite.com/travel/mexico.asp;cat=mountains?uid=xx www.mysite.com/travel/mexico.asp;cat=inland?uid=xx www.mysite.com/travel/mexico.asp;cat=lakes?uid=xx www.mysite.com/travel/mexico.asp;cat=rivers?uid=xx If a variable rule is defined for the unique identifier (uid) of a URL, and this variable has an unlimited number of values in the traffic, there will be a continuous growth of URLs that are stored in the Page and Hit databases this does not provide useful data for the users of the appliance. Exposing the form variable, action, allows the appliance user to see the performance of submit or search functions of the page on which the variable exists. For example, a URL that is configured to capture and report the overall search performance of our example URL might look like the following URL: www.mysite.com/travel/mexico.asp;cat=xx?uid=xx[action=search] This configuration allows all of the www.mysite.com/travel/mexico.asp URLs with the action of search (action=search) to be examined separately from an action of submit. By displaying parameter values and the form variable you can better understand the pages and the functionality of the web site. The following URLs display in the appliance database along with their own statistics: www.mysite.com/travel/mexico.asp;cat=beaches?uid=xx[action=search] 128
www.mysite.com/travel/mexico.asp;cat=beaches?uid=xx[action=submit] www.mysite.com/travel/mexico.asp;cat=mountains?uid=xx[action=search] www.mysite.com/travel/mexico.asp;cat=mountains?uid=xx[action=submit] www.mysite.com/travel/mexico.asp;cat=inland?uid=xx[action=search] www.mysite.com/travel/mexico.asp;cat=inland?uid=xx[action=submit] www.mysite.com/travel/mexico.asp;cat=lakes?uid=xx[action=search] www.mysite.com/travel/mexico.asp;cat=lakes?uid=xx[action=submit] www.mysite.com/travel/mexico.asp;cat=rivers?uid=xx[action=search] www.mysite.com/travel/mexico.asp;cat=rivers?uid=xx[action=submit] It is important to understand the logic regarding how the site was monitored and the depth of detail collected. It is also important to understand that you can significantly affect system performance by exposing a lot of details and thereby more data points can be tracked and correlated. Managing URL transformation rules Using the pages accessed by navigating to the Configure > URLs menu you can define a data collection policy that determines how the appliance processes monitored URLs, and transforms them into URLs that appear in the database. The development of your data collection policy can include the addition of site rules, path rules, query and variable rules, hints, page definitions, and advanced options. The following sections describe how to manage the information found in each page. IMPORTANT: Typically, URL transformation rules are established shortly after your appliance is initially installed and configured. If you make dramatic changes to the rules after the appliance has been monitoring and collecting data for some time, this can obsolete historical data in the database that may still show up in your reports. In this case, you may want to purge the page and hit resource list after modifying URL transformations. For detailed information about URL transformation rules, see the following sections. Table 18. URL transformation rules Rule Type Defining Site Rules Managing Path Rules Managing Variable Rules Advanced URL Options Information For more information, see Defining site rules. For more information, see Defining path rules. For more information, see Managing variable rules. For more information about purging resources, see Database metrics. Defining site rules Some web sites use aliases, as a means to redirect requests to another web site. For example, if you are monitoring the web site acorp.com, you may also be using aliases that include: acmecorp.com, acme.com, and acc.com. Without configuring the aliases, the appliance stores the primary web site URL and its aliases in its database, and captured metrics are divided among the primary web site and each alias. In order to report metrics accurately, the data for aliases and the primary web site should be aggregated into a single entry in the database. The Configuration > URLs > Site Rule page allows you to transform the site portion of the alias (information that appears before the.com) to the primary web site URL, creating a single site entry. The transformation rule also helps to configure lengthy URL names that display in the Metric Analysis Page and Hit categories. Using the Site Rule, you can shorten the length of the URL by transforming the site name. For example, if you see acme.sectors.application.corp.com you can transform this site URL to acme.com. 129
For more information, see these topics: Declaring site rules Automatically discovering site rules Editing and removing site rules Declaring site rules This procedure explains how to configure site rules. To declare site rules: 1 On the Site Rules page, click Add a Site Rule to display the Edit Site Rule page: 2 In the Find box, type the site name that you would like to execute from the URLs for Sites, Pages and Hits. Alternatively, use Select Site and select the site from the Resource List. 3 Click Test to test the Find sting against the current list of URLs. The Site Rule Test page displays a list of items from the resource list that match the Site Rule that you have entered in the Find box. 4 In the Display box, type the site name that you would like to see appear in the URLs for Sites, Pages, and Hits. Alternatively, use Select Site and select the site from the Resource List. 5 Click OK to accept the new site rule. The Site Rules section lists all define site rules. Automatically discovering site rules 1 On the Site Rules page, click the Auto-Discover Site Rules link if you would like the appliance to monitor the URLs in your traffic and suggest possible site rules to assist in transforming the URLs into a concise list. The auto-discovery process examines network traffic and looks in URLs for the most common used site name. This is considered the primary site name. Next the process finds sitenames that are common in structure and compares them to the primary sitename.the system constructs possible site rules that will help to transform the site portion of the alias that are similar to the primary sitename. A list of site rules displays. You can select the site rules that you want to apply to the URLs. During auto-discovery the following site names are not processed: 130
IP addresses sites that do not contain a dot (.). For example site.com. The page updates every few seconds expanding the list of site rules as they are discovered. 2 Click Stop at any time to stop the auto-discover process and use the site rules list that has been compiled up to that point. All site rules that have been detected appear in a checklist (see the following illustration). Editing and removing site rules To edit a site rule, click its corresponding Edit link. This displays the Site Rule window described in the previous section, allowing you to modify the site rules information. To remove a site rule, select its corresponding check box, then click Delete. Defining path rules The Configure > URLs > Path Rules page allows you to define rules that determine how the appliance transforms the path section of URLs. 131
The Path Rules page shows the list of path rules currently defined in your system. These rules are applied in the order in which they are listed. To reorder the list, click the up or down arrow associated with each path rule. The Enabled column indicates whether the corresponding path rule is currently active and in use. For detailed information about creating and managing path rules, see Managing path rules and Editing and removing path rules. Path rules are sometimes necessary when sections of the URL contain strings that cause the system to record multiple URLs when just a single URL is desired. This can happen when an application has more than one path that leads to the same URL. For example, the following URLs might refer to the same URL in the application. If that is the case, you would want to create a path rule that maps the second and third URLs to the first URL in the metric database: quest.com/support/home.asp quest.com/support_us/home.asp quest.com/support_na/home.asp For more information, see these topics: Segment rules as path rule components Segment rule actions Managing path rules Editing and removing path rules Segment rules as path rule components A segment is a section of the URL that is delimited by a forward slash (/). A path rule consist of multiple segment rules that are applied sequentially to URLs in the monitored traffic. You can edit segment rules by clicking the Add a Segment Rule link from the Path Rules > Edit Path Rules page. A segment rule defines what transformations are applied to a particular segment of the path, that is, replacing the existing string with a new string. For example, consider these three URLs: www.travel.com/regions/mexico/home.asp www.travel.com/provinces/quebec/home.asp www.travel.com/states/florida/home.asp Each URL contains three segments in its path: segment 1: regions, provinces, states segment 2: mexico, quebec, florida 132
segment 3: home.asp If you want to transform the first segment of each URL path so that regions, provinces, and states all map to the term areas, and the rest of the URL is recorded to the database as they appear, you need to define three-segment rules: 1 For the first segment, replace the existing string with the area string. 2 For the second segment, include the existing string. 3 For the third segment, include the existing string. Each of these segment rules are defined in the Edit Segment Rule window. NOTE: Since segment rules only apply to the path section of a URL, and not the web site or alias, there would be no rule needed for the www.travel.com portion of these sample URLs. As you define segment rules for a path rule, the sequence of segment rules is updated in the URLs > Edit Path Rule page. Segment processing Segment rules are processed in the order in which they are listed. Each segment rule must match the current URL segment before the next segment rule is processed. If there is no match, and the segment rule is marked as 133
Required, then this path rule does not apply to the URL in question. If the segment rule is not required, then processing advances to the next segment rule. After all segments of the URL have been matched by the segment rules and all segment rules that are flagged as required are successfully matched, the entire URL is modified according to the segment rules as a whole (for example, the path rule). For example, consider a site with the following URLs: quest.com/support/path1/path2/page3.asp quest.com/support_au/path1/page2.asp quest.com/support_ch/page3.asp To change the first segment of the path support, support_au, and support_ch so that it displays as support in the metric database, you create the rules shown in the following image. Figure 65. Path Rules: Edit Path Rule page The first rule matches when the first segment in a URL starts with support, and replaces it with support. Note that this rule matches only URLs with one segment, such as quest.com/support_ch/. However, the test data includes URLs with one or more subsequent segments, such as quest.com/support/path1/path2/page3.asp. The second rule instructs the system to match URLs with any number of subsequent segments. Segment rule actions The following section outlines the types of actions that a segment rule can perform on URL segments. Including or removing segments The Include action includes the segment in the transformed URL while the Exclude action removes the segment in the transformed URL. Options in the Segment Rule s Action box performs basic operations on the matched URL segment: Include keeps the segment as is in the resulting URL, while Exclude removes it altogether. Replacing segments Selecting the Replace segment rule action causes the segment to be replaced by the string in the Replacement string box if it matches the string in the Matching string box. For example, consider the following path section of a monitored URL: /products/database/mysql1/download.asp Next, consider the following sequence of segments rules. 134
Table 19. Replacing segments example Match Action Matching String Replacement String Required Reg. Expression Exclude products yes Reg. Expression Replace database database_products yes Reg. Expression Include mysql yes Reg. Expression Include download.asp yes When these sequence rules are applied together as a path rule to the monitored URL segments, the resultant URL is: /database_products/mysql1/download.asp Including segments while matching The Include while matching segment rule action specifies that any segments matching the value in the Matching string box is included in the transformed URL. The rule begins with the current URL segment. If the URL segment matches the segment rule, it is marked for inclusion, and processing continues to the next URL segment. If the next URL segment matches the segment rule, it too is marked for inclusion. This processing continues until a match fails, and the appliance advances to the next segment rule and continues processing. A segment rule that uses this action can be used to simplify a path rule. A series of segment rules whose action is set to Include can be replaced by a single segment rule whose action is set to Include while matching. In the previous example, four segment rules were used to transform the URL. As an alternative, the following sequence of three segment rules can be used instead. Table 20. Including segments while matching example Match Action Matching String Replacement String Required Reg. Expression Exclude products yes Reg. Expression Replace database database_products yes Reg. Expression Include while matching.* yes The third segment rule combines the Include while matching action with the regular expression to match any string (.*). This rule essentially continues to include URL segments in the resulting URL until they have been exhausted. Excluding segments while matching The Exclude while matching segment rule action specifies that any segments matching the value in the Matching string box is excluded from the transformed URL Similar to inclusion, the rule begins with the current URL segment. If the URL segment matches the segment rule, it is marked for exclusion, and processing continues to the next URL segment. If the next URL segment matches the segment rule, it too is marked for exclusion. This processing continues until a match fails, and the appliance advances to the next segment rule and continues processing. A segment rule that uses this action can be used to simplify a path rule. A series of segment rules whose action is set to Exclude can be replaced by a single segment rule whose action is set to Exclude while matching. Consider the following URL: http://newssite.com/2006/01/25/news/companies/quest/index.htm where the path portion of the URL is: /2006/01/25/news/companies/quest/index.htm 135
Using this newssite.com URL as an example, every URL begins with the current date. If you wanted to aggregate the performance of your organization s news articles about Quest Software, regardless of the articles publishing date, you want the path portion of the URL transformed to: /news/companies/quest/index.htm Table 21. Excluding segments while matching example Match Action Matching String Replacement String Required Reg. Expression Exclude while matching [0-9]+ yes Reg. Expression Including while matching.* yes The first segment rule uses a common regular expression to match any string consisting solely of numbers. This effectively matches all the URL segments that represent the article date, and excludes them from the resulting URL. Excluding segments until a match The Exclude until match segment rule action discards one or more non-matching URL segments until a segment match is found, after which, the matching URL segment is included in the resulting URL. The rule begins with the current URL segment. If the URL segment does not match the segment rule, it is marked for exclusion, and processing continues to the next URL segment. If the next URL segment also does not match, it too is marked for exclusion. This processing continues until a match has occurred, after which the combined URL segments are discarded, and the matching segment is included in the resulting URL. The appliance then advances to the next segment rule and continues processing. Consider the following path sections of several fictional newssite.com URLs: /2006/01/25/news/companies/quest/index.htm /2005/08/19/news/companies/quest/index.htm /2004/jan13/news/companies/quest/index.htm As was the case in the previous example, all news URLs begin with a date. If you wanted to aggregate the performance of all news articles, regardless of publishing date, you would want the path portion of the URL transformed to: /news/companies/quest/index.htm In this case, you cannot use the Exclude While Matching action because newssite.com has used non-numerical date formats in the past. Therefore, you cannot simply exclude any segments that contain numerical values. Instead, the following sequence of segment rules could be used: Table 22. Excluding segments until a match example Match Action Matching String Replacement String Required Reg. Expression Exclude until match news yes Reg. Expression Include while matching.* yes The first segment rule discards all URL segments found until it finds a segment that matches its news string. Next, the date is effectively stripped from the URL, and the second segment rule takes effect, and includes the rest of the monitored URL in the resulting URL. The Exclude until match and replace action is identical to Exclude until match, the only difference being that the matched segment is replaced with a specified string, instead of simply included in the resulting URL. 136
Including segments until a match The Include until match segment rule action combines one or more non-matching URL segments until a segment match is found. Then the segment match is found, it is excluded from the resulting URL, and the previously combined non-matching URL segments are included in the resulting URL. The rule begins with the current URL segment. If the URL segment does not match the segment rule, it is marked for inclusion, and processing continues to the next URL segment. If the next URL segment also does not match, it is also marked for inclusion. This processing continues until a match has occurred, after which the combined URL segments are included, and the matching segment is discarded. Considering the following path section of a URL: /news/companies/quest/2006/01/25/index.htm Similar to the previous two examples, assume that you are interested in aggregated data on news articles, regardless of date. You want the path portion of the URL to be transformed to: /news/companies/quest/index.htm The following sequence of segment rules could be used: Table 23. Including segments until a match example Match Action Matching String Replacement String Required Reg. Expression Include Until match [0-9]+ yes Reg. Expression Exclude Until Match index.htm yes The first segment rule includes all URL segments found until the match, the latter representing the date that is unwanted in the resulting URL. The first URL segment is discarded as part of the Include Until Match rule, after which the second segment rule initiates, excluding the remaining URL segments until the final segment is found (index.htm). The Include until match and replace action is identical to Include until match, the only difference being that the matched segment is replaced with a specified string, instead of being excluded from the resulting URL. Replacing any matched segment The Replace any match segment rule action examines all remaining URL segments, looking for a match. Any URL segment that matches is replaced, while segments that do not match remain unmodified. Consider the following path section of a URL: /news/companies/quest/2006/01/25/index.htm Assume the desired resulting URL is: /news/companies/quest/xx/xx/xx/index.htm and you want to ensure that only news-related URLs are replaced. This can be accomplished with the following sequence of segment rules: Table 24. Replacing any matched segment example Match Action Matching String Replacement String Required Reg. Expression Include news yes Reg. Expression Replace Any [0-9]+ XX yes The first action transforms the URL if and only if the /news/ segment is found, after which all numerical segments that represent the date are masked with XX. The Replace any match action always applies to all remaining URL segments. 137
Identifying a user session The Session Identifier segment rule action allows you to specify a segment within the URL that should be used for identifying user sessions. This is useful for cases where the monitored application embeds session identifiers in the URL itself rather than in a cookie (for example, quest.com/support/1042004/home.asp). The Session Identifier Transform segment rule allows you to specify a replacement string that will appear in the URL in place of the session identifier. See Managing variable rules for more information. Managing path rules The path rule definition and its component segment rules are displayed on the Path Rules > Edit Path Rule page. You can manage path rules by editing the path rule definition and by adding, editing, removing, or reordering its segment rules. You can also click Test at the bottom of the page to open the Test Path Rule dialog box and test your path transformation rule. To manage a Path Rule: 1 On the Path Rules page, click Add a Path Rule to display the Path Rules > Edit Path Rule page. 2 In the Name field, assign a symbolic name to the path rule. 3 In the Description field, enter the description of this path rule and its purpose. 4 If you want to activate this path rule, select the Enabled check box. 5 If you want the path rule to also apply to hits transmitted from Foglight Experience Monitor to Foglight Experience Viewer, select the Apply to Foglight Experience Viewer hits check box. See Sending URLs to the Foglight Experience Viewer for more information. 6 Click Add a Segment Rule to begin constructing your path rule. The Edit Segment Rule dialog box displays. 138
7 In the Match list, select the type of matching the segment rule should perform. 8 In the Action list, select the action the rule should perform on the segment. 9 In the Matching String field, specify the string to be included in the segment rule, if applicable. NOTE: This field only appears if Regular Expression was chosen from the Match list. 10 In the Replacement String field, specify the replacement string to be included in the segment rule, if applicable. 11 Ensure the Required check box is selected, if the segment is required to appear in the URL that is captured from monitored network traffic. 12 Select the Append a slash check box if the segment rule should append a slash ( / ) to the transformed segment. 13 Click OK to complete the definition of this segment rule. You are returned to the Path Rules > Edit Path Rule page, and your segment rule is appended to the current list. 14 Continue to add segment rules as described in this procedure. The following illustration represents an example of a path rule for which three segment rules have been defined. 15 When your segment rule set is complete, click Test to display the Test Path Rule window. 139
16 Enter test URLs into the Input URL field, then click Test. NOTE: Even though you are building rules for the path portion of a URL, in the Test Path Rule dialog box you can input the full URLs you expect the appliance to monitor. For example, if the URL entered in the dialog box is www.travel.com/states/florida/home.asp any of the following URLs would be successfully processed: http://travel.com/states/florida/home.asp travel/states/florida/home.asp //travel/states/florida/home.asp 17 Continue testing until you are satisfied that the segment rules you have defined for your path rule are exhaustively complete, then click Close. 18 On the Path Rules > Edit Path Rule page, click OK to accept the completed path rule definition. You are returned to the main Path Rules page, where your latest path rule definition is listed. Editing and removing path rules On the Path Rules page, all current definitions are displayed. To edit a path rule definition, or enable/disable a rule, click its corresponding Edit link to display the Path Rules > Edit Path Rule page, where you can modify the path rule definition. To remove a path rule altogether, select its corresponding check box, then click Delete. Managing variable rules The Configuration > URLs > Variable Rules page lists all defined variable rules, and allows you to define new rules. 140
Figure 66. Variable Rules page Variable rules allow you to specify how the system transforms the query and parameter segments of URLs, and also define how form variables (POST parameters), XML variables, cookies, and HTTP headers are appended to URLs that appear in the metrics. In the Variable Rules section, all currently defined variable rules are listed, and the Type column indicates whether the corresponding variable rule applies to a query variable, form variable, HTTP variable, parameter, XML or Cookie variables. NOTE: If you have enabled SharePoint monitoring, default variable rules automatically appear in the list. For more information, see Monitoring Microsoft Office SharePoint Servers. For more information, see these topics: Transforming query variables Transforming form variables Transforming session identification variables Performing actions on variable names and values Transforming query variables When transforming URLs found in the network traffic into the URLs that appear in the database, the system masks the values of all query variables that appear with XX. This is done so as to prevent the number of URLs from rapidly proliferating as can happen when the values of a query variable continually change. An example of this is when a query variable contains a timestamp that changes every second. To expose the values for any query variable, you can define a variable rule which causes a unique URL to be written to the database for every combination of values seen for the query variable. For example, consider the following URL, in which id and task are query variables: 141
http://www.abc.com/test?id=12&task=1 If a variable rule for id was previously configured in the appliance, a different URL is stored in the database each time the id has a different value: http://www.abc.com/test?id=12&task=xx http://www.abc.com/test?id=132&task=xx http://www.abc.com/test?id=1234&task=xx When configuring query variables, it is important to specify only those variables that have a limited range of values. For example, the following URL has includes resort and time as query variables: http://www.mysite.com/travel/mexico.asp?resort=acapulco&time=0830 In this case, you can define a variable rule that exposes resort which represents a fixed set of resorts in Mexico. However, it would probably not be appropriate to expose time which represents the time of day. If a variable rule was configured for time, the appliance would generate a new URL that would appear in the Page and Hit categories every time this value changed. Within a single URL, the appliance alphabetically orders URL queries based on the query variable name. For example, the following URL: http://www.abc.com/test?d=12&e=1&a=2 appears in the database as: http://www.abc.com/test?a=2&d=12&e=1 Transforming form variables Form variables are HTML constructs that appear in HTTP POST requests. These are typically tied to HTML elements that allow the user to enter data in boxes and pick lists, and are posted to the application server when the user clicks a Submit button or preforms a similar action that transmits user-supplied data to the application. If desired, you can have the appliance monitor the values of form variables and append them to the URLs that appear in the database by creating the appropriate variable rule. For example, consider an HTML form that uses a hidden variable named step, which is assigned a number identifying the current stage of a purchasing transaction. If no form variable has been defined, any instance of this page appears in the database as: /purchase.html Without a defined form variable, you are not able to evaluate each step of the transaction, since all steps are aggregated into a single data set. However, if the step form variable is defined, the appliance records web pages for all steps: /purchase.html[step=1] /purchase.html[step=2] /purchase.html[step=3] Transforming session identification variables Transformations can also be performed on user session identification variables that have already been defined in the appliance, see Identifying user sessions. Typically, session identifiers are cookies, although they may also be query, form or parameter variables. With most monitored applications, the session identifier remains constant for the entire length of the user's session. However, if some portion of the session identifier changes during a user s session (for example, an application appends extra information such as a server name or IP address to the session identifier) the appliance interprets this change as the start of a new user session. The end result is truncated user sessions, which are displayed in the User Sessions Log, reflecting only a subset of the user s actual activity. To configure the appliance to accurately track a user session whose identifier changes during the course of the session, define a variable rule that ignores the changing portion of the session identifier. 142
The variable you change should already have been defined in the Session Identification Variables section of the User Sessions page. For more information about User Sessions, see Identifying user sessions. Performing actions on variable names and values All variable rule definitions and modifications are made on the Variable Rules > Edit Variable Rule page. Figure 67. Variable Rules: Edit Variable Rule page Configuration of a variable rule requires that you specify actions on both parts of the name-value pair. For example, consider the URL www.store.com?catalog=pet. This URL consists of a single variable whose name is catalog, that may have many values such as pet. By defining a variable rule, you can control how both the name and value are transformed when the appliance adds the URL to the database. You could choose to ignore catalog altogether, to mask its values as XX, or to expose all of its values in the database. The operations that can be performed on each half of the pair are respectively found in the Name Action and Value Action sections. Transforming variable names The Name Action section allows you to identify the variable name and type, as well as apply transformations to it. Declaring the variable The Variable name box and Evaluate as regular expression box allow you to specify which variables are affected by the rule. The name that is entered must precisely match that which you wish to find in monitored URLs. For example, if you enter catalog as the variable name, this sample URL: www.store.com?catalog=pet will be affected by this rule, but this URL: www.store.co.uk?catalogue=pet will not be transformed. 143
By selecting the Evaluate as regular expression check box, the value that you entered in the Variable Name box is treated as a regular expression. With this setting selected and the string catalog.* specified in the Variable Name box, this rule would affect both of the URLs: www.store.com?catalog=pet www.store.co.uk?catalogue=pet The Variable type list allows you to specify the variable type: Query Variable these are query strings that appear after the question mark in a URL, and are the most common type of variable to which you can apply transformations. Form Variable these are variables that appear in HTTP POST requests, and are typically tied to HTML elements that allow the user to enter data, such as text entry boxes. XML Variable these are variables that appear in HTTP POST requests, but are encoded as XML. For example, if a POST is submitted to this URL: http://www.store.com/entryform.asp and the following XML structure is passed in that HTTP POST on submission: <Screen Action= Save Panel= Address > the URL will appear in the system as: http://www.store.com/entryform.asp However, if Screen.Action is declared as an XML variable, then the URL would appear as: http://www.store.com/entryform.asp [ Screen.Action= Save ] Cookie Variable these are variables that appear in cookies contained in the HTTP header. From the perspective of end users, cookies are transparent, and are not visible in URLs. Parameter Variable these are values that appear after the path portion of a URL. HTTP Variable these are values that appear in the HTTP headers and contain information about the client, server and the web page. Some applications can create custom HTTP headers that contain useful state information relevant to the application itself. In these cases, you can configure these HTTP variables and expose their values in the user interface. Retaining variable names When you select the Show as is option button, the name portion of the name-value pair is not transformed, and only the operation performed on the value portion of pair changes the URL. In the case of a Cookie variable, the name portion of the name-value pair is appended to the recorded URL. If you are identifying an XML variable in the Variable type box, this is the only available option. You cannot transform the name portion of an XML variable. Replacing variable names If you are identifying a Query Variable, Form Variable, Cookie Variable, Parameter Variable, or HTTP Variable in the Variable type box, you can replace the name value of the variable with either a literal string, or a regular expression. Selecting the Replace with option button requires that you enter a direct replacement string in the accompanying text entry box. Selecting the Transform with option button necessitates the entry of a regular expression in the accompanying text entry box. Removing variable names Select the Hide option button if you would like to completely remove the name portion of the name-value pair from the recorded URL. This option can be performed only on a Query Variable, Cookie Variable, Parameter Variable, or HTTP Variable, as selected in the Variable type box. When you hide the name portion of a name-value pair, all options in the Value Action section are disabled. 144
Transforming variable values The Value Action section allows you indicate what type of operation you want to perform on the value portion of the name-value pair. Leaving the value unchanged When you select the Show as is option button, the value portion in the URL is not modified before being recorded to the database. Replacing the value Selecting the Transform option button requires that you enter two regular expressions in the accompanying text entry boxes: the first being the string to search for in the value portion of the name-value pair, and the second being the string that replaces the detected string, before the URL is recorded to the database. Masking the value Selecting the Mask with XX option button causes all values of the name-value pair to be replaced with XX in the transformed URL. Decoding the value as XML Selecting the Decode as XML option button indicates XML is embedded in the variable value, and needs to be expanded. Consider the following example: http://www.store.com/page.asp?xml=<screen Action= Save Panel= Address > In this URL, there are two XML variables in the XML string: Screen.Action, and Screen.Panel. By default, the appliance would detect and display the URL as: http://www.store.com/page.asp?myxml=xx In the Name Action section, if you identify xml as the variable name, and in the Value Action section, choose to Decode as XML, the URL would transform to: http://www.store.com/page.asp?xml=<screen Action=XX Panel=XX> If you continued by defining another variable rule for Screen.Action, the URL would then be transformed to: http://www.store.com/page.asp?xml=<screen Action= Read Panel=XX> Decoding the value as a URL Selecting the Decode as URL option button indicates the variable value is an embedded URL, and needs to be expanded. Consider the following example: http://www.store.com/page.asp?step=http://www.store.com/ actualstep.asp?action=save&session=12345 In this URL, you can create a variable rule that instructs the appliance to decode step as a URL. This allows you to further manipulate variables that are embedded in the step URL (that is, action, and session). By default, the appliance would detect and display the URL as: http://www.store.com/page.asp?step=xx If you identify step as a variable name in the Name Action section, and in the Value Action section, choose Decode as URL, the URL would transform to: http://www.store.com/page.asp?step=http://www.store.com/ actualstep.asp?action=xx&&session=xx If you continued by adding an additional variable rule for action, the URL would then be transformed to: http://www.store.com/page.asp?step=http://www.store.com/ actualstep.asp?action=save&&session=xx 145
Managing variable definitions The following section discusses how to add variable rules. Adding variable rules To create a variable rule: 1 On the Configure > URLs > Variable Rules page, click Add a Variable Rule. The Variable Rules > Edit Variable Rule page displays. 2 In the Name Action section, begin by entering a string in the Variable name box that identifies the variable that appears in monitored URLs. If you are entering a literal string, the name must match the actual variable name. 3 Select, or leave the Evaluate as regular expression check box clear, depending on whether you want to use a regular expression to define multiple variable names, or use a literal string to find an exact match. 4 From the Variable type list, select whether the variable that you are looking for is a query, form, XML, cookie, parameter, or HTTP variable. 5 Select one of the name action option buttons to configure how the variable name is transformed, if at all. 6 If you wish the path rule to also apply to hits transmitted from the Foglight Experience Monitor to the Foglight Experience Viewer, enable the Apply to Foglight Experience Viewer hits box. For more information, see Sending URLs to the Foglight Experience Viewer. 7 In the Value Action section, select one of the option to configure how the variable rule transforms the variable value. 8 Click OK to save the variable rule definition. You are returned to the URLs page, and the variable rule list is updated to reflect your latest addition. Auto-discovery for variables and parameters Rather than entering variable names manually, you can choose the auto-discovery links and have the system generate lists of variables that currently appear in the monitored network traffic. IMPORTANT: The following procedure discusses auto-discovery specifically for form variables. You can follow this same procedure for parameters, query, and HTTP auto-discovery. To auto-discover variable and parameters: 1 On the Configure > URLs > Variable Rules page, click Auto-Discover Form Variables. The appliance begins to detect URL form variables currently in use on your network, and the Auto- Discovery page updates every few seconds, adding any newly detected query variables to the list. 146
2 Click Stop at any point to cancel the search, and use the list that has been compiled up to that point. All form variables discovered up to that point are displayed in a list with accompanying check boxes. Each form variable discovered provides the following information. Table 25. Information provided for discovered form variables Data 3 Clicking the details link for a variable opens a dialog box that displays a list of values detected for the variable. The value for the variable (for example EMC-TIGERMAIL), and the percentage that this value represents of all variables (24) displays in this window. Close the window when you have finished reviewing the values. 4 Select the check boxes that correspond to any form variables that you would like to use to build new variable rules. 5 Click Add. Definition Displays the names of the variables that are passed in the URL, POST data, cookies, HTTP request headers, and HTTP response headers. Form Variable You can select its check box and then click Add to include it in a new variable rule. A high value (number) in this column means that there are many different values that appear for the variable. Ensure you do not select query variables that are unique for every user that connects through the monitored application. Unique variables generate unique URLs, all of which are recorded to the Variation database; an excessively large number can result in an overloaded appliance on which data loss may occur. Review Transforming query variables, Transforming form variables and Transforming session identification variables for information on variable transformations. NOTE: Use only those variables with a low number of Unique Values Instances The number of times that the variable was found in the traffic. # Unique Values The number of unique values of the variable found in the traffic. Displays up to three of the most utilized values for the variable. Each Sample Values variable is no more than ten characters in length. 147
On the Variable Rules page, the new rules appear that are based on the newly added form variables. NOTE: Currently, all the rules that are based on the form variables that you added from the autodiscovery process are configured to retain the original form variable name and value (for example, both name and value actions are set to Show as is). 6 If required, click a newly added variable rule s corresponding Edit link to display its Variable Rules > Edit Variable Rule page, and reconfigure its name or value actions. For more information on these actions, see Transforming variable names and Transforming variable values. 7 Repeat the previous step for all newly added variable rules. Enabling variable rules for SharePoint services If you are monitoring SharePoint servers, you can automatically populate the list of variable and query rules by clicking Apply on the Configure > Applications > SharePoint page. After the SharePoint monitoring is enabled the list of default variables appears in the variable rules sections. Using hints Foglight Experience Monitor is typically able to identify which URLs represent the start of a new page download, and then from these URLs the appliance can correctly identify which URLs are hits that belong to the page. However, there may be cases where the appliance is unable to correctly distinguish a page from a hit. This can occur when a URL is not recognized as the starting point for a new page download, or when a URL is incorrectly recognized as a new page, when it is actually an element of the current page download. These situations typically arise with HTML frame-based applications. You can use the URLs > Hints page to create hints, in the form of regular expressions, that are matched against incoming URLs to force URLs to either always be treated as the start of a new page, always be treated as a hit that is part of another page download, or treated as a component of a page. 148
Figure 68. Hints page These page hints ensure the accuracy of the appliance page and hit metrics for highly complex web applications. For more information, see these topics: Defining page hints Defining hit hints Defining asynchronous page hints Defining asynchronous hit hints Testing hints Deleting hints Using page definitions Defining page hints A page hint tells the system to always treat the URL as the start of a new page. URLs matching the page hint will never be treated as components of another page. If a page download is in progress when this URL appears in the traffic, that page download will be considered complete. To define page hints: 1 On the URLs > Hints page, click Add a Page Hint. The Page Hint window displays. 2 In the Hint box, enter the regular expression that is matched against incoming URLs. 149
3 Click OK. You are returned to the URLs > Hints page, and your hint is now listed in the Page section. For information about testing page hints, see Testing hints. Defining hit hints A hit hint tells the system to always treat the URL as a hit rather than a page. URLs matching a hit hint are never considered to be the start of a new page download. When these URLs appear in the traffic they do not cause the system to consider an ongoing page download to be complete. If a page download is in progress, these URLs are treated as components of that page download. Defining a hit hint is similar to creating a page hint. To define hit hints: 1 On the URLs > Hints page, click Add a Hit Hint. The Hit Hint window displays. 2 In the Hint box, enter the regular expression that will be matched against incoming URLs. 3 Click OK. You are returned to the URLs > Hints page, and your hint is now listed in the Hit section. For information about testing page hints, see Testing hints. Defining asynchronous page hints An asynchronous page hint tells the system how to recognize asynchronous URL requests that are commonly used in Rich Internet Applications (RIAs) to enhance interactivity of a web application without requiring a full page download. Ajax (asynchronous JavaScript and XML), or AJAX, is a group of interrelated web development techniques commonly used by RIAs to enhance interactivity. A primary characteristic of RIAs is the increased responsiveness of web pages achieved by exchanging small amounts of data with the server so that entire web pages do not have to be reloaded each time there is a need to retrieve data from the server. This is intended to increase the web page's interactivity, speed, functionality and usability. Ajax is asynchronous, in that extra data is requested from the server and loaded in the background without interfering with the display and behavior of the existing page. By defining an asynchronous page hint for a set of URLs, you tell the system to effectively treat these URLs as standalone pages. When these URLs are seen in the traffic they do not affect the analysis of any existing page downloads. They are treated as the start of a new page download and do not cause an existing page download to be considered complete. In addition, these URLs are not treated as page components of an existing page (in contrast to a hit hint). To define Asynchronous hints: 1 On the Configure > URLs > Hints page, click Add an Asynchronous Page Hint. The Hit Hint window is displayed. 2 In the URLs > Hint box, enter the regular expression that will be matched against incoming URLs. 3 Do one of the following: 150
In the Hint box, enter the regular expression that will be matched against incoming URLs. OR 4 Click OK. In the Tag box, enter the HTTP header with a value that will be matched against incoming URLs. In the Value box, enter a value for the HTTP header that you want match against. 5 You are returned to the URLs > Hints page, and your hint is now listed in the Asynchronous Page Hint section. For information about testing page hints, see Testing hints. Defining asynchronous hit hints By defining an asynchronous hit hints for a set of URLs, you tell the system to effectively treat these URLs as a hit rather than a page. For a general explanation about Rich Internet Applications (RIAs) and Ajax (asynchronous JavaScript and XML), see Defining asynchronous page hints. URLs matching an asynchronous hit hint are never considered to be the start of a new page download. Asynchronous hit hints are always excluded from page download metrics. The asynchronous hit hints will display as individual Hit category metrics but do not display in the Page category metrics. To define Asynchronous hit hints: 1 On the Configure > URLs > Hints page, click Add an Asynchronous Hit Hint. The Hit Hint window is displayed: 2 In the Hint box, enter the regular expression that will be matched against incoming URLs. 3 Do one of the following: OR 4 Click OK. In the Hint box, enter the regular expression that will be matched against incoming URLs. In the Tag box, enter the HTTP header with a value that will be matched against incoming URLs. In the Value box, enter a value for the HTTP header that you want matched against. 5 You are returned to the Hints page, and your hint is now listed in the Asynchronous Hit Hint section. For information about testing page hints, see Testing hints. Testing hints After adding a hint (page, hit, asynchronous page, asynchronous hit), you can verify that there are URLs that match the hint. Click the Test link to the right of any hint. The Hint Test window displays a list of all resources that match the hint that you have entered. 151
Figure 69. Hint Test page Deleting hints To delete any hint, select its corresponding check box, then click Delete. Using page definitions Page definitions allow you to define a page by identifying all of the components on that page. This is usually not needed, but may be required if your site makes heavy use of HTML frames. In this case, the system might not be able to accurately determine the components of a page and the metrics for the page will be inaccurate. Figure 70. Page definitions example To see the list of components that are considered to be part of a page, click the Hits link shown in the Diagnosis section of the Metric View for the page. 152
Figure 71. Viewing a metric for a page When viewing a metric for a page (Figure 71), users can view all hits associated with the page. Figure 72. Viewing hits associated with a page The appliance is normally able to determine which hits are associated with a page by employing internal algorithms. By knowing which hit URLs reside on a page, the appliance can accurately calculate metrics for that page. However, in some cases, the appliance cannot correctly associate hits with a page. This can result in some hit URLs not being associated with its proper parent page, or some URLs not being recognized as the beginning of a new page download. Creating page definitions When it is difficult for the appliance to discern what constitutes a complete page, you can create page definitions to describe these complete pages for the appliance. Using a page definition overrides the appliance s default page structuring processes, and allows you to define a page by explicitly identifying those hits via their URLs. 153
To create page definitions: 1 On the Configure > URLs > Page Definitions page, click Add a Page Definition. The URLs > Page Definitions > Edit Page Definition page displays. 2 In the Page Definition section, click Select Page to locate the page to which this definition applies. A Resource List window appears, displaying all pages that have been monitored so far. 3 If desired, narrow the list to a more specific set of pages by entering a search string in the Search box, then clicking Go. You can use an asterisk (*) as a wild card to define a segment of the string that matches any combination of characters. Optionally, reduce the list further by opting to display only secure (HTTPS) or insecure (HTTP) web pages from the Protocol list, then clicking Go. Show URLs with encoding or decoding. 4 When you have located the page to which you would like the page definition to apply, click the page URL. You are returned to the Edit Page Definition page, and the newly added page URL appears in the Page Definition section. 5 In the Hits section, click Add a Hit. The same Resource List window appears. 6 Repeat step 3 through step 5 until you have added all the hits associated with the page. 7 Click OK to accept this page definition. You are returned to the URLs > Page Definitions page, where your newly created definition is listed. 8 Click Done to return to the main URLs page. Automatically discovering page definitions When creating a definition on the URLs > Page Definitions page, clicking Auto-Discover Pages initiates a process that identifies pages you want to include in the page definition. The appliance achieves this by monitoring pages accessed from your IP address. Using this option lets you build page definitions faster than manually adding pages one by one. To automatically discover page definitions: 1 Open a web browser and navigate to the desired page on your site that you want to include in the page definition. 2 On the Configure > URLs > Page Definitions page, click Auto-Discover Pages. 154
3 In the Session Identification area select one of the following options to tell the system how to discover your usage of the monitored application for auto-discovery: a b c Select the IP Address option and then type a valid IP address in the box. This option works only if your client IP address is not being translated by an HTTP proxy before it reaches the monitored application. Select the Session Identifier option. Select a Session Identifier from the box. In the Value box, enter the value for the selected session identifier that has been assigned to your session. Typically, a session identifier is a cookie. Determine what value has been assigned to your session cookie using a tool like the Mozilla Web Developer Toolkit. For more information, see Managing user session identification variables. Select the Login Variable option. Choose a Login Variable from the list. In the Login Name text box, type a login name then choose a Session Identification variable from the list to tell the system how to track your session after the login name you defined has been discovered. 4 Click Start to initiate a process whereby the appliance monitors the pages that you access in the first browser. 5 In the first browser, navigate through each of the pages in your service until it is completed. The list of pages you access are displayed in the Page Definitions > Auto-Discovery page in the web console. 6 To stop monitoring network traffic, click Stop. A list of server IPs discovered in the traffic appears. 155
All discovered pages are displayed in a list with the following information: The time in milliseconds for each URL based on the session identifier The type of data returned in the URL. For example HTML, Text or a graphic type The URL 7 In Discovered Servers, click Configure to add the IP to the list of monitored servers. Currently configured indicates that the IP address is already in the monitored servers list. 8 Select the check boxes for the page of the corresponding URL that you would like to add to the page definition. 9 Click Add. 10 After being returned to the URLs > Page Definitions page, click Done. Advanced URL options Several optional settings that affect URLs can be found by clicking Configure > URLs > Advanced Options. The Advanced Options page allows you to toggle various processing operations in the Preferences area. For more information, see these topics: Show ports in URLs option Show parameters in URLs option Show HTTP request methods in URLs option POSTs with no content-type should be handled as XML option Proxy tunneling 156
Do not strip the www prefix from URLs option Ignore 401 codes during NTLM authentication option Exclude redirections from metric calculations option Use HTML parser for page recognition option Extensions Response codes Show ports in URLs option The Show ports in URLs check box controls whether the port numbers that appear in URLs encountered by the appliance are included in the URLs stored in the database. For some sites, the port numbers can help to distinguish identical applications that are running on the same web sever. If you leave the check box clear (the default setting), the setting is disabled, and the system automatically strips the port numbers out of monitored URLs. Show parameters in URLs option The Show parameters in URLs check box controls whether the parameter section of URLs that are monitored by the appliance is included in the URLs stored in the database. For example, this URL: http://www.mysite.com/travel/mexico.asp;cat=beaches?city=cancun with parameters included, are stored in the database as: mysite.com/travel/mexico.asp;cat=beaches?city=cancun and with parameters excluded, are stored in the database as: mysite.com/travel/mexico.asp?city=cancun If you are monitoring web sites that use parameters in a widespread manner, you may want to clear this check box, so that these parameters do not keep appearing in the URLs recorded in the database. Show HTTP request methods in URLs option During the monitoring of traffic, the Show HTTP request methods in URL check box allows you to control the display of the HTTP request method (GET, POST) in the URLs recorded by the system. By default this option is enabled, but for upgrades this option is disabled. For example, the Resource List would contain the following URLs if this option is enabled: GET /somesite.com/page1 POST /somesite.com/page1 HEAD /somesite.com/page1 For a single web page, there could be several different URLs that now appear in the metric database. This may or may not be desired, depending on the application being monitored. For users who are monitoring SharePoint, it is recommended that this option should be enabled. NOTE: Exercise caution when modifying this setting as it will create new URLs in the metric database that are separate from the URLs that were collected before the change. 157
POSTs with no content-type should be handled as XML option Select the POSTs with no content-type should be handled as XML check box to monitor applications where XML content is transmitted in POST requests with a blank content-type HTTP header. The system can then parse the XML according to the Variable Rules you have constructed for XML variables. If your application always has correct content-type HTTP headers, then you would not need to enable this option. Proxy tunneling In some network environments, the appliance may be placed in a position where it is monitoring traffic going into a HTTP proxy server that supports dynamic switching to a tunneled protocol. One of the most common cases, is that of a SSL tunnel opened inside of a normal HTTP connection. The appliance supports monitoring this type of tunneling and creates special entries in the Hit resource list to track when these events occur. The following provides the general sequence of events for an SSL tunnel created through such a proxy: 1 The client sends HTTP CONNECT to proxy. The CONNECT specifies the final destination host and port (for example, CONNECT www.quest.com:443). 2 The proxy opens a TCP connection to the specified host and port. 3 The proxy sends HTTP OK back to client. 4 The client now switches to new protocol for remainder of connection (usually initiating a SSL handshake). 5 All subsequent SSL messages are forwarded to the end point host and port. NOTE: Once the client switches to SSL, the appliance begins monitoring and decrypting the SSL traffic, sent over a proxy, which is being sent over the HTTP connection if the administrator has configured a private key for the proxy server. For the appliance s SSL decryption to work inside of the tunnel, all the end-point hosts must be using this same private key. This HTTP CONNECT method is unique from all other HTTP methods in that it does not specify a resource with a full URL. To keep track of how often these tunneled connections are being started, the appliance creates an artificial URL to represent the above operation. This artificial URL has the format: /CONNECT_TO_host:port From the above example, this would translate to: /CONNECT_TO_www.quest.com:443 The above statement /CONNECT_TO_host:port URL appears in the appliance s Hit resource list for each unique tunneled host/port. NOTE: Any subsequent HTTP GET/POST requests that are sent inside of the SSL tunnel (and then decrypted by the appliance) appear normally in the appliance s Hit resource list. Do not strip the www prefix from URLs option The Do not strip the www prefix from URLs check box allows you to specify whether the appliance includes the www prefix in URLs that are recorded in the database and displayed in the user interface. By default, the www prefix does not appear in URLs displayed in the Site, Page, and Hit categories. By omitting the www prefix, the lists of URLs that appear in the Page and Hit categories are simplified. However, when your Foglight Experience Monitor is deployed alongside a Foglight Experience Viewer you may want to enable this option. Sometimes the Foglight Experience Viewer needs to retrieve GIFs, JPEGs, and other 158
page components directly from the site that is being monitored. Without the www prefix, Foglight Experience Viewer cannot download and display page components directly from the site in order to provide a realistic playback of a user session. Ignore 401 codes during NTLM authentication option The Ignore 401 codes during NTLM authentication check box allows you to specify whether or not the appliance includes 401 error codes in metrics such as Response Codes. By default this option is enabled. The NTLM protocol uses a challenge-response mechanism for authentication. It consists of three messages, commonly referred to as Type 1 (negotiation), Type 2 (challenge), and Type 3 (authentication). During the authentication process, 401 errors can be returned from the server. The client browser then performs an auto negotiate in the background and then establishes the session with the server. When 401 error messages are generated during this type of situation, they are collected by metric categories, such as the HTTP Fault Log, and may give a false impression that there are problems that requires addressing. Exclude redirections from metric calculations option The Exclude redirections from metric calculations check box allows you to specify whether or not hit retrievals that are based on redirections are excluded from metric calculations. Requests for hits can sometimes be redirected for a variety of reasons, each of which are represented by the HTTP 300-series response codes. Table 26. Response codes Response Code Definition 300 Multiple Choices 301 Moved Permanently 302 Found 303 See Other 304 Not Modified 305 Use Proxy 307 Temporarily Redirect As an example, web browsers can make conditional requests for a resource by the request headers they include. If a server responds with 304 Not Modified, the browser can then use its locally cached resource rather than downloading it again from the server. When the check box is selected, requests for hits that end up as redirections are excluded from the calculations for these metrics: Hit - Command Processing Time Hit - Command Response Time Hit - Hit End-to-End Time In general, it is recommended that you select this check box (the default setting) to exclude values incurred by redirections from any of these metrics. This ensures the metrics are not distorted by the inclusion of values from redirections or browser cache accessing, and instead they reflect activity that resulted from a full download from the site the appliance is monitoring. 159
Use HTML parser for page recognition option The Use HTML parser for page recognition check box allows you to enable an algorithm for page recognition that relies on parsing of HTML. The algorithm searches for the presence of the <HTML> tag to determine if a hit is actually a Page. This algorithm also searches for the <FRAMESET> tag and, if present, it does not mark the hit as a Page. If this check box is cleared, the system uses an algorithm for page recognition that utilizes a set of heuristics rather than HTML parsing. This technique requires fewer CPU resources, but is not as accurate as the HTML parsing algorithm. Extensions To view all extensions that are used by your site for different purposes, navigate to Configure > URLs > Advanced Options and click Extensions. The Advanced Options > Extensions page lists all hits, by their extension, that the appliance is configured to recognize. Figure 73. Advanced Options: Extensions page Content types are divided into categories and include: active extensions (such as PHPs and JSPs) static extensions (such as JARs and class files) image extensions (such as PNG tunes and GIF files) audio extensions (such as MIDI tunes and WAV files) video extensions (such as MPEG and RealMedia) document extensions (such as text files or PDFs) These lists assist the appliance in differentiating between a Page and a Hit. If you discover URLs that are not appearing in the correct category, you may need to add some items to these lists. The image, video, and static extension lists ensure that URLs with those extensions are treated as Hits. The document extension list ensures that URLs with those extensions are treated as Pages. URLs on the active extensions list will be treated as Pages unless a consistent pattern is observed where the URL is accessed after 160
another active URL and refers back to that same active URL. If this occurs, the URL is treated as a Hit of the previous URL. To add an additional extension, simply enter it in the appropriate category s text entry box, then click Add. When you have finished adding all new extensions, click Apply to return to the main URLs page. To remove an extension, select its corresponding check box, then click Delete. Response codes To identify HTTP response codes that should be excluded from the HTTP Fault Log, navigate to Configure > URLs > Advanced Options and click Response Codes. The Advanced Options > Response Codes page allows you to define HTTP response codes that should be ignored in the HTTP Fault log. Figure 74. Advanced Options: Response Codes page Type the code in the box and then click Add. Once configured, faults with the response codes you defined will no longer appear in the HTTP Fault log. 161
6 Managing applications The Foglight Experience Monitor s configuration options include those that allow you to customize how the appliance monitors and analyzes network traffic. This section explains how you can define and manage services, applications, application components, and SOAP transactions. For more information, see these topics: Integrating with CA SiteMinder Integrating with SafeNet Hardware Security Modules (HSMs) Managing application components Managing services definitions Monitoring Microsoft Office SharePoint Servers Monitoring PeopleSoft applications Monitoring Siebel applications Monitoring SOAP applications Configuring enterprise-wide service levels Integrating with CA SiteMinder CA SiteMinder is a centralized web access management system that enables: user authentication and single sign-on policy-based authorization identity federation auditing of access to web applications and portals. The SiteMinder Policy Server is a SiteMinder component that provides authorization and authentication services to applications. To monitor applications that utilize SiteMinder, you need to configure Foglight Experience Monitor so that it performs the same type of user authentication with CA SiteMinder that your applications currently perform with SiteMinder. Applications that utilize SiteMinder identify user sessions through the SMSESSION cookie (at some sites, a different cookie name may be used). If your application relies solely on this cookie to identify user sessions, you need to configure the appliance to work with your SiteMinder installation. Failure to setup the configuration means that your appliance cannot correctly identify user sessions resulting in inaccurate metrics. If your application has other means of identifying user sessions, such as using the JSESSION cookie variable, that are independent of SiteMinder, you should not configure the appliance to access SiteMinder. NOTE: Foglight Experience Monitor supports SiteMinder version 6.0 with Service Pack 5. For more information, see Communicating with SiteMinder. 162
Communicating with SiteMinder The FxM web console allows you to configure the appliance to communicate with your SiteMinder Policy Server. To communicate with the SiteMinder Policy Server: 1 On the appliance, navigate to the Configure > Applications > SiteMinder page. 2 Do one of the following: To define a new SiteMinder Policy Server, click Add a SiteMinder Server definition. To modify an existing SiteMinder Policy Server, click Edit beside that server name. NOTE: If your site employs more than one SiteMinder Policy Server repeat this configuration for each server. 3 On the SiteMinder > Edit page, configure the following settings. Table 27. SiteMinder settings Field IP Address Authorization Port Authentication Port Accounting Port Cookie Agent Name Shared Secret Probes Description IP address of the host on which the SiteMinder Policy Server is installed. The default authorization port number for the policy server is 44443. If the default port was changed during the SiteMinder installation, check the port setting in the Policy Server Management Console and enter that port number in this box. The default authentication port number for the policy server is 44442. If the default port was changed during the SiteMinder installation, check the port setting in the Policy Server Management Console and enter that port number in this box. The default accounting port number for the policy server is 44441. If the default port was changed during the SiteMinder installation, check the port setting in the Policy Server Management Console and enter that port number in this box. The name of the sign-on cookie, either SMSESSION or GMWSESSION that is created when signing into SiteMinder Policy Server. Using the Secure Cookies setting on the SiteMinder web agent tells the browser to send the SMSSESSION or GMWSESSION cookie to the web server. A name that is used when configuring the SiteMinder Policy Server. This can be any string such as dell_agent. A key used for initial connection to the SiteMinder Policy Server. The appliance is assigned a new dynamic shared secret if the SiteMinder Policy Server is configured to provide it with one. In a multiple-appliance cluster, a Probe list appears. To constrain the use of this definition to a specific probe, click the Selected probes option button and select the check box beside the probe s IP address. Otherwise, use the default All option button. 4 To verify the SiteMinder settings defined in Step 3, click Verify settings at the bottom of the SiteMinder Server section. A dialog box appears, displaying the status of the SiteMinder server. 163
5 To close the dialog box, click OK. 6 In the Sites section, click Pick a Site to select one or more sites for which this Policy Server provides authentication. Ensure that this list is complete, otherwise some user sessions may not be correctly identified by the appliance. If your site makes use of site aliases, ensure that those have all been previously configured (navigate to Configure > URLs page). If your site uses a single SiteMinder Policy Server for all user authentication, then leave the list of Sites empty. By default, the FxM authenticates any user session accessing a site that is not listed, against all the SiteMinder Policy Servers that have been configured, in a specific order, until the authentication is successful. 7 Click OK to save this definition. 8 Navigate to the Configure > Monitoring > User Sessions page. 9 In the Session Identification Variables section, add smsession to the list. The appliance extracts the user s sign-on session ID. 10 In the Login Variables section, add smsession to the list. The appliance extracts the user s login name from the smsession cookies. Creating an agent entry on the SiteMinder Policy server After configuring the appliance to access SiteMinder, you need to create and configure an Agent on the SiteMinder Policy server. To create an agent entry: 1 Open the SiteMinder Agent Dialog page. 2 In the Name field, type the name of the agent previously defined on the appliance. See Agent Name for more information. 3 Select the Support 4.x agents check box. 4 In the IP address field, type the IP address of the Foglight Experience Monitor. 5 In the Shared Secret area, type the secret that was previously defined on the appliance. See Shared Secret for more information. 164
6 Click OK. After you configure the Agent properties, your appliance can identify user sessions in the monitored traffic. The Login Name field for your user sessions should now contain the login name that users enter when they log in to the monitored application. Integrating with SafeNet Hardware Security Modules (HSMs) SafeNet Hardware Security Modules (HSMs) store and manage cryptographic keys, providing organizations with secure encryption, decryption, authentication, and digital signing services. When integrated with a SafeNet HSM, Foglight Experience Monitor can use the keys stored within the HSM server to decrypt HTTPS traffic. Foglight Experience Monitor uses the keys in a secure manner consistent with the SafeNet HSM model. To integrate Foglight Experience Monitor with SafeNet HSM servers and access the HSM private keys, you need to complete the following tasks: 1 Configuring Foglight Experience Monitor to use SafeNet HSM 2 Associating monitored SSL ports with HSM keys Before starting with the integration, see Before you begin. Before you begin For each of your HSM servers, you need the following information: HSM server name HSM server IP address Location of your HSM server certificate TIP: You can request the HSM server certificate from your HSM Administrator. If you are managing your own HSM appliance, see the SafeNet documentation for instructions on how to retrieve the server certificate. A list of your SSL ports with the HSM partition, password, and private key used For each Foglight Experience Monitor appliance, you need the appliance s IP address. 165
Configuring Foglight Experience Monitor to use SafeNet HSM To configure Foglight Experience Monitor to work with SafeNet HSM servers, you need to complete the following tasks: 1 Adding SafeNet HSM server definitions 2 Creating SafeNet HSM client certificates 3 Uploading HSM client certificates 4 Verifying HSM server definitions Adding SafeNet HSM server definitions You need to add an HSM server definition for each HSM server that manages private keys for the ports you want to monitor. To add an HSM server definition: 1 Navigate to Configure > Applications > SafeNet. 2 Click Add a SafeNet HSM Server definition. 3 In the SafeNet:Edit page, type the name and IP address of your HSM server. 4 Beside the HSM Server Certificate box, click Browse. Navigate to your server certificate and click Open. The HSM server certificate file name appears in the field. 5 Click OK. You return to the SafeNet page. The HSM server definition is displayed in the table. 6 Next step: Creating SafeNet HSM client certificates Creating SafeNet HSM client certificates The HSM appliance requires client certificates for all applications that need to interact with an HSM server. You need to generate an HSM client certificate for each Foglight Experience Monitor appliance that needs access to an HSM server. The client certificates are created using the IP address of the Foglight Experience Monitor appliance. To create an HSM client certificate: 1 In the SafeNet page, click Create Client Certificate. 166
The SafeNet Client Certificate dialog box appears. 2 Click Go. The SafeNet client certificate is created. 3 Right-click the link and save the client certificate file to a local drive. 4 Click Close. 5 Next step: Uploading HSM client certificates Uploading HSM client certificates Contact your HSM Administrator. Your HSM Administrator uploads the client certificates for Foglight Experience Monitor to the HSM appliance, registers it using the IP address of the Foglight Experience Monitor appliance (not the hostname), and assigns access to the partitions containing the required private keys. Next step: Verifying HSM server definitions Verifying HSM server definitions After you have added an HSM server definition, created the client certificate, and uploaded it to the HSM server, you can verify the definition. NOTE: If the verification test fails, try the following solutions: Ensure that you completed all the prerequisite procedures before verifying the server definition. If any of the procedures were missed, the test will fail. Ensure that your HSM Administrator registered the client certificate using the Foglight Experience Monitor appliance s IP address, not its hostname. Upload the current client certificate. If for any reason another client certificate was generated for an HSM server, the preexisting client certificate becomes invalid. To verify an HSM definition: 1 Obtain a confirmation from your HSM Administrator that the client certificate is successfully loaded to the HSM Server. 2 To verify that the definition is correct, click Verify Settings. A message box displays the results of the test. 167
3 Next steps: Associating monitored SSL ports with HSM keys Editing HSM server definitions To edit an HSM server definition: 1 Navigate to Configure > Applications > SafeNet. 2 In the row containing the HSM server definition you want to edit, click Edit. 3 In the SafeNet:Edit page, edit the fields as necessary. 4 Click OK. You return to the SafeNet page. The HSM server s details are updated in the table. Removing HSM server definitions To remove an HSM server definition: 1 Navigate to Configure > Applications > SafeNet. 2 In the row containing the HSM server definition you want to remove, select the check box. 3 Click Delete. The HSM server definition is removed from the table. The appliance can no longer access the keys controlled by this HSM server. Associating monitored SSL ports with HSM keys For each server handling encrypted traffic that can be decoded using a private key stored in your SafeNet HSM appliance, you need to associate each of the server s SSL ports with an HSM partition and an HSM key. You need the password for the HSM partition. To associate a monitored SSL port with an HSM key: 1 From the main menu, click Configure > Monitoring > Servers. 2 In the row containing the server that requires an HSM key, click Configure. The Configure Secure HTTP page appears. 3 Click Link HSM Key. The Link HSM Key wizard opens. 168
4 In the Select Port screen, select an SSL port. 5 In the Select Partition screen, select the HSM server partition and type the password. 6 In the Select Key screen, select the HSM private key to associate with the port. TIP: If the list is empty, you may have entered an incorrect partition password. Click Back and retype your password. 7 In the Confirm screen, review the details. 8 Click Finished. Managing application components The Application Components category provides metrics from the perspective of a web-based application or subset of an application. On the Application Components page, accessed by navigating to Configure > Applications > Application Components, you can define application components and instrumented application components that your system should monitor. 169
Figure 75. Application Components page The metrics of an application component represent an aggregation of the metrics for all of the URLs that match the regular expressions that are defined in the application component. By defining application components, you can monitor web-based applications as a single entity, so that performance metrics and SLA compliance can be evaluated from the perspective of the application as a whole. The metrics of an instrumented application component represent an aggregation of the metrics that are tracked by FxM using the browser-based instrumentation. An instrumented application component is defined as a subset of pages that have URLs matching one or more regular expression patterns. These pages have been instrumented to return metrics to the originating web site from the user s browser (for details, see Configuring browser-based instrumentation in the Foglight Experience Monitor ). For more information, see these topics: Defining an application component Editing an application component Deleting an Application Component Defining an instrumented application component Editing an instrumented application component Deleting an instrumented application component Defining an application component Configuring an application component involves defining a set of regular expressions, assigning service level thresholds, and selecting appropriate distribution configurations. To define an application component: 1 Click Add a new Application Component, located at the bottom of the Application Components section. This displays the Application Components > Edit page. 170
2 Name the Application Component Definition In the Name box, type the symbolic name you want to associate with this application component. Each application component must have a unique name. Use this name to locate the metrics recorded for it in the web console. 3 Set the Application-Related Service Levels a b In the End-to-End Time Service Level Threshold box, type the desired service level for the end-toend time associated with this application component. For example, if you decide that most of the page downloads for this application component should be serviced within five seconds, enter 5000 (units are in milliseconds). The End-to-End Time Service Level metric shows you the percentage of page downloads that satisfy this service level. For information on how this metric is calculated, see Page End-to-End Time in the Foglight Experience Monitor Metric Reference Guide. In the Processing Time Service Level Threshold box, type the desired service level for page processing times associated with this application component. For example, if you decide that the time to process the pages of this application component should generally not exceed two seconds, enter 2000 (units are in milliseconds). The Page Processing Time Service Level metric shows you the percentage of pages whose processing time satisfies this service level. For information on how this metric is calculated, see Page Processing Time in the Foglight Experience Monitor Metric Reference Guide. 4 Set the Application-Related Distributions Configurations a b In the End-to-End Time Distribution Configuration box, select the desired distribution configuration for the end-to-end time associated with this application component. This option allows you to specify the buckets and ranges of the distributions. In the Processing Time Distribution Configuration box, select the desired distribution configuration for page processing times associated with this application component. 5 Add Regular Expressions For existing application components, the Regular Expressions section lists all the regular expressions that have been defined. Since you are defining a new application component, this section is empty. a Type a regular expression in the box. For example, if you know where the application components are located (all components are found under quest.com/support/), enter the expression (in this case, quest.com/support/.*). Multiple regular expressions can be added. 171
b Click Add and proceed to Step f. c Alternatively, if you don t know where application components are located, click the Pick a Directory link to display a window that lists the directories in use on your site. This window allows you to easily select an existing directory to serve as the basis for a regular expression. By selecting an item in this window, the box is initialized with a regular expression that matches the directory you chose. This list is not obtained by an active search of your web site but rather from the list of URLs that have been monitored and recorded by the appliance in the past. d e From the Directory List, navigate to and select a directory. This directory is added to the Selected Directory section. Some directories contain subdirectories (directories that are nested beneath the top-level directory. For example, siebel.prod.quest.corp is a subdirectory of siebel. Click OK. f To display a list of URLs that match the regular expression and application component definition, click Test. NOTE: This test is performed against the list of URLs that have been monitored and recorded by the appliance in the past. 172
g Close the Application Test window when you are finished reviewing the resource list. Editing an application component You can modify an application component definition by clicking its corresponding Edit link, which takes you to the Application Components > Edit page. In the Application Component Definition section, you can edit the application component name, set service level thresholds and select distribution configurations. NOTE: If you modify an existing application component definition name, this change takes up to 30 minutes to propagate throughout the metrics that are displayed in the web console. You can also modify the regular expressions that identify the application component. Select the check box that corresponds to an expression, and then click Delete to remove it. You can then replace it by adding regular expressions in the manner outlined in the previous section. In the End-to-End Time Distribution Configuration and Processing Time Distribution Configuration lists, choose the appropriate distribution configuration for each of these metrics. See Distribution metrics. Deleting an Application Component To delete an application component from the list displayed in the Application Components page, select its corresponding check box and click Delete. Defining an instrumented application component Configuring an instrumented application component involves defining a set of regular expressions that match the URLs in use on your site. To define an instrumented application component: 1 Click Add a new Instrumented Application Component, located at the bottom of the Instrumented Application Components section. This displays the Instrumented Application Components > Edit page. 173
2 Name the Application Component Definition In the Name box, type the symbolic name you want to associate with this instrumented application component. Each instrumented application component must have a unique name. Use this name to locate the metrics recorded for it in the web console. 3 Add Regular Expressions For existing application components, the Regular Expressions section lists all the regular expressions that have been defined. Since you are defining a new application component, this section is empty. a b c Type a regular expression in the box. For example, if you know where the application components are located (all components are found under quest.com/support/), enter the expression (in this case, quest.com/support/.*). Multiple regular expressions can be added. Click Add. The regular expression appears in the Regular Expressions section. To display a list of URLs that match the regular expression and application component definition, click Test. NOTE: This test is performed against the list of URLs that have been monitored and recorded by the appliance in the past. d Close the Application Test window when you are finished reviewing the resource list. Editing an instrumented application component You can modify an instrumented application component definition by clicking its corresponding Edit link, which takes you to the Instrumented Application Components > Edit page. In the Application Component Definition section, you can edit the application component name. 174
NOTE: If you modify the name of an existing instrumented application component, this change takes up to 30 minutes to propagate throughout the metrics that are displayed in the web console. You can also modify the regular expressions that identify the instrumented application component. Select the check box that corresponds to an expression, and click Delete to remove it. You can then replace it by adding regular expressions in the manner outlined in the previous section. Deleting an instrumented application component To delete an instrumented application component from the list displayed in the Application Components page, select its corresponding check box and click Delete. Managing services definitions A service represents a sequence of pages that a user must traverse to complete a specific process (for example, checking out and completing a purchase on an e-commerce site). You can define a service by identifying a sequence of pages, then giving the sequence a name, which is used to identify the service in the web console. Figure 76. Services page For more information, see these topics: Defining a service Configuring services auto-discovery Configuring synthetic transactions Managing existing service definitions Using alternative methods to define a service Defining a service Configuring a service involves defining the sequence of steps, assigning service level thresholds, and choosing appropriate distribution configurations. To define a service: 1 To begin defining a service, on the Services page navigate to Configure > Applications > Services and then click Add a new Service. This displays the Services > Edit page. 175
Naming the service 2 In the Name box (in the Service Definition section), type the symbolic name you want to associate with this service. Each service must have a unique name, and users can use this name to locate the metrics recorded for it in the web console. Setting service-related service levels 3 In the End-to-End Time Service Level Threshold box, type the service level objective for the total time taken for all pages to be delivered to the user. For example, if you decide that most of the services should be fulfilled within 15 seconds, enter 15000 (units are in milliseconds). The Service End-to-End Time Service Level metric then shows you the percentage of services that satisfy this service level. For information on how this metric is calculated, see Service End-to-End Time in the Foglight Experience Monitor Metric Reference Guide. 4 In the Processing Time Service Level Threshold box, type the service level objective for the time taken by the web server farm to process all of the page requests in the service. For example, if you decide that the time to process the pages in this service should generally not exceed six seconds, enter 6000 (units are in milliseconds). The Service Processing Time Service Level metric shows you the percentage of transactions pages whose processing time satisfies this service level. For information on how this metric is calculated, see Service Processing Time in the Foglight Experience Monitor Metric Reference Guide. 5 In the Timeout box, type the period of time that is used as an expiration timer for users of the service. If any user exceeds the time specified they are considered to be timed out and their activity cannot influence the metrics collected for the service. The Service Timeout metric reflects the number of times the service has timed out based on the timeout setting that you specified. 6 In the End-to-End Time Distribution Configuration box, select the distribution configuration that you would like to use for this service. Distribution configurations allow you to specify the buckets and ranges of the distribution. By default, the Service End-to-End Time Distribution is selected when you create a new service. If you have defined additional distribution configurations for this metric they appear in this list and can be chosen for this service. For more information about how this metric is calculated, see End-to-End Time Service Level in the Foglight Experience Monitor Metric Reference Guide. 176
7 In the Processing Time Distribution Configuration box, select the distribution configuration that you would like to use for this service. Distribution configurations allow you to specify the buckets and ranges of the distribution. By default, the Service Processing Time Distribution is selected when you create a new service. If you have defined additional distribution configurations for this metric, they appear in this list and can be chosen for this service. For more information about distributions see Distribution metrics. 8 Select the Non-sequential page traversal included box if users do not have access to the pages in the listed sequence. This option needs to be selected in order for that action to be considered an instance of a service. By default, this check box is cleared, but in some cases you may want to allow non-sequential traversal. For example, if users are interacting with a web application that uses HTML frames, you may define a service that includes the URLs for all of the frames that appear as a single page to the user. In this scenario, the frames are accessed simultaneously so the order in which they are accessed by the end user is not significant. 9 Click OK. Adding a service step You can add a service step to the service by clicking Add a Step. Doing so requires navigating to the Services > Service > Edit page, which allows you to provide a symbolic label for the service step, define its type, and list the pages and regular expressions that determine which URLs qualifies as instances of this service step. To add a service step: 1 In the Name box, type the symbolic name you want to associate with this service step. Every service step must have a unique name among all of the services that you have defined. This is required in order to be able to uniquely identify service steps in the web console. It is recommended that you use a naming convention for service steps that includes the name of the service. For example, if your service is named Checkout you could name your service steps Checkout - Step 1. 2 From the Type list, select one of the following choices: Required: The user must traverse this step in order to complete an instance of the service. Optional: The user does not have to traverse this step in order to complete an instance of this service. If they do traverse this step then the metrics for this step are included in the overall metrics for the service. Note that the last step of a service cannot be optional. Abort: This step indicates the end of the service. When a user hits this step the service is considered to have been stopped prematurely. The abort count is incremented. All other service metrics are not updated. 3 In the Service Step Criteria box, type a regular expression and then click Add. Each service step can be defined by a single URL, regular expression or a list of URLs and regular expressions. To define URLs in the service step, see Configuring services auto-discovery. 4 Click OK. 177
Defining pages in a service step Next a service step can be defined to reference one or more pages (or URLs). To define pages in a service step: 1 From Services > Service > Edit page, click Add a Page. A Resource List window appears, displaying all pages that the system has seen up to this point in time. 2 In the Protocol list, select All to display all pages, HTTP (insecure) or HTTPS (secure) pages displayed. Service steps always match the defined URLs whether they are accessed using HTTP or HTTPS. IMPORTANT: There is no way to define a service step that is restricted by HTTP or HTTPS. 3 Narrow the list to a more specific set of pages by entering a search string in the Search box, then clicking Go. You can use an asterisk (*) as a wild card to define a segment of the string that matches any combination of characters. Optionally, reduce the list further by opting to display only secure (HTTPS) or insecure (HTTP) web pages from the Protocol list, then clicking Go. Show URLs with encoding or decoding. 4 When you have located the page you would like to add to the service definition, click the URL. When a page has been selected, the Resource List window closes, and the selected page appears in the Service Step Criteria list back on the Services > Service > Edit page. 178
5 Continue to add pages to the list until the service step has been completely defined. 6 Click OK. Defining the order of service steps From the Service > Edit page, if required, resort the list of pages into the desired order by using the buttons. Click OK to save this service definition. and Configuring services auto-discovery Services auto-discovery displays a list of URLs accessed by a user while logged into a user session. The URLs are used to create a service definition with each URL being considered a step in the service. Often environments are controlled using Network Address Translation (NAT) which involves removing the client IP address from the IP headers by an HTTP proxy and making it difficult to identify the user session by IP. If your environment is controlled by NAT you can use either the Session Identifier or the Login Name option so that the appliance can determine which URLs are specific to the user session. Before you can use Auto-Discover Service you must configure user sessions and define service steps. For more information, see Configuring user session identification and Defining a service. To configure Auto-Discover service: 1 From the Service page click Auto-Discover Service. 2 In the Session Identification area configure select one of the following options to tell the system how to discover your usage of the monitored application for auto-discovery: Select the IP Address option and type a valid IP address in the box. This option works only if your client IP address is not being translated by an HTTP proxy before it reaches the monitored application. Select the Session Identifier option. Select a Session Identifier from the list. In the Value box, type the value for the selected session identifier that has been assigned to your session (typically, a session identifier is a cookie). Determine what value has been assigned to your session cookie using a tool like the Mozilla Web Developer Toolkit. For more information, see Managing user session identification variables. Select the Login Variable option. Enter a login name then choose a Session Identification variable from the list to tell the system how to track your session after the login name you defined has been discovered. 3 Click Start. 179
The appliance begins to detect URLs based on the session identification option that you configured. The Services Auto-Discovery page updates every few seconds, adding any newly detected session identification to the list. 4 Click Stop at any point to cancel the search, and use the list that has been compiled up to that point. All session identifications discovered up to that point are displayed in a list with accompanying check box. The following information for each is displayed: the time in milliseconds for each URL based on the session identifier the type of data returned in the URL (for example HTML, Text or a graphic type) the URL details link for each session identification 5 Click the details link for any entry in the list. The Hit Details window opens displaying the following information. NOTE: The Services Auto-Discovery window remains open until it is manually closed. 180
Table 28. Hit Details information Data Hit Details Cookies Request Header Form Variables Description Full URL The URL address as it was captured by the appliance from monitored traffic. Transformed URL Displayed the URL with transformation rules applied. Any cookies found in the request. Any HTTP request headers found in the request. Any form variables. 6 Select the check boxes that correspond to any URL that you would like to use as a service step in a service. 7 Click Add. The URLs is added to the Service Steps section on the Services > Edit page. Configuring synthetic transactions The Foglight Transaction Recorder is a Foglight component that provides the ability to monitor end-user response time for web applications, whether they are packaged applications such as PeopleSoft, Siebel ebusiness Applications, Oracle E-Business Suite, or custom applications. The Foglight Transaction Recorder uses simulated business transactions to test performance, and notifies administrators of performance degradation before end users are affected. Foglight Transaction Recorder users can interactively record scripts through an embedded web browser that simulates the activity generated by an actual user of your web site. It captures and records all mouse movement, link or button clicks, and any data entry that may occur during the session. If you are using Foglight for Synthetic Transactions against your monitored applications, the transactions appear in the Synthetic Transactions category of metrics on the appliance. You can configure a Service that matches the Synthetic Transaction using the Synthetic Transactions section of the Configure > Applications > Services page. Viewing synthetic transaction scripts In the Foglight Experience Monitor web console, click Configure > Applications > Services. This displays the Services page which includes the Synthetic Transactions section. Figure 77. Services page Any synthetic transactions scripts monitored by the appliance are listed. When viewing detected Synthetic Transactions scripts, whether a script is currently associated with an existing Foglight Experience Monitor service, clicking Create or Edit takes you to the Services > Edit page. 181
Creating and editing synthetic transactions scripts When viewing detected Synthetic Transactions scripts, if a a script is currently associated with an existing Foglight Experience Monitor service, clicking create or edit takes you to the Services > Edit page. Figure 78. Services: Edit page To create and edit Synthetic Transaction Scripts: 1 For new service definitions, provide the service with a symbolic name in the Name box. This name references the service definition throughout the Foglight Experience Monitor console. 2 If required, continue to edit the service definition. (See Managing services definitions for more information.) 3 Click OK. You are returned to the Services page. Managing existing service definitions You can managing service definitions by editing or removing them. Removing a service When viewing the list of defined services on the Services page, you can delete an entry by selecting its corresponding check box, then clicking Delete. Use the Check All option to select all entries for deletion. The Clear All option is used to clear any entries that were selected for deletion. Editing a service You can also modify a service definition by clicking its corresponding Edit link. Doing so takes you do the Services > Edit page, which allows you to rename the service set service level thresholds and choose distribution configuration. NOTE: If you modify an existing service definition name, this change takes up to 30 minutes to propagate throughout the metics that are displayed in the web console. 182
You can also modify the list of steps that comprise the service. Remove steps by clicking their corresponding check boxes, then clicking Delete. Reorder steps by using the and buttons. When you have finished making changes to the service, click OK. Editing a service step You can modify a service step definition by clicking its corresponding Edit link. Doing so takes you do the Services > Edit page, which allows you to modify the service step definition. Using alternative methods to define a service Defining a service by selecting the steps manually is just one method that you can use. This section describes alternative approaches. Using user sessions From the User Sessions Log you can drill-down into the All Metrics View for a user session. At the bottom of that page, the link Create a Service based on this User Session appears. By clicking that link, you can begin the process of defining a service that matches all of the pages traversed during the user session. Clearly, this option is only useful if you are controlling the pages accessed by that user so that it matches the service you would like to define. Using paths The Path category tracks commonly accessed sequences of pages. By navigating to Analysis > Metric Analysis and selecting Path > Path Count you can generate a report that displays the most commonly traversed paths in the application you are monitoring. By clicking the link for one of the paths you will be taken to a Metric View for that path. At the bottom of that page, the link Create a Service based on this Path appears. By clicking that link you can begin the process of defining a service that matches all of the pages recorded in this path. Monitoring Microsoft Office SharePoint Servers Microsoft Office SharePoint Server is used to facilitate a collaborative environment for SharePoint sites. You can enable the appliance to monitor requests sent to and responses received from the server and report statistics about these user activities. 183
Figure 79. SharePoint page Configuring the SharePoint options controls which URLs are treated as SharePoint requests. Hits with URLs that match any regular expressions (configured on the SharePoint page) are processed by the appliance to reconstruct the user's actions based on the sequence of SharePoint URLs and then determines the performance of these user actions. These SharePoint user actions display in the Page category. For more information, see these topics: Configuring SharePoint monitoring Configuring how URLs display SharePoint and SOAP Configuring SharePoint monitoring Configuring SharePoint monitoring involves defining a set of regular expressions and installing a set of predefined configuration settings. To configure SharePoint monitoring: 1 Navigate to the Configuring > Applications > SharePoint. Defining a regular expression 2 Type a Perl regular expression in the box. For example, if you know where the SharePoint applications are located (all components are found under quest.com/mysharepoint/support/), type the expression (in this case, quest.com/mysharepoint/support/.*). Multiple regular expressions can be added. 3 Click Add and proceed to Testing regular expressions validity. Alternatively, if you don t know where application components are located, click the Pick a Directory link to display a window that lists the directories in use on your site. 184
This window allows you to easily select an existing directory to serve as the basis for a regular expression. By selecting an item in this window, the box will be initialized with a regular expression that matches the directory you chose. This list is not obtained by an active search of your web site but rather from the list of URLs that have been monitored and recorded by the appliance in the past. 4 From the Directory List, navigate to and select a directory. This directory is added to the Selected Directory section. Some directories contain subdirectories (directories that are nested beneath the top-level directory. For example, my.prod.quest.corp is a subdirectory of my. 5 Click OK. The directory is added to the regular expressions list. Testing regular expressions validity 6 To display a list of URLs that match the regular expression, click Test. This test is performed against the list of URLs that have been monitored and recorded by the appliance in the past. 7 Close the window when you are finished reviewing the resource list. Configuring FxV sessionizing 8 Optionally, select the Enable FxV sessionizing by IP address to override the default behavior of not sessionizing hits for Foglight Experience Viewer that do not contain any session ids. Enabling this option means that user sessions will appear in Foglight Experience Viewer based on the client IP address for each hit. Only enable the Enable FxV sessionizing by IP address check box when monitoring intranet SharePoint sites where there are no cookies available to serve as a session identifier. If you are not sure if cookies 185
are available, navigate to the Session Identification Variable list in the User Sessions page (Configure > Monitoring > User Sessions) and run the auto-discover variables option. When this option is enabled, the FxM agent transmits user sessions to the Foglight Experience Viewer appliance based on the client s IP address that appears in the traffic. If you are monitoring an internet application, your data may be incorrect since proxy servers will sometimes combine the traffic for multiple user sessions into the same connection using a single client IP address. If you are monitoring intranet applications, your data may also be incorrect if your network utilizes TCP connection multiplexing. When your network configuration is setup so that the IP address cannot appropriately be used to track user sessions, this can result in the Foglight Experience Viewer combining the pages of multiple sessions into a single session. If this problem occurs you must disable this setting to prevent sessions from being exported to the Foglight Experience Viewer. Configuring SharePoint configuration settings 9 In the SharePoint Configuration Settings section, click the Apply button to install pre-defined configuration settings. The appliance monitors servers previously configured on the Servers page using these settings. Configuring how URLs display When SharePoint monitoring is configured, the appliance records when information is transmitted to and from the server. For example, when a user publishes content to a SharePoint server, in a form (POST parameter), the appliance monitors the URLs that contain this activity. You can define how the URLs display in the web console using several options. For example, you can display request methods (POST, GET) in URLs by setting the Show HTTP request methods in URL option using the Advanced Options on the URL page. For more information, see Advanced URL options. NOTE: It is recommended that this option be enabled when monitoring SharePoint. By default, SharePoint encodes its URLs in a manner that makes their URLs difficult to read. In the web console, you can decide whether to display URLs as they are originally encoded or decoded in a manner that makes them more readable. For information about displaying URLs in the web console, see Encoded and decoded URLs. When SharePoint monitoring is first enabled, the appliance automatically populates a list of default variable rules. You can customize a variable rule defining how the system will transform the query segments of URLs. For more information, see Managing variable rules. When a document is edited on a SharePoint site, its name is stored on the server and by default, the appliance displays this document name in the URL displayed in the web console. If you do not want to display this information in URLs, you can configure whether these document names are displayed by changing the variable rules. For more information, see Managing variable rules. SharePoint and SOAP If you are monitoring SOAP based web services you can view SOAP Operations for SharePoint, on the All Metrics View > User Sessions. For many of the user actions performed in SharePoint, a SOAP operation call is made by the browser. Examples of user actions that result in SOAP operations are document checkout and check in. These SOAP operations are often intermixed with non-soap requests. The SharePoint analysis module within the appliance will combine these SOAP and non-soap requests to form a page. The metrics for this page display in the Page category. The display of SOAP operations names is handled differently from non-soap names. Regular hits display as a standard URL, and might include additional form variables appended to the URL. For example: GET site.com/path1/index.html;param1=a?query1=b&query2=c [ form1=d&form2=e] A SOAP operation however will show the soap action and request method. SOAP example: POST site.com/path1/soapaction RequestMethod 186
Additionally, there are several more options for controlling how the SOAP operation name is constructed, such as including the adaptor in SOAP Operations, including server display names in SOAP Web Service and SOAP Operation names, and including the port in SOAP Web Services, SOAP Operation and SOAP Server names. To configure these options navigate to Configure > Applications > SOAP page. Monitoring PeopleSoft applications PeopleSoft s applications help to improve and maintain critical relationships with customers and employees. By enabling PeopleSoft monitoring, the appliance monitors the web pages (HTTP request and responses) being passed between the client and the server. Any critical business processes delivered by PeopleSoft applications are monitored. Metrics and data from these processes are displayed in the web console. Figure 80. PeopleSoft page Configuring the Peoplesoft options controls which URLs are treated as Peoplesoft requests. Hits with URLs that match any regular expressions (configured on the Peoplesoft page) are processed by the appliance to reconstruct the user's actions based on the sequence of Peoplesoft URLs and then determine the performance of these user actions. These Peoplesoft user actions display in the Page category. For more information, see these topics: Monitoring PeopleSoft applications Configuring how URLs display Monitoring PeopleSoft applications Configuring PeopleSoft monitoring involves defining a set of regular expressions and installing a set of predefined configuration settings. 187
To configure PeopleSoft monitoring: 1 Navigate to the Configuring > Applications > PeopleSoft. Adding regular expressions 2 Enter a Perl regular expression in the box. For example, if you know where the PeopleSoft applications are located (all components are found under quest.com/mypeoplesoft/support/), enter the expression (in this case, quest.com/mypeoplesoft/support/.*). Multiple regular expressions can be added. 3 Click Add and proceed to Testing regular expressions validity. Alternatively, if you don t know where application components are located, click the Pick a Directory link to display a window that lists the directories in use on your site. This window allows you to easily select an existing directory to serve as the basis for a regular expression. By selecting an item in this window, the box will be initialized with a regular expression that matches the directory you chose. This list is not obtained by an active search of your web site, but rather from the list of URLs that have been monitored and recorded by the appliance in the past. 4 From the Directory List, navigate to and select a directory. This directory is added to the Selected Directory section (some directories contain subdirectories, directories that are nested beneath the top-level directory. For example, my.prod.quest.corp is a subdirectory of my). 5 Click OK. The directory is added to the regular expressions list. Testing regular expressions validity 6 To display a list of URLs that match the regular expression, click Test. This test is performed against the list of URLs that have been monitored and recorded by the appliance in the past. 7 Close the window when you are finished reviewing the resource list. Configuring PeopleSoft configuration settings 8 In the PeopleSoft Configuration Settings section, click Apply to install pre-defined configuration settings. The appliance will monitor servers previously configured on the Servers page use these settings. 188
Configuring how URLs display When PeopleSoft monitoring is configured, the appliance records when information is transmitted to and from the server. You can define how the URLs display in the appliance using several options. For example, you can display request methods (POST, GET) in URLs by setting the Show HTTP request methods in URL option using the Advanced Options on the URL page. For more information, see Advanced URL options. PeopleSoft as well as other applications encode their URLs. For ease of use, you can decide whether to display URLs encoded or decoded. For information about displaying URLs in the web console, see Encoded and decoded URLs. Figure 81. Variable Rules page. When you apply the pre-defined configuration settings, the system automatically populates a list of default variable rules. You can customize a variable rule defining how the system will transform the query segments of URLs. For more information, see Managing variable rules. Monitoring Siebel applications Siebel s customer relationship management application enables organizations to automate particular aspects of a business streamlining the processes which affect the overall bottom and top line of the business. You can enable the appliance to monitor activities performed by the Siebel application server and report statistics about these user activities. 189
Figure 82. Siebel page Configuring the Siebel options controls which URLs are treated as Siebel requests. Hits with URLs that match any regular expressions (configured on the Siebel page) are processed by the appliance to reconstruct the user's actions based on the sequence of Siebel URLs and then determines the performance of these user actions. These Siebel user actions display in the Page category. For more information, see these topics: Monitoring Siebel applications Configuring how URLs display Monitoring Siebel applications Configuring Siebel monitoring involves defining a set of regular expressions and installing a set of pre-defined configuration settings. To configure Siebel monitoring: 1 Navigate to the Siebel page, Configuring > Applications > Siebel. Adding regular expressions 2 Type a Perl regular expression in the box. For example, if you know where the Siebel applications are located (all components are found under quest.com/mysiebel/support/), enter the expression (in this case, quest.com/mysiebel/support/.*). Multiple regular expressions can be added. 3 Click Add and proceed to Testing regular expressions validity. Alternatively, if you don t know where application components are located, click the Pick a Directory link to display a window that lists the directories in use on your site. This window allows you to easily select an existing directory to serve as the basis for a regular expression. By selecting an item in this window, the box is initialized with a regular expression that matches the directory you choose. This list is not obtained by an active search of your web site but rather from the list of URLs that have been monitored and recorded by the appliance in the past. 190
4 From the Directory List, navigate to and select a directory. This directory is added to the Selected Directory section. Some directories contain subdirectories (directories that are nested beneath the top-level directory. For example, my.prod.quest.corp is a subdirectory of my. 5 Click OK. The directory is added to the regular expressions list. Testing regular expressions validity 6 To display a list of URLs that match the regular expression, click Test. This test is performed against the list of URLs that have been monitored and recorded by the appliance in the past. The regular expression is implemented in Perl. 7 Close the window when you are finished reviewing the resource list. Configuring PeopleSoft configuration settings 8 In the PeopleSoft Configuration Settings section, click Apply to install pre-defined configuration settings. The appliance monitor servers previously configured on the Servers page using these settings. Configuring how URLs display When Siebel monitoring is configured, the appliance records when information is transmitted to and from the server. For example, when a user publishes content to a Siebel server, in a form (POST) parameter, the appliance monitors the URLs that contain this activity. You can define how the URLs display in the appliance using several options. For example, you can display request methods (POST, GET) in URLs by setting the Show HTTP request methods in URLs option using the Advanced Options page. For ease of use, you can decide whether to display URLs encoded or decoded. For information about displaying URLs in the web console, see Encoded and decoded URLs. 191
Figure 83. Variable Rules page When you apply the pre-defined configuration settings, the system automatically populates a list of default variable rules. You can customize a variable rule defining how the system will transform the variables that appear in URLs. For more information, see Managing variable rules. When you initiate Siebel monitoring the following session identification variable _sn is automatically created. Because Siebel is an internal application, ensure that the Sessionizing by TCP Connection is set because it can track a user session across changes in the session identification variable that can sometimes occur during a TCP connection. Monitoring SOAP applications The SOAP page enables you to configure how the appliance monitors applications that use the SOAP over HTTP. You can configure SOAP transactions to generate service level metrics to assist in the management of SLAs for your SOAP applications by navigating to Configure > Applications > SOAP. Figure 84. SOAP page You can learn more about SOAP Transactions in the Foglight Experience Monitor User Guide. Chapter 5 provides an overview of web services and the SOAP specification, and the breakdown of metric categories. For more information, see web services category in the Foglight Experience Monitor Metric Reference Guide. 192
IMPORTANT: The appliance currently supports SOAP-based web services that use HTTP or HTTPS as the transport protocol. Additionally, only the request-response messaging pattern is supported. Including the SOAP adapter in operation names By default SOAP operations, when displayed in the user interface include only the names of the operation and not the adapter. The adapter is the URL that is the endpoint for request to the SOAP operation. Select the Include adapter in SOAP operation names option so that the adapter is prefixed in the name for the SOAP operation. Including server display names in SOAP web service and SOAP operation names By default, the labels for SOAP web services and SOAP operations that appear in the web console do not include their associated server IP. If you would like to see the server IP included in these names, select the Include server display name in SOAP Web Service and SOAP Operation names. When this option is enabled, you can examine how particular SOAP services and SOAP operations perform with specific servers. Including port information in SOAP web service, SOAP operation, and SOAP server names By default, the labels for SOAP web services, SOAP operations, and SOAP servers that appear in the web console do not include the associated TCP port on which the activity occurs. If you would like to see the TCP port included in these names, select the Include port in SOAP Web Service, SOAP Operation and SOAP Server names. When this option is enabled, you can examine how particular SOAP web services, SOAP operations and SOAP Servers perform with specific ports. For more information, see these topics: Defining a SOAP application Modifying SOAP application definitions Defining a SOAP transaction Managing existing SOAP transaction definitions Mapping SOAP operations to a web service Adding and removing SOAP tags Defining a SOAP application Configuring a SOAP application involves defining a set of regular expressions, assigning service level thresholds, and choosing appropriate distribution configurations. To define a SOAP application: 1 Navigate to the SOAP page, Configure > Applications > SOAP. 2 In the SOAP Application section, click Add a SOAP Application. This displays the SOAP > SOAP Application > Edit page. 193
Naming the SOAP application 3 In the Name box, enter a unique, symbolic name you wish to associate with this SOAP application. Each SOAP application component must have a unique name, and users will use this name to locate the metrics recorded for this in the web console. Defining a service level threshold for SOAP operations 4 In the End-to-End Time Service Level Threshold box, type the total time taken for SOAP methods (the service level objective) that are included in this application to complete. For example, if you decide that most of the requests should be fulfilled within 15 seconds, enter 15000 (units are in milliseconds). The End-to-End Time Service Level shows you the percentage of requests downloads that satisfy this service level. 5 In the Processing Time Service Level Threshold box, type the desired service level for the total processing times for requests in the SOAP application. For example, if you decide that the processing time for most requests should occur in under one second, enter 1000 (units are in milliseconds). The Processing Time Service Level metric shows you the percentage of requests whose processing time satisfies this service level. 6 In the End-to-End Time Distribution Configuration box, select the distribution configuration that you would like to use for this application. Distribution configurations allow you to specify the buckets and ranges of the distribution. For more information about how this metric is calculated, see End-to-End Time Distribution in the Foglight Experience Monitor Metric Reference Guide. 7 In the Processing Time Distribution Configuration box, select the distribution configuration that you would like to use with this SOAP application. For information on how this metric is calculated, see Processing Time Distribution in the Foglight Experience Monitor Metric Reference Guide. Adding regular expressions For existing SOAP applications, the Regular Expressions section lists all the regular expressions that have been defined. Since you are defining a new application component, this section is empty. 1 In the Regular Expression box, type the regular expression that is matched against incoming SOAP Operations. If you know where the SOAP applications are located (for example, all components are found 194
under quest.com/support/), the expression can be entered directly in the box (in this case, quest.com/support/.*). 2 Click Add to add it to the list. 3 To display a list of SOAP Operations that match the regular expression and application component definition, click Test. NOTE: This test is performed against the list of SOAP Operations that have been monitored and recorded by the appliance in the past. The regular expression is implemented in Perl. 4 Close the Application Test window when you are finished reviewing the resource list. Modifying SOAP application definitions You can modify a SOAP application definition by clicking its corresponding Edit link, which takes you to the SOAP > SOAP Application > Edit page. In the SOAP Application section, you can edit the application name, set service level thresholds and choose distribution configurations. NOTE: If you modify an existing application definition name, this change takes up to 30 minutes to propagate throughout the metrics that are displayed in the web console. You can also modify the regular expressions that identify the application. Select an expression s corresponding check box, and click Delete to remove it. You can then replace it by adding regular expressions in the manner outlined in the previous section. Deleting SOAP application definitions When viewing the list of application components on the SOAP page, you can delete an entry by selecting its corresponding check box and then clicking Delete. Defining a SOAP transaction You can create a new SOAP transaction on the SOAP > Edit page. To define a SOAP transaction: 1 From the SOAP page, in the SOAP Transactions section, select Add a SOAP Transaction. 195
Naming the SOAP transaction 2 In the Name box, type a unique, symbolic name you wish to associate with this SOAP transaction. Each SOAP transaction must have a unique name, and users will use this name to locate the metrics recorded for this transaction in the web console. Defining a service level threshold for SOAP transaction 3 In the End-to-End Time Service Level Threshold box, type the service level objective for the total time taken for all SOAP requests identified in this transaction. For example, if you decide that most of the services should be fulfilled within 15 seconds, enter 15000 (units are in milliseconds). 4 In the Processing Time Service Level Threshold box, type the desired service level for the total processing times for all steps in the SOAP Transaction. For example, if you decide that the processing time for most transactions should occur in under one second, enter 1000 (units are in milliseconds). 5 In the End-to-End Time Distribution Configuration box, select the distribution configuration that you would like to use for this service. Distribution configurations allow you to specify the buckets and ranges of the distribution. For more information about how this metric is calculated, see End-to-End Time Distribution in the Foglight Experience Monitor Metric Reference Guide. 6 In the Processing Time Distribution Configuration box, select the distribution configuration that you would like to use with this SOAP transaction. For information on how this metric is calculated, see Processing Time Distribution in the Foglight Experience Monitor Metric Reference Guide. Adding SOAP operations to the transaction 7 Click Add a SOAP Operation to begin building a sequence. A Resource List window is displayed, listing all SOAP operations that the appliance has detected (as shown below). 8 If desired, narrow the list to a more specific set of pages by typing a search string in the Search box, and then clicking Go. You can use an asterisk (*) as a wild card to define a segment of the string that matches any combination of characters. 196
Optionally, reduce the list further by opting to display only HTTPS secure or HTTP insecure web pages from the Protocol list, then clicking Go. Show URLs with encoding or decoding. 9 Click any page to add it to the transactions sequence, after which the window closes. 10 Repeat this procedure until all SOAP operations that form the transaction have been added to the definition. NOTE: Every defined SOAP transaction definition must have a unique sequence of SOAP operations. 11 After all operations have been added, if required, the sequence of SOAP operations can be rearranged by using the and buttons. 12 Click OK to complete the process. The appliance begins monitoring for activity immediately. Managing existing SOAP transaction definitions When viewing the list of defined SOAP transactions on the SOAP page, you can delete definitions by selecting their corresponding check box, and then clicking Delete. Check All is used to select all entries for deletion, and Clear All is used to clear any entries that were selected for deletion. Modify a SOAP transaction definition by clicking its corresponding Edit link. Doing so takes you to the SOAP Transactions > Edit page, which allows you to rename the SOAP transaction, add or remove SOAP operations, or reorder the operations by using the and buttons. Mapping SOAP operations to a web service The system attempts to correlate SOAP Operations with a corresponding SOAP Web Service when monitoring SOAP traffic. While SOAP v.1.1 s SOAPAction HTTP header tag can provide this information, web services that follow the SOAP v.1.2 standard may not be able to rely on it, since its use is no longer mandatory. As an alternative, the appliance can use a WSDL (Web Services Definition Language) file to establish these relationships. A WSDL file provides the mapping of a SOAP Operation to a SOAP Web Service in FxM. The SOAP Web Service appears in the name of the SOAP Operation in the Resource List. For example: AlphaStockQuoteService getstockquote where AlphaStockQuoteService is the SOAP Web Service name and getstockquote is the SOAP Operation name. If you do not have a WSDL file loaded, this SOAP Operation is displayed as: UNKNOWN_SOAP_SERVICE getstockquote Therefore, by loading the WSDL file that contains the description of the AlphaStockQuoteService you instruct the system to replace UNKNOWN_SOAP_SERVICE with AlphaStockQuoteService. You need to configure Servers and Ports for the WSDL file only if you have two SOAP Web Services using the same SOAP Operation name (in other words, only if the SOAP Operation name appears in more than one WSDL files). For example: AlphaStockQuoteService getstockquote BetaStockQuoteService getstockquote In this case, you need to specify Servers and Ports so that the system knows which WSDL file to use. For example, AlphaStockQuoteService may run on server 192.168.1.10, while BetaStockQuoteService 197
runs on server 192.168.1.11. You need to specify those server IPs for each WSDL file in order to get the correct SOAP Web Service name to appear for each SOAP Operation. Uploading a WSDL file and indicating servers and ports To upload a WSDL file and indicating server and port: 1 Click Add a WSDL file on the main SOAP page. The SOAP > SOAP WSDL File page is displayed. 2 In the WSDL File section, click Browse, and search for the WSDL file on your network file system. 3 In the Prefix box, specify the prefix the WSDL file uses for its elements (for example, wsdl:message), to assist the appliance when parsing the file. 4 Click Upload to upload, and begin examining the WSDL file. When this process has been completed, the Service Name and the Operations list is populated with names obtained from the WSDL file. 5 If you have two SOAP Web Services using the same SOAP Operation name: a b In the Servers section, click Add Server to open a list of monitored servers from which SOAP services are delivered. You can select all servers, or select individual servers to build a list. In the Ports section, select the check boxes that correspond with the monitored ports over which SOAP services are delivered. 6 Click OK to complete the process. You are returned to the main SOAP page, where the newly added SOAP WSDL file and server information are displayed. Adding and removing SOAP tags Click SOAP Tags, at the bottom of the SOAP page, to display a list of those that are in use by the appliance. These tags help the appliance recognize constructs in SOAP messages. The Fault Code Tags, Fault Subcode Tags, Fault String Tags, Fault Actor Tags, Fault Details Tags, and Envelope Body Tags sections allow you to add any uncommon SOAP tags that you may be using in your SOAP application. 198
Figure 85. Fault Code Tags Type the tag in the appropriate section, then click Add. Figure 86. Fault Details Tags NOTE: Less commonly used SOAP tags can be added. To remove a tag, select its corresponding check box, and then click Delete. Configuring enterprise-wide service levels The Enterprise Service Levels settings allows you to define service level policies across the entire enterprise, through the metrics available in the Enterprise category. You can define these settings by navigating to the Configure > Reporting > Options page. 199
Figure 87. Options page Depending on your site, it may be more appropriate to monitor service level compliance at the Application Component or Service level. For more information, see Setting enterprise-based service levels. Setting enterprise-based service levels To set enterprise-based service levels: 1 In the End-to-End Time Service Level Threshold box, type the desired service level for page response time across your enterprise. For example, if you decide that most of the page downloads should be serviced within five seconds, type 5000 (units are in milliseconds). The End-to-End Time Service Level metric shows you the percentage of page downloads that satisfy this service level. For information on how this metric is calculated, see Page-End-to-End Time in the Foglight Experience Monitor Metric Reference Guide. 2 In the Processing Time Service Level Threshold box, enter the desired service level for page processing times across the enterprise. For example, if you decide that the time to process the all pages should generally not exceed two seconds, type 2000 (units are in milliseconds). The Processing Time Service Level metric shows you the percentage of pages whose processing time satisfies this service level. For information on how this metric is calculated, see Page Processing Time in the Foglight Experience Monitor Metric Reference Guide. 3 Click Apply to save your changes. NOTE: The appliance users working with pre-5.x versions should note that the End-to-End Time Service Level used to be known as the Download Time Service Level. 200
Foglight components and the appliance 7 The Foglight Experience Monitor is an integral component of Foglight and offers a number of integration points with other Foglight components such as Foglight Experience Viewer, Synthetic Transactions, and Foglight Management Server. This section describes how to configure and use these integration points. For more information, see these topics: Connecting to the Foglight Experience Viewer Foglight Management Server Synthetic transaction scripts Connecting to the Foglight Experience Viewer The Foglight Experience Viewer is an appliance-based solution that gives organizations the ability to capture, store and play back web user sessions in real time for immediate insight into application failures. With Foglight Experience Viewer, you can store and replay user sessions to see the exact content that was delivered to the user by your application. This gives you a visual and factual record of all end-user and application activity which can be used to troubleshoot application errors, improve application reliability, and analyze user activity. Foglight Experience Viewer is highly scalable, and manages large amounts of user session data through a distributed relational database across multiple appliances, thus enabling virtually unlimited session storage. For more information, see these topics: How Foglight Experience Monitor and Foglight Experience Viewer work together Connecting the appliance to your network Connecting Foglight Experience Viewer to a Foglight Experience Monitor Configuring monitoring appliances How Foglight Experience Monitor and Foglight Experience Viewer work together Foglight Experience Monitor can be configured to transmit HTTP or decoded HTTPS traffic to one or more Foglight Experience Viewer archivers. Foglight Experience Monitor does this by connecting to the Foglight Experience Viewer server and using the obtained configuration information to communicate with the Foglight Experience Viewer archivers. This capability allows both products to operate in a passive mode, utilizing a single network tap that is connected to Foglight Experience Monitor. 201
Figure 88. Foglight Experience Viewer integration This is how Foglight Experience Viewer and Foglight Experience Monitor can use a single network tap to gather and store HTTP and network traffic, respectively. Connecting the appliance to your network For instructions on how to install, connect, and configure your appliance, see Installing and configuring. The server rack connection instructions in Connecting the appliance assume that the Foglight Experience Monitor is not being connected to a Foglight Experience Viewer archiver. If you are integrating Foglight Experience Viewer with Foglight Experience Monitor, the network cables are connected differently than described in that procedure. The following diagram shows the connection between a Foglight Experience Monitor R610 machine and a Foglight Experience Viewer R610 appliance. Figure 89. Connecting the appliance to your network category 5e crossover path cable category 5e path cable Foglight Experience Monitor D C Network Switch E A B Network Tap Foglight Experience Viewer A - Connection between Foglight Experience Monitor and Foglight Experience Viewer interface ports (eth1) B - Connect monitor port A to eth2 C - Connect monitor port B to eth3 D - To control port eth0 E - To control port eth0 The next two sections replace the connection steps outlined in Connecting the appliance. Follow the appropriate section, depending on your network environment. 202
Connecting Foglight Experience Viewer to a Foglight Experience Monitor Refer to the diagram when following the procedure in this section. To connect Foglight Experience Viewer to Foglight Experience Monitor: 1 Attach network cables to the monitoring ports. The number of monitoring ports available on your appliance varies, depending on the hardware type. For a list of supported hardware platforms and guidance in identifying the monitoring ports, see Multiple monitoring ports and Appendix: Dell PowerEdge system appliance. For fiber networks, the monitoring ports in the appliance will use an LC connector. If your network uses SC connectors, you will need to use an SC-to-LC adapter for each cable. 2 Verify that the cables you attached in step 1 are connected to the network tap. 3 Attach a second network cable to the control port. 4 Plug the other end into the network switch through which Foglight Experience Monitor users can access the web console. 5 Attach a category 5e crossover patch cable to the auxiliary port, whose location is shown below: Monitoring Ports Control Port Monitoring Ports Auxiliary Port 6 Attach the other end of the network cable to the Foglight Experience Viewer Archiver. NOTE: If your installation requires multiple archiver appliances then you should use a standard network cable and attach it to the network switch through which the Foglight Experience Monitor can transmit data to the Foglight Experience Viewer archivers. Configuring monitoring appliances When integrating Foglight Experience Monitor and Foglight Experience Viewer, you need to ensure certain settings in the console program have been configured. Make sure you are viewing the Network Configuration menu in the console program, which is accessed by pressing the 1 key at the main menu. 203
Figure 90. Configure Control Device menu The first step you need to perform is to assign an IP address to the auxiliary port (typically eth1) on Foglight Experience Monitor which is used to communicate with Foglight Experience Viewer. To assign an IP for the Auxiliary Port: 1 At the Network Configuration Menu, press the 3 key to display the Auxiliary Device Configuration menu. 2 Press the 1 key. When prompted, enter the IP address that is being assigned to the auxiliary port on the Foglight Experience Monitor (eth1). IMPORTANT: The auxiliary port IP address you provide must be on its own unique subnet, and should not share the same subnet as the IP addresses assigned to the control ports on the Foglight Experience Monitor and Foglight Experience Viewer archiver. This ensures that traffic passes from one appliance to the other over the crossover cable and not over the network. If you fail to configure this correctly, everything will work but large amounts of traffic will be passing over your network unnecessarily. 3 Press the r key to return to the Network Configuration menu. 4 Press the s key to save and verify your settings. Configuring the Foglight Experience Viewer Archiver You must configure IP addresses for the control port (eth0) and archiver port (eth1) on the Foglight Experience Viewer in order to complete the integration. The Foglight Experience Viewer archiver and Foglight Experience Monitor auxiliary ports need to be on the same subnet, and this subnet must be different than the Control port IP addresses of both appliances (eth0). These steps can be performed in the Foglight Experience Viewer console program. 204
To configure the Archiver: 1 Log in to the Foglight Experience Viewer Console Setup program using support as the user ID, and support as the password. The main Setup program menu appears: 2 Press 2 to select the Network Configuration option, then press the Enter key to display the Network Configuration screen. 3 Press 2 to select the Configure Network Cards options, then press the Enter key. 4 Press Enter to accept the Traditional Method with ifup network setup method. The Network Card Configuration Overview screen is displayed: The first listed network card IP address is assigned to the Foglight Experience Viewer s control port (eth0). The second IP address is designated for the Archiver (eth1), and is used to receive data from Foglight Experience Monitor. 5 Ensure the first item in the list is highlighted and then select the Edit option to enter the IP address for the control port. IMPORTANT: Make a note of this address, as you will need to use it when logging in to the Foglight Experience Viewer console. 6 Ensure the second item in the list is highlighted and then select the Edit option to enter the IP address for the archiver port (eth1). This is the port over which Foglight Experience Monitor will transmit data to Foglight Experience Viewer using the crossover cable that is connected directly between the eth1 ports on the two machines. 205
Ensure the IP and subnet mask are on the same subnet as the IP address given for the Auxiliary port in the Foglight Experience Monitor console Setup program (see To assign an IP for the Auxiliary Port:). If you edit the address, make a note of it. You will need it to perform a configuration step in the Foglight Experience Viewer console. 7 When the IP addresses have been configured, select Next and press the Enter key to save your settings. 8 Access the Foglight Experience Viewer console by entering the designated Foglight Experience Viewer appliance IP address in a browser s URL box, at port 80, using the /console path (for example, http://192.168.1.1:80/console). 9 Enter admin in the Login box, and admin in the password box, then click Login. 10 After logging in, click Configure in the main menu. 11 Click Collectors, found in the System Configuration section. If the Foglight Experience Monitor is successfully sending data to the Foglight Experience Viewer, a Monitor Group is automatically created. 12 Edit the Default Collector Group. In the Archiver IP Address box, enter the non-public IP address for (eth1) that you configured in the Network Configuration screen in step 6. NOTE: This address may already be configured by default. 13 Click OK. Your Foglight Experience Viewer should now be receiving HTTP traffic from the Foglight Experience Viewer channel over the proper private channels. You can test whether the Foglight Experience Viewer is receiving data by viewing the metrics on the Metrics > Archivers page after completing the steps below. 14 The final step is to provide Foglight Experience Monitor with the IP where it can find the Foglight Experience Viewer. You can access Foglight Experience Viewer server IP and server port settings from the Network Settings page by clicking Configure > Appliance > Network Settings. 206
Configuring Foglight Experience Monitor to write to a Foglight Experience Viewer Archiver 1 Access the Foglight Experience Viewer configuration page. 2 In the Foglight Experience Viewer Server IP box, enter the IP address of the machine that is acting as your Foglight Experience Viewer Server in your environment. This module performs configuration and reporting functions. 3 In the Control Port box, enter the TCP port number that your appliance will use to communicate with the Foglight Experience Viewer Server. 4 Click Verify settings to ensure the appliance is able to connect to the Foglight Experience Viewer Server using the provided information. The Verify Foglight Experience Viewer Settings dialog appears, indicating whether the connection attempt was successful. 5 Click Verify Settings to ensure the Foglight Experience Monitor can establish a connection with the Foglight Experience Viewer server. The Verify Foglight Experience Viewer Settings window displays, providing status messages regarding the success or failure of the verification process. 6 Click Apply to save your changes. Foglight Management Server Many of Foglight s broad monitoring and reporting capabilities come from its specialized cartridges. The Cartridge for FxM transmits key metrics, captured by the appliance, to the Foglight Management Server. These metrics can then be displayed in Foglight browser interface. Communication between the Cartridge for FxM dashboards in Foglight and the Foglight Experience Monitor appliance allows users to seamlessly view data between applications. After the appliance has been configured, and the cartridge has been installed and configured on Foglight, users who are logged into Foglight can drilldown to more detailed information monitored by the appliance. IMPORTANT: Port 3306 must be opened through any firewalls that separate the Foglight Management Server and Foglight Experience Monitor. No other ports are required to be open. For more information about Foglight and how Foglight Experience Monitor users can take advantage of their integration, see Integrating Foglight Experience Monitor in the Foglight Experience Monitor User Guide. For more information, see these topics: Ensuring product compatibility Setting up the Cartridge for FxM Configuring the appliance for data communication Exporting metrics to Foglight 207
Ensuring product compatibility It is important that you install compatible versions of Foglight, Foglight Experience Monitor, and the Cartridge for FxM. See the Foglight Experience Monitor Release Notes for detailed compatibility information. Setting up the Cartridge for FxM The Cartridge for FxM is installed on the Foglight Management Server (FMS) or anywhere that the Foglight Agent Manager is installed (this location is typically the FMS). The Cartridge for FxM is comprised of two CAR files, the EU-Core and the EU-FxM. In order for these files to display Foglight Experience Monitor data, the Foglight Agent Manager requires the FxM Agent be installed and configured to facilitate data communication. NOTE: For multiple appliance environments, you can configure multiple agents for data communication. There are several steps required to install and configure the cartridge on the FMS: 1 Obtain the EU-Core and EU-FxM CAR from Dell Support Portal and install them on the FMS. 2 Install the FxM Agent Package on the FMS. 3 Create the FxM Agent on the FMS. 4 Configure the FxM Agent Properties. 5 Activate the FxM Agent on FMS. 6 Enable data collection from the appliance on the FMS. For view detailed information about configuring the Cartridge for FxM, see the End User Monitoring Installation and Configuration Guide. Configuring the appliance for data communication Before data collection can begin on the Foglight Management Server, you need to establish communication between Foglight and the Foglight Experience Monitor, by configuring the following settings on the appliance: Configure the Foglight Server IP address and the Foglight Server Port on the Network Settings page. To navigate to this page, click Configure > Appliance > Network Settings. Enable remote database access for a special user account named dbuser and set the database password on the Database > Remote Access page. To navigate to this page, click Configure > Appliance > Database > Remote Access. This password is needed to configure the FxM Agent property settings on the Foglight Management Server. IMPORTANT: Enabling the remote database access allows users to access the appliance s MySQL database (metric database) from Foglight, but also through standard SQL query tools, such as Dell s TOAD. For example, any user could log in through TOAD using the dbuser account and the password entered on the Remote Access page, and see the contents of the database. 208
Select the metrics that you want to export to the Foglight Management Server using the Foglight Metrics page. To navigate to this page, click Configure > Reporting > Foglight Metrics. After the metrics are configured, the appliance sends the data to Foglight in five-minute intervals. See Exporting metrics to Foglight. Exporting metrics to Foglight You can configure a list of metrics that you want the appliance to export to the Foglight Management Server using the metric categories that appear on the Configure > Reporting > Foglight Metrics page. After the metrics are configured, the appliance sends the data to Foglight in five-minute intervals. Before you configure metrics for export, you must first configure the metric categories on the Database page. For more information, see Metric categories. Categories that have been disabled on the Database page are not eligible for export regardless if they are configured on the Foglight Metrics page. When configuring metrics for export, metrics that are contained in child categories, for example Application by City is a child category of Application, are only exported when the parent category (Application) and the corresponding cities are selected. 209
Figure 91. Foglight Metrics page Configuring metrics for export To configure metrics for export: 1 Click the link located at the bottom of any category. For example, click Add Application Component. The Resource List page opens. 2 To refine your search, in the Search text box, type a regular expression to filter the metrics list and click Go. You can use an asterisk (*) as a wild card to define a segment of the string that matches any combination of characters. 210
3 Select the metrics that you want to export. 4 Click Apply. The metrics selected will be exported to Foglight. Synthetic transaction scripts The Foglight Transaction Recorder is a Foglight component that provides the ability to record and play back synthetic transaction scripts that generate synthetic traffic against a web site. The appliance can detect synthetic transaction scripts, and allows you to associate them with service definitions. For more information, see Configuring synthetic transactions. 211
8 Using the console program The console program is used for performing initial configuration when you first install your appliance. It also contains a number of administrative and troubleshooting features that you may need to use from time to time. This section provides a description of the console program and its capabilities. For more information, see these topics: Accessing the console program Exploring the main menu Network configuration System date and time configuration Database management Account management Appliance update Advanced options System restart Troubleshooting Reporting system status Logging out of the console program Accessing the console program In order to view the console program, you must connect a VGA monitor and USB keyboard to the appropriate ports on the rear panel of the appliance to which you wish to connect. For more information, see Appendix: Dell PowerEdge system appliance. The login name is always setup, and by default, the password is setup. If you have changed the setup account password during either the appliance s initial configuration, or after installation, use the new password instead. TIP: It is possible to log in to the appliance remotely using secure shell (SSH). This function is enabled in either the web console or the console program, and is normally used to facilitate assistance from external users (Dell Technical Support). For more information, see these topics: Logging in using the Dell Remote Access Controller (DRAC) Logging in from Microsoft Windows Logging in from Linux and UNIX 212
Logging in using the Dell Remote Access Controller (DRAC) For information about connecting to the DRAC, see the Dell Remote Access Controller documentation. To log in to the DRAC using Microsoft Internet Explorer web browser: 1 Open Microsoft Internet Explorer Trusted Sites, and add the IP address of the appliance to the location DRAC is installed. This enables Internet Explorer to download and install signed ActiveX controls. 2 Log in to DRAC using the default settings: login name = root password = calvin The DRAC window displays. 3 From the Console tab, in the Console Redirection page, click Connect. A session Viewer window opens. 4 Log in as an administrative user with terminal access. The Foglight Experience Monitor setup utility window appears. You may now configure various aspects of the system. See Exploring the main menu. Logging in from Microsoft Windows Administrators with Windows workstations must install PuTTY, in order to access the console program. PuTTY is free software, which provides a remote shell interface to the appliance, allowing users to enter and execute commands from another computer. You can download PuTTY from: http://rc.vintella.com/topics/ putty/. To login from a Microsoft Windows workstation: 1 Run PuTTY. The PuTTY Configuration screen appears. 2 Type the appliance IP address or DNS name in the Host Name (or IP address) text box. 3 Ensure that the SSH option button is selected. 4 Click Open. The first time you connect to the appliance, you will see a warning that the RSA fingerprint cannot be authenticated. 213
5 To accept the RSA certificate, click Yes. 6 Log in as an administrative user with SSH access. The Foglight Experience Monitor setup utility window appears. You may now configure various aspects of the system. See Exploring the main menu. Logging in from Linux and UNIX Most administrators with UNIX or Linux workstations already have command-line tools installed for SSH and SCP. To log in from a Linux or UNIX workstation: 1 Start the SSH client. ssh -l [username] myappliance Where [username] is the login name of an administrative user with SSH access and myappliance is the name of the appliance you want to access. The first time you connect to the appliance, you will see a warning that the RSA fingerprint cannot be authenticated. 2 To accept the RSA certificate, click Yes. 3 Enter your password. You are now logged in to the appliance. Exploring the main menu After logging into the appliance, the main menu appears. Figure 92. Welcome to Setup menu 214
The appliance remains running for most operations, but could restart for some operations like, changing the IP address of the appliance, modifying the timezone, modifying the appliance type or repairing the database. You will receive an message before the appliance restarts. IMPORTANT: Whenever any changes are made using the console program, you must always return to the main menu in order to properly exit and retain your changes. Console program navigation Use the arrow keys to scroll through the list of menu items. To select a menu item, ensure that it is hightlighted then press Enter. Alternatively, you can select the number that corresponds to the highlighted menu item and press Enter. To navigate to any of the menu controls at the bottom of the screen (for example, OK, Cancel and Exit), use the Tab key to move the cursor and then press Enter. Receiving error messages The console displays error messages when data is entered incorrectly. For example, if you enter an IP address incorrectly, the console responds with an error message. Figure 93. Example of an error message Saving changes and exiting When prompted, ensure that you save your changes each time you modify a setting. When changing the appliance type, you must save your changes in the Appliance Type Configuration screen. For more information, see Configuring the appliance type. Network configuration The Network Configuration menu and submenu options identify essential elements on your organizations network that the appliance requires to function correctly. 215
Figure 94. Network Configuration menu There are six different sections: Control Device allows you to configure the network settings for the control device. Auxiliary Device allows you to configure the settings for the auxiliary network device used to connect to a Foglight Experience Viewer appliance. Monitor Devices allows you to display network configuration for the monitor devices on this appliance. Advanced Network Settings allows you to setup the network routing. Enable/Disable Monitoring Jumbo Frames allows you to configure the system to handle jumbo frames in the monitored traffic. By default, this feature is disabled. IMPORTANT: This option should only be enabled if you have confirmed that the traffic you are attempting to monitor contains jumbo frames. Exercise caution when enabling this option, as there will be some performance penalty, particularly with older Dell models. Save Settings allows you to save the configured settings. For more information, see these topics: Configuring the control device Configuring the auxiliary device settings Displaying information for monitor devices Advanced network settings Configuring the control device The configure control device options allow you to configure the network settings for the control device (eth0). From the Networking Options menu, select Control Device. The Configure Control Device menu displays with configuration options. 216
Figure 95. Configure Control Device menu Table 29. Configure Control Device menu options Menu Option IP Address (required) Netmask Default Gateway (required) Hostname Primary DNS Secondary DNS Description Assigns an IP address to the control device (eth0). This IP address will be used by end users to gain access to the appliance's web console. It is also the IP address that is used to open an SSH session with the appliance. Allows you to enter a subnet mask for the IP address. Allows you to enter the IP address of your network's gateway. Allows you to enter a hostname for the appliance. Users can utilize the hostname, rather than the IP address, to access the web console provided the hostname is mapped to the IP address of the appliance in your Domain Name Server (DNS). Make sure you enter a fully-qualified domain name (FQDN) such as agent5.dell.com. Allows you to enter the IP address of the primary Domain Name Server for your network. Allows you to enter the IP address of the secondary Domain Name Server for your network. Configuring the auxiliary device settings From the Networking Options menu, select Auxiliary Device. The Configure Auxiliary Device menu displays with configuration options for setting the IP address, netmask, and broadcast address for the appliance s auxiliary device (eth0). Figure 96. Configure Auxiliary Device menu 217
Table 30. Configure Auxiliary Device menu options Menu Option IP Address Netmask Description Assigns an IP address to the auxiliary device (eth1). This IP address must be on a different subnet than the IP address assigned for the control device (eth0). It must also match the subnet of the IP address assigned to the corresponding port on the Foglight Experience Viewer. Allows you to enter a subnet mask for the IP address. Displaying information for monitor devices The Monitor Devices menu displays the network configuration information for the ports that are available for monitoring on the appliance. The main menu displays a list of NIC ports. The link detected status means that the NIC is active. Select the NIC port whose information you would like to view. Figure 97. Monitor Devices menu Select a NIC port and view the device status. Figure 98. Device Status menu To read the entire report, use the arrow keys to scroll down. To return to the Monitor Devices screen, tab to OK and press Enter. 218
Advanced network settings From the Networking Options menu, select Advanced Network Settings. The menu displays advanced settings for a routing configuration. Figure 99. Routing Configuration menu Table 31. Routing Configuration menu options Menu Option Routing Configuration Alt + X Alt + B Alt + R Alt + F Description Allows you to create custom routings. This options opens the YAST Route Configuration screen. Allows you to enter advanced network configurations such as custom routes. Accesses the Back command. Returns you to the Advanced Network Settings menu. Abort the current process and return to the Advanced Network Settings menu Finishes and saves the routing configuration. System date and time configuration The System Date/Time Configuration menu options allow users to change the following: Change Date allows you to enter a month and year. Using the calendar and the arrow keys move to the day that you want to set. Change Time allows you to enter a new time in 24-hour hh:mm:ss format. Change Timezone displays a new screen, allowing you to select a timezone. 219
Figure 100. System Date/Time Configuration menu The appliance clock can also be synchronized with a Network Time Protocol (NTP) server. For more information, see Designating Time Servers and Network Time Servers (NTP). For more information, see these topics: System date configuration System time configuration System timezone configuration System date configuration The System Date Configuration menu allows users to set the month, day and year of the appliance. Use the tab key to move between month, year and calendar day settings. While in one of the options (for example, month) use the arrow keys to move between selections (calendar months). Figure 101. System Date Configuration menu You receive the following message when the date is changed. 220
Figure 102. Success message System time configuration The System Time Configuration menu allows user to change the appliance s system time. Enter a new time in 24-hour hh:mm:ss format. Figure 103. System Time Configuration menu System timezone configuration The System Timezone Configuration menu allows users to change the appliance s system timezone. From the System Date/Time Configuration menu, select Change Timezone. The YAST window opens. This menu allows you to select the clock (for example, Universal Time Clock (UTC)) the timezone and where necessary reset the time and the date of the appliance. TIP: You can use the following Alt+C to move to Cancel, Alt+A to move to Accept, and Alt+F to Finish. 221
Figure 104. Time Zone and Clock Settings menu Database management The Database Management menu options allow users to reset and repair the configuration and metric databases. For more information about database configuration, see Database configuration. IMPORTANT: After selecting options 2 through 6, a dialog appears asking if you want to continue with the process that you selected. To continue the process you will need to respond Y to the message. Figure 105. Database Management menu The following options are available: Reset Configuration Database re-initializes the configuration database. This option clears all of the configuration settings found in the web console, but it does not clear the network settings (such as the IP address, host name, and gateway IP) that are configured in the console program. Reset Metric Database re-initializes the metrics database. All metrics captured up to this point are discarded. This option does not affect any configuration settings. If you have enabled remote database access, you need to re-enable access after resetting the metric database. 222
Repair Metric Database runs a repair utility that finds and fixes inconsistencies or corruption in the metrics database. The appliance applications are disabled while the repair operation runs. This utility may take several hours to run. When the utility is finished running, the applications are restarted after the operation is completed. When necessary the system runs an extended version of the repair utility that finds and fixes inconsistencies or corruption in the metrics database. This utility may take 10 hours or longer to complete. It should be used only as a last resort when option 5 does not fix the problems. Backup Database to USB Device this option performs a full database backup to a USB device. The USB device needs to be formatted in FAT32 or Linux, depending upon you environment, and the storage capacity must be large enough to hold the entire database backup. The USB device can only hold one database backup, previous backups will be overwritten. Restore Database from USB Device this option restores the database backup from the USB device to the appliance. Account management The Account Management menu allows you to change your password and enable or disable SSH and web console access for your account. These functions are available in the console program primarily to provide a way for rescuing the system, if you have forgotten your password for the web console account. If this happens, you should be able to log in to the console program using the default setup account, which allows you to access the web console, assuming that you have not removed that account or lost its password. Figure 106. Account Management menu For more information, see these topics: Change password Enable and disable SSH access Enable and disable web access Change password Use this option to change the password for the user account whose credentials you used to log in to the console program. NOTE: You are required to enter a strong password if this option is in force for the appliance or if your account has SSH access enabled. 223
Figure 107. Enter Password menu IMPORTANT: If you modify the setup account password, make sure that you note the new password, so that you can log in to the console program in the future. Enable and disable SSH access This option allows you to enable SSH access for the user account whose credentials you used to log in to the console program. NOTE: You must have a strong password for this option to be enabled. Enable and disable web access This option allows you to enable web console access for the user account whose credentials you used to log in to the console program. Appliance update The Appliance Upgrade menu provides options that allow you to update the software on the Foglight Experience Monitor from a CD, using a PKG file, or using a USB flash drive. Figure 108. Appliance Upgrade menu For more information, see these topics: Update from CD 224
Update from PKG file Update from USB flash drive Update from CD Allows users to update the appliance from an update CD. IMPORTANT: Obtain the CD file from the Support Portal. After selecting this option, you will be prompted to insert the CD into the CD drive and press the Enter key. The update program will update the appliance with the software contained on the CD. After the update is complete, press the Enter key to eject the CD and return to the Appliance Update menu. Update from PKG file This option allows users to update the appliance, using a PKG file, when you do not have access to FTP. The PKG file can contain either a hot fix or a full upgrade. To update from PKG file: 1 Obtain the PKG file from the Support Portal. 2 Copy the PKG file to the home directory (/home/[username]) of the user logged in to the appliance. For example, if you are logged in as setup, copy the PKG file to /home/setup. 3 Select this option to begin installing the update. 4 Reboot the appliance to complete the update. Update from USB flash drive This option allows users to update the appliance using a PKG file that is saved to a USB flash drive. The PKG file can contain either a hot fix or a full upgrade. NOTE: The USB flash drive must be formatted using standard Fat32 or NTFS. To update from USB flash drive: 1 Obtain the PKG file from the Support Portal. 2 Copy this file to the root of the USB device. The file name must use the.pkg extension. NOTE: The USB drive should contain only one PKG file. 3 Insert the USB drive into an open USB port on the appliance. NOTE: Dell PowerEdge series appliances have USB ports on the front and back of the machine. For more information, see Appendix: Dell PowerEdge system appliance. 4 From the console Appliance Update menu, select option 4 to run the update from the USB flash drive. 5 Reboot the appliance to complete the update. Advanced options Use the Advanced Options menu to configure the following options: 225
Configuring the appliance type Configuring Foglight Experience Viewer settings Reset firewall to allow HTTP Add and remove network device Configuring the license More advanced options Figure 109. Advanced Options menu Configuring the appliance type The Appliance Type Configuration menu displays a submenu with a list of appliance types. Figure 110. Appliance Type Configuration menu Enter the option number of the appliance type that you want to set. The following tables display the information that needs to be configured for each appliance type. IMPORTANT: Use the Save Settings option after configuring the appliance type. Option 1 - Portal Table 32. Option 1 - Portal Field Data Port Description The TCP port number (default is 5000) that the appliance uses for data communication with other appliances. 226
Table 32. Option 1 - Portal Field Control Port Encrypt Using SSL Description The TCP port number (default is 5001) that all appliance in the cluster use to communicate control and configuration information using MySQL. Set if you want the data communication between the portal and probes to be encrypted using SSL. Option 2 - Probe Table 33. Option 2 - Probe Field Portal IP Control Port Description Enter the IP address of the appliance that you configured to serve as the portal. The TCP port number (default is 5001) that the probe uses to retrieve control and configuration information from the MySQL database that is hosted on the portal. Option 3 - Stand-Alone Installs the appliance as a stand-alone appliance. For more information about configuring appliance types, see Configuring the Appliance Type. Configuring Foglight Experience Viewer settings The Foglight Experience Viewer Configuration menu displays a submenu with a list of Foglight Experience Viewer options. For more information about Foglight Experience Monitor and Foglight Experience Viewer integration, see Connecting to the Foglight Experience Viewer. Figure 111. Foglight Experience Viewer Configuration menu To set the Foglight Experience Viewer integration: 1 Select Foglight Experience Viewer Server IP and set the IP address. 2 Select Control Port. The default port is set to 80. 3 Select Save Settings. 4 Select Verify Settings to verify that the settings are working properly. 227
The Verify Settings option will not work unless the settings are first saved. Reset firewall to allow HTTP If you have configured the web server on the appliance to run in HTTPS mode this option allows you to reset the firewall to HTTP mode so the web console can be accessed through HTTP. Add and remove network device This option automatically detects whether any NICs have been added or removed from the system and configure their ports as monitoring ports. Configuring the license This option displays a submenu with licensing options. Figure 112. License Configuration menu Press 2 to enter your activation key. Figure 113. License Key Configuration menu 228
More advanced options The More Advanced Options menu displays a submenu of advanced settings. Figure 114. More Advanced Options menu The following options are available: Enable/Disable Detailed Logging enables either a core dump capture or call stack tracing, which may be useful for troubleshooting issues in agent and Sniffer processes. This option is password-protected. Contact Dell Support for assistance. Clear Datastore allows you to clear the temporary data cache files used to store data before it is sent to the database, portals, and to Foglight. Clear SQL Load Files clears temporary SQL and load files that were written to the database. Use this option when a hard disk is full or when a corrupted SQL load file is interfering with database processing (only your Dell Support representative can determine if this case applies to you). Reset Appliance to Factory Settings resets the appliance to factory settings. Delete Special Support Account deletes a hidden account that may be useful for Dell Support to troubleshoot some issues. Remove Shell Access From Console removes the Access Shell option from the More Advanced Options menu and, therefore, removes the capability to access the command-line Linux shell from the console program. IMPORTANT: The Remove Shell Access From Console operation cannot be undone. Access Shell allows for access to the command-line Linux shell. System restart From the System Restart menu, you can restart or shut down the appliance. 229
Figure 115. System restart Options menu Reboot the Appliance allows you to restart the appliance. Shut Down Appliance allows you to shutdown the appliance. Stop All Applications allows you to stop all appliance processes. You receive a message: All applications stopped from the console. When all processes are stopped, there is no user interface access. Start All Applications allows you to start all appliance processes. You receive a message: All applications started from the console. Troubleshooting If you are experiencing problems, you can use the Troubleshooting menu to help diagnose them. Figure 116. Troubleshooting Options menu Table 34. Troubleshooting options Menu Option Verify Network Configuration Verify Monitor Device Traffic Description This option runs a series of tests to validate the appliance s network settings. This option allows you to verify whether the NIC ports are receiving any traffic. If no traffic is displayed, verify the following: The port numbers you entered have TCP traffic on them. The monitor network cable works and is plugged into the monitoring port. 230
Table 34. Troubleshooting options Menu Option Verify Access to Web Server Verify Access to IP Address Verify DNS Resolution Verify Network Device Status Verify Trace Route Using UDP Verify Trace Route Using ICMP Create Support Bundle Create TCP Dump File Blink LED on NIC Port Description This option verifies the web server is running and can be accessed locally. This option allows you to ping an IP address to verify that there is a network path from the appliance to the machine responding to the IP address you specify. This option is useful if some users report they are unable to access the web console. This option allows you to enter a site name and verify DNS resolution is working. If DNS resolution is working, one or more IP addresses will appear next to the site name. If DNS resolution is not working, an error message will appear. This option displays the status of the Network Interface Cards. Use this option to verify the NIC drivers are loaded, and there are no hardware problems. If the output appears in the same format as the sample shown below, the NIC drivers have been loaded correctly. Be sure there are not an excessive amount of RX and TX errors, dropped packets or overruns. This may indicate a hardware problem with the NIC. If you plan on using trace routes in alarm actions, these options allows you to verify which protocols work with your current network configuration. If the UDP protocol test succeeds, no changes are necessary to the system. If only the ICMP protocol test succeeds, you must configure the system to use the ICMP protocol for trace routes (For more information, see Traceroute protocol). If both tests fail, you need to configure your network to allow trace route capability. NOTE: Consult your network administrator to ensure that traceroutes are enabled on your network. Each option prompts you for an external host name to trace route to. Enter a valid external host name, and press Enter. A trace route is executed to the host name you entered. After the trace route is executed, the output indicates whether the test succeeded or failed. This option creates a support bundle of all the log files on the appliance in the home directory of the user logged in to the appliance. For example, if you are logged in as setup, the directory /home/setup contains the support bundle. Secure Copy (SCP) these files to a path on your computer and then email them to Dell Technical Support. This option allows you to capture a tcpdump file and transmit it to the Dell FTP site. You need to specify the ports used to find traffic, and the amount of time during which packets will be captured. NOTE: A TCP dump file could contain confidential information depending on what type of application you are monitoring. Make sure you are following your organization s security policies and use appropriate safeguards when transferring TCP dump files to Dell Technical Support. In some cases, your security policies may not allow you to transfer such data Use this option to confirm which physical NIC port on the appliance corresponds to the port identifier: eth0, eth1, eth2, eth3, and if available eth4, eth5, eth6, and eth7. Sample output for NIC driver The following example shows a confirmation of a correctly loaded NIC drivers. eth0 231
Link encap: Ethernet Hwaddr 00:11:22:33:44:55 inet addr: 192.168.1.2 Bcast: 192.168.1.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1217 errors:0 dropped:0 overruns:1 frame:0 TX packets:118 errors:0 dropped:0 overruns:0 frame:0 collisions:0 txqueuelen:100 RX bytes:327343 (319.6 Kb) TX bytes:11041 (10.7 Kb) Interrupt:11 Base address:0xec80 eth1 Link encap: Ethernet Hwaddr 00:11:22:33:44:77 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:421312 errors:0 dropped:0 overruns:1 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 frame:0 collisions:0 txqueuelen:100 RX bytes:26128330 (24.9 Mb) TX bytes:0 (0.0 b) Interrupt:25 Reporting system status The options available in the Report system status menu help you verify the service tag, and that the appliance hardware is functioning correctly and within reasonable limits. Figure 117. System Status menu 232
Table 35. System Status menu options Menu Option System Health RAID Array Status Description This option displays vital system health information, including: appliance fan speeds CPU and chassis voltage system temperature CPU temperature power supply status This option displays information about the RAID array configuration in the appliance, including each drive s present state. Show Version Numbers Dell Service Tag Dell OpenManage Administrator Run Dell System E-Support Tool Older appliances do not have a RAID configuration, and do not report RAID status information. This option displays version numbers for software components such as the Linux kernel, Apache, PHP, OpenSSL, OpenSSH, and MySQL that are currently installed on the appliance. This option displays the service tag number that is located on the back of the Dell appliance. Use this option to enable or disable access to the Dell OpenManage Administrator on the appliance. This site is useful for performing certain functions with the system hardware. Use this option to create a compressed file of hardware diagnostics. This file is saved in the user s home directory (for example, for the setup user, it is saved under /home/setup), and can be sent to Dell Technical Support, for troubleshooting, in the event of Dell hardware issues. To access the Dell OpenManage utility on your appliance: 1 In the FxM console program, enable the Dell OpenManage Administrator option and define a password for the openmanage user account. a Navigate to the System Status menu and click Dell OpenManage Administrator (by default, this option is disabled). The Enable OpenManage Administrator screen displays. b Click Yes to proceed with the enabling process. The Enter Password screen is displayed. 233
c d Define a strong password for the openmanage user account that will be used to access the Dell OpenManage utility on your appliance, and then click OK. In the Confirm Password screen, re-type the password defined (in Step c) for the openmanage user account, and click OK. The user account openmanage is enabled and the Status screen is displayed. e f Write down the URL displayed on this screen; this is required in the following steps, to log in to the OpenManage utility. To return to the System Status menu, click OK. 2 Log in to the Dell OpenManage utility using the openmanage user account. a In a web browser, type in the URL for accessing the Dell OpenManage utility in your appliance (see Step e). The Dell OpenManage web interface is displayed. b Type in the login credentials for the openmanage user account (Username/ Password = openmanage/ password defined in Step c), then click OK. You are now logged in to the Dell OpenManage Server Administrator web interface. For detailed information about this utility, see the online help documentation (click Help on the menu bar). 234
Logging out of the console program Navigate to the main menu to exit. 235
Troubleshooting the appliance 9 This section describes recommended processes that may help you solve connectivity or reporting issues with the Foglight Experience Monitor, after successfully installing and configuring it as outlined in Installing and configuring. It also provides information about possible issues that administrators or users may experience when using the appliance. For more information, see these topics: Common issues when installing the appliance Appliance runtime issues Confirming system health Using the appliance support tools Common issues when installing the appliance When installing a new appliance, there are two common issues that may be encountered: appliance connection failure through a web browser all reports are empty For more information, see these topics: Appliance connection failure through a web browser All reports are empty Verify the network configuration and connectivity Verify the network tap configuration Verify the server configuration Appliance connection failure through a web browser This common issue occurs when the user points their web browser at the appliance and an error is returned, stating: The page cannot be displayed. In almost all cases, the problem is caused by incorrect entries in the appliance network settings, or the fact that the appliance is not plugged into an active switch port. All reports are empty If no data appears in Monitoring Reports or other reports, this may be the result of an incorrect deployment of the appliance. After the initial configuration, the system may need between five to ten minutes to collect data 236
before it starts appearing in reports. After this period, if there is still no data, then there may be a deployment problem. In almost all cases, the problem is caused by either incorrect configuration of the network tap, missing server IPs, and ports in the configuration. Where to start The troubleshooting process documented in this chapter is divided into three main sections: 1 Verify the network configuration and connectivity 2 Verify the network tap configuration 3 Verify the server configuration These troubleshooting sections are meant to be followed in the order they appear, and it is recommended that you stay with the prescribed order. However, the type of problem you are experiencing with the appliance helps determine exactly where along the troubleshooting process you should begin. Use the following table to determine which section is the best place to start. Table 36. Where to start troubleshooting Problem failure to connect to the web console all reports are empty Solution: Begin by... verifying the network configuration and connectivity verifying the network tap configuration Regardless of which section is the start point, continue through all of the subsequent sections until the problem has been resolved. Each section will help the user to verify whether the system is working correctly or whether a configuration error has occurred. Verify the network configuration and connectivity During the initial configuration of the system, the network configuration may not have been set up correctly. There may also be a hardware problem with one of the network interface cards (NICs), or network cables. It is also possible that the switch port for the control NIC is inactive or has been configured incorrectly (for example, on the wrong VLAN). Step 1: Review network settings These basic steps help ensure that the network configuration settings have been entered correctly. 1 Log in to the console program. 2 Select Network Configuration. 3 Compare the displayed information with that provided by the network administrator: a b c d e f Is the correct IP listed? Is this IP unique and not in use by another machine? Is the correct Netmask listed? Is the correct Broadcast Address listed? Is the correct Gateway listed? Is the Host Name correct? NOTE: A Host Name value is not required for network connectivity, but you should verify it anyway. 237
4 Fix any incorrect entries. 5 Use the Save and verify settings command. The system will stop applications and verify the network configuration. 6 Proceed to the next section, Step 2: Test network connectivity. Step 2: Test network connectivity 1 Go to the Console Setup Main menu 2 Select Troubleshooting. 3 Select Verify network configuration to run a series of tests. Yes: Access to Control NIC verified? Proceed to the next test. No: One of two problems exist: the network card is malfunctioning and needs to be replaced, or the network card requires an updated driver. Access to gateway IP verified? This test simply pings the gateway IP, and verifies the appliance can see another machine on the network, resolving any physical connectivity issues in one step. Access to the gateway is specifically tested since the appliance is not available to the rest of the network without the gateway IP (unless all browsing to the appliance will be done from a client on the same LAN/VLAN). This test may fail if the gateway does not respond to ping requests. Yes: Proceed to the next test. No: Advance to Step 3: Resolving failed network connectivity. Yes: Verify access to DNS test successful? (For this test, it is assumed that a DNS IP has been entered). Proceed to the next test. No: Advance to Step 4: Resolving routing issues. Yes: DNS Lookup on... test successful? (For this test, it is assumed that a DNS IP has been entered). Proceed to the next step. No: Advance to Step 4: Resolving routing issues. 4 If all of the above tests were successful, you can stop using this troubleshooting guide, and continue with normal system configuration and operation. Step 3: Resolving failed network connectivity This section assumes that the test of the previous section, Step 2: Test network connectivity, has failed. There are several common reasons for failed network connectivity. This section outlines possible reasons for this failure, and what can be done to resolve the issue. 238
IMPORTANT: While not explicitly stated in the following sections, the user should go back and retry the steps outlined in the previous section (Step 2: Test network connectivity) after each change made to cabling, appliance settings, or switch settings to see if the change has resolved the problem before trying a different change. Follow the instructions in that section to determine where to proceed next. Invalid IP addresses: Settings incorrectly entered Go back and complete Step 1: Review network settings. Network cables incorrectly connected 1 Verify the cable from the switch and network tap is connected to the appliance monitoring NIC. 2 Verify the cable from the switch is connected to the appliance control NIC. 3 If you are not able to identify the cable ports, If you are not able to identify the correct ports that you should be plugging into, you can use the console program to blink the LEDs on the NIC ports. Under 8) Troubleshooting choose C) Blink LED on NIC port. 4 Go back to Step 2: Test network connectivity. Defective cables When a port s link light is not on, this could be indicative of a faulty cable. Despite this, it is still possible that the link light is on even when a defective cable is being used. Replace cables that are suspected to be defective with different cables from a system that is known to be working. Auto-negotiation not working By default, the appliance network interface card (NIC) and switch port (for example, Cisco 2948) are configured to auto-negotiate both the link speed (10 Mbit, 100 Mbit, or 1000 Mbit), as well as the half/full duplex settings. Some NIC/switch combinations and network taps have difficulties automatically negotiating the correct line speed or duplex mode between the appliance monitor NIC and the switch port. Symptoms of a failed auto-negotiation process include a flashing link light, or a link light that only lights up for several seconds intermittently. Despite these symptoms, it is still possible that the link light will remain lit up even if there is an autonegotiation problem. When in doubt, try the solutions below. 1 Restart the appliance, as this will sometimes resolve auto-negotiation issues. a b c Go to the Console Setup Main Menu. Select System restart. Select Restart appliance. 2 Disable auto-negotiation for the appliance and manually set the speed. a b c Go to the Console Setup Main Menu Select Advanced Configuration. Select Force monitor NIC settings. Select the appropriate duplex and network speed. If you suspect the network tap is having problems negotiating the correct line speed or duplex mode, you can set it to operate at half duplex and a fixed speed. 3 Disable auto-negotiation on the switch port. Contact the network administrator to do this. 239
Control NIC switch port configured incorrectly Modern switches allow a variety of settings to be changed for a switch port. Switch ports can be incorrectly configured in numerous ways: the switch port is configured as a span port the switch port line speed value is incompatible with the appliance NIC the switch port is set to use jumbo Ethernet frames (gigabit switches only). The appliance supports frames from 9000 bytes in size not exceeding 64KB in size. the switch port is assigned to the wrong VLAN Try connecting a second machine using the same control port cable and the same network settings. Does this second machine correctly access the network? No: The switch port is configured incorrectly. The network administrator will need to correct the issue. Yes: There is a possible problem with appliance NIC. Contact Dell Technical Support about getting a replacement. Connect the appliance to a switch port that was previously being used successfully by another machine. Does it work now? Yes: The original switch port is configured incorrectly. The network administrator will need to correct the issue. No: There is a possible problem with appliance NIC. Contact Dell Technical Support about getting a replacement. Step 4: Resolving routing issues This section assumes that the following results have occurred based on diagnostic actions taken in Step 2: Test network connectivity: the Verify Access to DNS test has failed the Access to gateway IP test was successful the Foglight Experience Monitor is correctly connected to a LAN/VLAN A DNS access verification failure may be caused by a routing issue. In most cases, the DNS server is not located on the same LAN as the appliance. This means DNS requests from the appliance must cross one or more routers to get to the DNS server. If there are routing issues, this cannot happen. Also, if the appliance cannot access the DNS server, this typically means machines located on other segments cannot access the appliance. In this case, where the appliance is isolated to a single segment, only browsers running on client machines on the same network segment can access the appliance. The following sections outline possible reasons for this failure, and what can be done to attempt to resolve the issue. Appliance located on isolated LAN/VLAN It is possible that the appliance is located on an isolated LAN/VLAN, which does not have access to a network with DNS servers move the appliance control cable to a LAN that is attached to the intranet add a router between LAN and intranet 240
Appliance located on LAN/VLAN behind a firewall The appliance may be located on a LAN/VLAN that is connected to a network with DNS servers, but is also isolated behind a firewall. move the appliance control cable to a LAN that is not isolated behind a firewall add rules to the firewall to allow packets from the appliance IP to traverse the firewall DNS server configured to ignore pings Although this is not a routing issue, it does cause the Verify Access to DNS test to fail. For more information, see Step 2: Test network connectivity. This configuration is sometimes used on internal DNS servers that are hardened on extremely security conscious sites. 1 Go to the Console Setup Main Menu. 2 Select Troubleshooting. 3 Select Verify access to web client to try pinging different machines that are known to be attached to the network. Gateway IP incorrectly defined The Access to gateway IP test (mentioned in Step 2: Test network connectivity), tries to ping the target IP. If the target IP points to an operational machine, it is still possible that the machine is not the correct gateway for the LAN. If so, the target machine will respond to the ping, but will not be able to route requests from the appliance, leaving it isolated. Confirm with the network administrator that the gateway IP provided is correct for the LAN segment where the appliance is attached. Verify the network tap configuration At this point in the troubleshooting process, the appliance should be configured so that it has connectivity to the network. The user can successfully point a web browser at the appliance and log in to the web console. The next step in the verification process is to check that the Foglight Experience Monitor can actually see some web traffic on one of the default ports, either HTTP or HTTPS. IMPORTANT: If the appliance cannot see web traffic, then it will never generate any reports or alarms. Proceed with the following steps to verify that the appliance can indeed see web traffic. If there is a failure, then a series of steps are provided to correct the problem. Step 1: Check for IP packets These basic steps simply help ensure that all setup information has been entered correctly. This step confirms whether the appliance can actually see any TCP/IP packets transmitted on the network. Failure to see any packets means that the network tap is not mirroring an active LAN/VLAN. 1 Log in to the Console Setup Program. 2 Select Troubleshooting. 3 Select Verify monitor NIC can see traffic. 4 Select the All ports option. 5 Verify whether a list of 10 TCP/IP packets appears. Yes: The system is working correctly. Go to Step 3: Check for web traffic visibility. 241
No: Continue to Step 2: Resolving unseen IP packets in this section. Step 2: Resolving unseen IP packets This section assumes that the Step 1: Check for IP packets has failed. There are several common reasons why the appliance may not be seeing any packets, which are outlined in this section. After you have found a solution, and see TCP/IP packets, proceed to Step 3: Check for web traffic visibility. Network cable disconnected During the installation of the new system, the cable between the appliance, switch and network tap may have been accidentally disconnected. 1 Disconnect and reconnect monitor cabling. 2 Be sure both ends of the cable are firmly seated. 3 Verify the link light is lit up. 4 Go back to Step 1: Check for IP packets. No network traffic currently transmitted This would typically only be the case in a test lab environment. If the link light on the port is not flashing, this is indicative that packets are not being transmitted. Try generating network traffic with a client on the monitored VLAN. Do IP packets appear? Yes: Proceed to Step 3: Check for web traffic visibility. No: proceed with troubleshooting. Step 3: Check for web traffic visibility This step verifies the appliance can actually see TCP/IP packets transmitted on the network that contain either HTTP or HTTPS information. 1 Log in to the Console Setup Program 2 Select Troubleshooting. 3 Select Verify monitor NIC can see traffic. 4 Select the appropriate ports to monitor. For most sites, select the Default HTTP and HTTPS ports option. If your site uses non-standard web ports, then you may need to select the Specify ports option, and enter the list of non-standard ports. 5 Verify whether a list of the next 10 web packets appears. Yes: The Foglight Experience Monitor is working correctly. Continue with Verify the server configuration. No: Continue to Step 3: Check for web traffic visibility. 242
Step 4: Resolving web traffic visibility This section assumes that the outcome of Step 3: Check for web traffic visibility has failed. There are several common reasons for failed web traffic visibility, which are outlined in this section. After you can see web traffic, proceed to the next section, Verify the server configuration. No web traffic currently transmitted This would typically only happen in a test lab environment, or on a production network that has a very small number of active users. With a client browser, try generating web traffic that will hit the monitored web site. Does web traffic appear? Yes: Proceed to Verify the server configuration. No: Continue to the next potential solution. Web traffic transmitted on non-standard ports The standard port for HTTP is 80, and 443 for HTTPS. Port 8080 is also often used for proxy requests.the following URL shows how users might access a site using a different port: http://company.com:7070/mainpage.html In this example, you would need to add port 7070 to the list of monitored ports on the Configure > Monitoring > Protocols page. Contact your network administration to confirm which ports are being used for the applications you want to monitor. Appliance attached to wrong switch The appliance needs to be deployed in a location where it can monitor the network traffic that is being transmitted to and from your applications. he most common deployment location is near the switch that feeds the server farm servicing the monitored applications. To ensure your application is deployed in the correct location follow these steps: 1 Get the network topology from the network administrator. 2 Have the network administrator read Installing and configuring. 3 Verify the appliance is attached to the appropriate switch. Direct connection to VLAN A common misconception when deploying monitoring devices is that the Foglight Experience Monitor can be plugged directly into a VLAN and see all traffic transmitted on the VLAN. This misconception arises from the fact that VLANs transmit broadcast traffic to all ports on the VLAN. Despite this, the VLAN does not duplicate normal point-to-point traffic on every port, thus every port on a VLAN does not see all traffic to other ports. Correct appliance deployment involves the configuration of a network tap to copy packets from one of the locations. This is discussed in Network taps. Mirroring wrong port on switch Many networks will have multiple VLANs (Virtual LANs) configured on the switch. Ideally, the appliance should be monitoring the VLAN that is connected to the Internet. This is usually the connection from the firewall to the switch, or the connection from the load balancer to the switch. A common mistake occurs when the network tap is configured to mirror traffic from the incorrect location on the network. 243
Verify the server configuration The list of servers known by the appliance controls which packets are analyzed. Only packets sent to or from one of these servers are analyzed, and all other packets are discarded. Given this filtering, if the server list is not configured correctly, then the appliance will not show any data in its reports. At this point in the troubleshooting process, the following should all be true: the appliance can connect to the network (access to gateway verified) the network tap has been correctly configured, and the appliance can see web traffic on its monitor port (web traffic visibility verified) a user can successfully point a web browser at the appliance you have completed the Setup Wizard the appliance has been running for more than 10 minutes and no data is appearing on Monitoring Reports IMPORTANT: If all of the above are not true, then not all previous troubleshooting steps have been completed successfully. Restart the troubleshooting process from the beginning of the chapter. The following sections outline possible reasons behind empty reports, and how to resolve them. Virtual IPs Most networks will probably be using some form of a load balancer in conjunction with the server farm. To external users accessing the web site, the load balancer presents one or more Virtual IPs (VIPs). The web site names (DNS) all resolve to these VIPs. However, once a request from an end-user arrives at the load balancer, the request is redirected to a server using that server s physical IP. Assuming, the appliance is deployed between the load-balancer and the server, it is the physical IPs that will be monitored. In this case you will need to configure the physical IPs in the web console, click Configure > Monitoring > Servers to edit entries on the Servers page. For more information, see Managing monitored servers. Incorrect server IPs The appliance will only monitor traffic on the servers specified in the server list. It will filter out all other traffic. Verify the server IP address in question is not missing or mistyped in the server configuration list. In the web console, click Configure > Monitoring > Servers to edit entries on the Servers page. For more information, see Managing monitored servers. Appliance runtime issues For more information, see these topics: Foglight Experience Monitor reports Report sets are not updating Foglight Experience Monitor reports If your users report that they are interacting with an application that is monitored by the appliance, but reports relating to that application do not show any activity, one of several factors may be causing this behavior. 244
Confirming ports have been configured It is possible that the protocol used by the application has not been configured in the appliance. If this is the case, the appliance will not be listening on all the correct ports for traffic. In the web console, click Configure > Monitoring > Protocols, and verify the port used by the application is on this list. If the port is not listed on the Protocols page, and you are not sure which port the application is using, you can have the appliance detect port usage for you. For more information, see Automatically discovering ports. Confirming the web server is monitored It is possible that the server is not actually being monitored by the appliance. In the web console, click Configure > Monitoring > Servers, and verify the server from which the application is delivered is on the servers list. If the server is not on the list, you can add it manually, or have the appliance detect the server for you. For more information, see Automatically discovering servers. Network tap configuration It is possible that the network tap is configured in such a way that the appliance cannot see the application s traffic. To confirm this, use the Verify Monitor NIC tool to verify that the appliance is receiving incoming traffic. This tool is found on the Support Tools page, which is accessed by clicking Help > Support Tools in the main menu. For more information, see Monitor NIC activity test. IMPORTANT: Running a TCP dump will temporarily halt data collection by the appliance, which will result in a gap in your metrics. Use this feature with caution. Report sets are not updating Users may report that report sets are no longer updating with new data. 245
Figure 118. Troubleshooting report sets that are not updating The example above shows that the report sets have not updated in ten days. The cause of this behavior is most likely one of the following: the cable that attaches to the Monitor NIC on the appliance has been unplugged the network tap has been reconfigured and is no longer mirroring traffic the server that the appliance is monitoring has been moved, and its traffic is no longer being mirrored to the network tap Confirming system health The System Health page is a special Monitoring Report that provides a summary of the health of the appliance and its environment. 246
Figure 119. System Health page For more information, see these topics: System performance System events Database performance Error rates System performance This display reports on memory and CPU usage by the appliance over a period of time (default: two hours). Generally, memory usage and CPU usage up to 80% capacity is tolerable. If usage of either resource exceeds 80% of the total capacity, the appliance is overloaded. If this occurs regularly, it is indicative that the appliance work load should be lightened by reducing the number of monitored servers. System events This display indicates whether the appliance agent s performance has degraded enough to generate a systemlevel event. This includes low memory warnings and agent restarts. The two agent-specific reports show how many times the agent s memory use exceeded its allocated share, and how often it had to restart. Agent restarts can happen for various reasons, including the exceeding its allocated share of memory. In most cases, incidents of continuous memory warnings and restarts indicate the appliance is overloaded, and can be remedied by reducing the number of monitored servers. The appliance agent is a component of the appliance, or system as a whole. Memory consumption for the entire system is also reported here, and can help confirm whether other non-agent resources are contributing to high memory usage. 247
Database performance This display reports how long it takes for records to be loaded to and from the database during a standard fiveminute cycle over a period of time (default: two hours). Database loading occurs whenever incoming traffic data is written, or existing data is called through report generation in the web console. It is not unusual for the displayed load time to exceed thirty seconds; however, if this occurs consistently, it is indicative that preventative or corrective measures should be taken. The latter is accomplished by making better use of filters when searching through resources. Preventative measures include reducing the number of monitored servers, or creating URL transformations that result in fewer processed URLs. Error rates This display summarizes all transmission-related errors over a period of time (default: two hours). Rates are reported for: dropped packets missing segments in client-to-server or server-to-client traffic SSL connection errors SOAP errors (for example, unrecognizable SOAP messages) All of these reported rates should be zero, or at a low percentage. High rates occur for different reasons, depending on the type of error. For example, dropped packets or missing segments are signs that the appliance may be overloaded. Meanwhile, SSL errors can happen for a variety of reasons, including incorrectly configured SSL keys, security-related communication conflicts, or errors from dropped packets or missing segments. Using the appliance support tools The Support Tools page provides tools to help you investigate and troubleshoot any issues you may be having with the appliance. Many of the support tools would normally be used with the assistance of a Dell technical support representative. Figure 120. Support Tools page For more information, see these topics: Install log Creating a support bundle Monitor NIC activity test TCP dump 248
Install log The install log provides details about which version of the appliance, patch and upgrade that were installed on the system. These may be useful to Customer Support representatives to help solve a problem. Figure 121. Support Tools: Install Log page Creating a support bundle When working towards a resolution with Dell Support, there may be occasions where you are requested to send a support engineer a support bundle that will help to diagnose the issue. Click the Support Bundle link to generate a file containing the support bundle that you can save to your local hard drive. This file can then be emailed to a Dell technical support representative. Monitor NIC activity test The monitor NIC, when attached to your network tap, monitors designated HTTP and HTTPS traffic on your network. You can use the Verify Monitor NIC utility to confirm that this network traffic is visible to the appliance. When clicked, the Verify Monitor NIC window displays, asking whether you want to have the appliance listen for network traffic using the ports that were originally configured, or using all ports. Remember that you first configured which ports were used to monitor HTTP/HTTPS traffic in Configuring protocols. Also, see Managing protocols on a Portal for more information on managing port settings. After selecting the ports and clicking Go, the Verify Monitor NIC window displays network traffic messages, confirming whether the Monitor NIC is detecting network traffic. IMPORTANT: The packets dropped message at the bottom of this message window should not be considered significant. The Verify Monitor NIC test runs at a much lower priority than the internal the appliance system agent. Consequently, some packet drops are to be expected when reported here. Actual packet drop rates can be determined through examining the system logs. Contact Dell technical support for information on how to do this. 249
Figure 122. Verify Monitor NIC window TCP dump When working towards a resolution with Dell Support, there may be occasions where you are requested to send a support engineer a TCP file dump. Clicking TCP Dump Utilities allows you to create a file containing a TCP dump representing a specified amount of time. Figure 123. TCP Dump Utilities window The TCP Dump Utilities dialog box allows you to specify the amount of time the tcpdump collection process will run. Generally, you will want to limit the amount of time in order to keep the tcpdump file from growing to an extremely large size. The by device option provides a list from which you can specify the monitoring NIC that should used for the tcpdump capture. This choice gathers traffic for all ports that are currently configured on the Configure > Monitoring > Protocols page. If you need more control over the traffic that will be captured, select the by filter option and specify a filter that tcpdump process will use. An example of a filter is host 192.168.1.10 and port 80. This captures all traffic going to and from server 192.168.1.10 over port 80. The filter string you type is appended to the following default settings: -i any -s 1518 -w outputfile. Using this filter string means that you cannot specify a particular monitoring port (such as eth2) with this option. TIP: You can see any Linux manual for a description of the parameters that can be passed to a tcpdump. Once complete, you should contact your Dell Technical Support representative to determine the best way to transmit this file. TCP dumps can be quite large and generally cannot be sent using email. IMPORTANT: A TCP dump file could contain confidential information depending on what type of application you are monitoring. Make sure you are following your organization s security policies and use appropriate safeguards when transferring TCP dump files to Dell technical support. In some cases, your security policies may not allow you to transfer such data. 250
A Appendix: Third party software This section provides details about third party software that enables you to configure and use the Foglight Experience Monitor more effectively. For more information, see Using X-Forwarded-For. Using X-Forwarded-For The X-Forwarded-For (XFF) HTTP header is a method for identifying the originating IP address of a client connecting to a web server through an HTTP proxy. Without the use of XFF or another similar technique, any connection through the proxy reveals only the originating IP address of the proxy server. This makes the detection and prevention of abusive accesses significantly harder than if the originating IP address was available. You can use a product such as Big-IP to translate the source IP address of the incoming packet to the original client IP address or what is referred to as a Secure Network Address Translation (SNAT). A SNAT provides a secure mechanism for translating internal, non-routable addresses into routable addresses. When the BIG-IP system translates the source IP address of the incoming packet to the SNAT address, the web server sees the request as originating from the SNAT address, not the original client IP address. NOTE: If the web servers are required to log the original client IP address for requests, the SNAT address translation behavior can become problematic. To configure the BIG-IP system to insert the original client IP address in an X-Forwarded-For HTTP header, you can use one of the following methods: Enable Insert XForwardedFor in the HTTP profile irule For more information, see these topics: Enabling the insert X-Forwarded-For in the HTTP profile irule Configuring the web server to extract the IP address from the HTTP header Enabling the insert X-Forwarded-For in the HTTP profile To configure the BIG-IP system to insert the original client IP address in an X-Forwarded-For HTTP header, perform the following procedure: 1 Log in to the BIG-IP Configuration utility. 2 Click Local Traffic. 3 Click Profiles. 4 Click HTTP from the Services list. 251
5 Click Create. 6 Type a name for the HTTP profile. 7 Select the Insert XForwarded For check box. A list appears. 8 Select Enabled from the list. 9 Click Finished. You must now associate the new HTTP profile with the virtual server. irule To configure the BIG-IP system to insert the original client IP address in an X-Forwarded-For HTTP header using an irule, perform the following procedure: 1 Log in to the BIG-IP Configuration utility. 2 Click Local Traffic. 3 Click irules. 4 Click the Create. 5 Type a name in the Name field. 6 Copy and paste the following irule in the Definition field: when HTTP_REQUEST { HTTP::header insert X-Forwarded-For [IP::remote_addr] } 7 Click the Finished. You must now associate the new irule with the virtual server. Configuring the web server to extract the IP address from the HTTP header Once you have configured the BIG-IP system to insert the original client IP address in an HTTP header using an X- Forwarded-For HTTP header, you must also configure the web server to extract the IP address from the HTTP header, and log the IP address to the web server log file. IMPORTANT: For specific details about using HTTP header information in log files, see the documentation provided by the vendor for your specific web server. Apache web server You can configure an Apache web server to extract the IP address from the X-Forwarded-For HTTP header and log the IP address to the web server log file by adding the appropriate logging directives to the Apache httpd.conf file. For example: LogFormat "%v %{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" X-Forwarded-For CustomLog /var/log/apache/www.example.com-xforwarded.log X-Forwarded-For For more information about Apache logging, see the Apache documentation. 252
Microsoft IIS web server You can configure the Microsoft IIS web server to extract the IP address from the X-Forwarded-For HTTP header and log the IP address to the web server log file. To do so, you will need to download and install the IIS X-Forwarded-For ISAPI Log Filter from http://devcentral.f5.com. The IIS ISAPI filter will look for the X-Forwarded-For HTTP header in the HTTP request. If the IIS ISAPI filter finds an X-Forwarded-For HTTP header in the HTTP request, it will replace the client IP address in the W3SVC log traces with the value of the X-Forwarded-For HTTP header. 253
Appendix: Dell PowerEdge system appliance B This section provides details about Dell PowerEdge System appliances that can be used with the Foglight Experience Monitor. IMPORTANT: The Dell PowerEdge 2950 is no longer supported for Foglight Experience Monitor. For more information, see these topics: Dell PowerEdge R300 and R310 Dell PowerEdge R610 Dell PowerEdge R710 Dell PowerEdge R720 Updating the Dell system firmware on the appliance Dell PowerEdge R300 and R310 IMPORTANT: The Dell PowerEdge R300 and R310 are no longer being issued with new versions of the Foglight Experience Monitor. The following information is for existing customers who have these appliances implemented as part of their monitoring solution. For Dell PowerEdge R300 and R310 series, the appliance back panel is similar to the following diagram. The two monitoring ports are located on the dual-port add-in NIC, and are configured for use with either fiber-based networks or copper-based networks (copper NIC ports shown in the following diagram). NOTE: Additional monitoring ports can be added to your default configuration, if necessary. Foglight Experience Monitor currently supports up to eight monitoring NIC ports. 254
Figure 124. Dell PowerEdge R300/ R310 back panel 1 Monitoring NICs (eth2, eth3) 2 Power Supply 3 System Identification Button 4 System Status Indicator 5 System Status Indicator Connector 6 Auxiliary Port (eth1) 7 Control Port (eth0) 8 2.0-Compliant USB Connectors (2) 9 Video Connector 10 Serial Connector Dell PowerEdge R610 IMPORTANT: The Dell PowerEdge R610 is no longer being issued with new versions of the Foglight Experience Monitor. The following information is for existing customers who have this appliance implemented as part of their monitoring solution. For Dell PowerEdge R610 series, the appliance back panel is similar to the following diagram. The four monitoring ports are located on the dual-port add-in NIC, and are configured for use with either fiber-based networks or copper-based networks (copper NIC ports shown in the following diagram). NOTE: Additional monitoring ports can be added to your default configuration, if necessary. Foglight Experience Monitor currently supports up to eight monitoring NIC ports. Figure 125. Dell PowerEdge R610 back panel 1 Dell Remote Access Controller 2 Serial Port 3 Monitoring NICs (eth4) 4 Monitoring NICs (eth5) 5 Video Port 6 USB Port 7 Control Port (eth0) 8 Auxiliary Port (eth1) 9 Monitoring NICs (eth2) 10 Monitoring NICs (eth3) 11 System Status Indicator 12 System Status Indicator Connector 13 System Identification Button 14 Power Supply 15 Power Supply 255
Dell PowerEdge R710 IMPORTANT: The Dell PowerEdge R710 is no longer being issued with new versions of the Foglight Experience Monitor. The following information is for existing customers who have this appliance implemented as part of their monitoring solution. For Dell PowerEdge R710 series, the appliance back panel is similar to the following diagram. The four monitoring ports are located on the dual-port add-in NIC, and are configured for use with either fiber-based networks or copper-based networks (copper NIC ports shown in the following diagram). NOTE: Additional monitoring ports can be added to your default configuration, if necessary. Foglight Experience Monitor currently supports up to eight monitoring NIC ports. Figure 126. Dell PowerEdge R710 back panel 1 Dell Remote Access Controller Port 2 Serial Port 3 Video Port 4 USB Ports 5 Control Port (eth0) 6 Auxiliary Port (eth1) 7 Monitoring NICs (eth2) 8 Monitoring NICs (eth3) 9 System Status Indicator 10 System Status Indicator Connector 11 System Identification Button 12 Power Supply 13 Redundant Power Supply 14 Monitoring NICs (eth4) 15 Monitoring NICs (eth5) Dell PowerEdge R720 For Dell PowerEdge R720 series, the appliance back panel is similar to the following diagram. The six monitoring ports are located on the dual-port add-in NIC, and are configured for use with either fiber-based networks or copper-based networks (copper NIC ports shown in the following diagram). NOTE: Additional monitoring ports can be added to your default configuration, if necessary. Foglight Experience Monitor currently supports up to eight monitoring NIC ports. Figure 127. Dell PowerEdge R720 back panel 256
1 System Status Indicator 2 System Identification Button 3 System Status Indicator Connector 4 Dell Remote Access Controller Port 5 Serial Port 6 Video Port 7 USB Ports 8 Control Port (eth0) 9 Auxiliary Port (eth1) 10 Monitoring NICs (eth2) 11 Monitoring NICs (eth3) 12 Power Supply 13 Redundant Power Supply 14 Monitoring NICs (eth4) 15 Monitoring NICs (eth5) 16 Monitoring NICs (eth6) 17 Monitoring NICs (eth7) Updating the Dell system firmware on the appliance You can update the firmware on your Dell appliance by downloading ISO images from the Dell Support web site. There are two ISO images that you need to download, Dell System Management Tools and Documentation and Dell Server Updates images. After you have downloaded the images to your system, you can create DVDs that are used to update the appliance s firmware. IMPORTANT: The Dell Support web site (www.dell.com) is maintained regularly. Updates to the site may differ from the steps in the following procedure, however the process is similar. For more information, see these topics: Accessing the drivers and downloads Downloading the ISO images Updating the firmware on the appliance Accessing the drivers and downloads To access the drivers and downloads: 1 Log in to the Dell web site at www.dell.com. 2 Under Support, click the Drivers & Downloads link. 3 From the Drivers & Downloads page, select the Choose a Model option. 257
4 Select your product from the list (for example, for a Dell R710 appliance, click Servers, Storage, Networking > PowerEdge Server > R710), then click Confirm. The information for your appliance displays. 5 Expand the System Management section. 6 Navigate to the Dell System Management Tools and Documentation and Dell Server Updates iso image section. Downloading the ISO images To download the ISO images: 1 In the Dell System Management Tools and Documentation section, click Download to download the file to your system. 258
2 In the Dell Server Updates section, click Download to download the file to your system. IMPORTANT: Files that are larger than 2 GB are separated into smaller download files. Once downloaded, you need to combine them on your machine before burning them to a DVD. For example, the Dell Server Updates ISO image consists of three files on the Dell support site: OM_6.2.0_SUU_A01.iso.001, OM_6.2.0_SUU_A01.iso.002, and OM_6.2.0_SUU_A01.iso.003. Download these files and combine them into a single.iso file using the following command: copy /b OM* OM_620_SUU_A01.iso 3 Using your DVD burning software, create a DVD that contains the ISO images, one each for Dell System Management Tools and Documentation and Dell Server Updates. 4 To create the DVD, use the burn from disc image functionality on your DVD burning software. Updating the firmware on the appliance You can update any of the Dell PowerEdge appliances, R300, R310, R610, or R710 using the DVDs that were created in the previous procedure. It is not necessary to create separate DVDs for each appliance type. To update the firmware on the appliance: 1 Insert the Dell System Management and Documentation DVD into the appliance DVD tray. For more information about creating DVD images, see Downloading the ISO images. 2 Log in to the appliance as the Setup user. 3 To reboot the appliance, select the Appliance Shutdown/Reboot option then click Yes to confirm the operation. The system boots from the System Management and Documentation DVD. The boot sequence can take several minutes to complete. When the boot sequence is complete, the Dell menu appears. 4 Select the Dell Systems Build and Update Utility option. 5 Select the Configure action link for the Firmware Update option. 6 Specify the update location by selecting the Update from CD/DVD option. 7 Click Continue. The system ejects the System Management and Documentation DVD and prompts you for the disk containing the update repository. 8 Insert the Dell Server Updates DVD and click OK. For more information about creating DVD images, see Downloading the ISO images. 9 Click Continue. The Build and Update Utility scans the appliance and compares it to the updates available on the Server Updates DVD. 10 Click OK to update all the firmware on the system. Apply all the updates that are available. The updating can take several minutes to complete. After the update process finishes, the system reboots. Do not interrupt this process once it has started. TIP: The system fan speed may increase during the update; this is normal. 11 Remove the Server Updates DVD from the DVD tray. 259
About Dell Dell listens to customers and delivers worldwide innovative technology, business solutions and services they trust and value. For more information, visit www.software.dell.com. Contacting Dell Technical support: Online support Product questions and sales: (800) 306-9329 Email: info@software.dell.com Technical support resources Technical support is available to customers who have purchased Dell software with a valid maintenance contract and to customers who have trial versions. To access the Support Portal, go to https://support.software.dell.com/. The Support Portal provides self-help tools you can use to solve problems quickly and independently, 24 hours a day, 365 days a year. In addition, the portal provides direct access to product support engineers through an online Service Request system. The site enables you to: Create, update, and manage Service Requests (cases) View Knowledge Base articles Obtain product notifications Download software. For trial software, go to Trial Downloads. View how-to videos Engage in community discussions Chat with a support engineer 260