Justin Bruns, Performance Test Lead, Fireman's Fund Insurance Robert May, Enterprise Software Tools Administrator Fireman's Fund Insurance Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP Tools for Software Performance Testing and Analysis Justin Bruns, Performance Test Lead - Fireman s Fund Insurance Company June 2012 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Agenda Fireman s Fund Overview Problem Statement Solution (Sitescope, Performance Center, Diagnostics) Performance Analysis Methodology Test Execution and Monitoring, Analysis File Review, Exploratory Testing Additional Tools for Analysis Q & A 3
Fireman s Fund Overview Company profile: Fireman's Fund Insurance Company is a premier property and casualty insurer, providing personal and commercial insurance products nationwide. The company is differentiated from its competition through its product and service excellence, and exceptional claims handling. Based in: Novato, California Employees: 3,900 Company history: Founded in San Francisco in 1863, Fireman's Fund was named for its founding social mission: contributing a portion of its profits to widows and orphans of firefighters. Today the company continues its philanthropic mission, supporting firefighters for safer communities through its Fireman's Fund Heritage SM program. Parent company: Allianz SE of Munich, Germany is one of the world's largest providers of insurance and other financial services with more than $1 trillion in assets. It is rated A+ by A.M. Best. 4
Problem Statement In addition to performance test response times, customers want answers. Performance bottlenecks sometimes difficult to isolate and correct Challenges Encountered Multiple causes of performance slowdowns Insufficient communication between performance testing and development teams Disparate monitoring tools and techniques 5
Solution Establish integrated and comprehensive monitors HP Sitescope for Load Testing 11.10 HP ALM Performance Center 11.00 HP Diagnostics Server 9.12 HP LoadRunner Analysis Tool 11.03 HP Virtual User Generator 11.03 Gather and analyze results Performance Analysis Methodology Inspect information in real-time or after completing tests Make recommendations and necessary changes revalidate results 6
Application Architecture
Sitescope Monitors 8
Sitescope Monitors Wintel performance counters (CPU, memory, disk, network) AIX performance counters (CPU, memory, disk, network) AIX process level details (CPU, memory size, threads) WebSphere MQ counters Queue details (current depth, open input count, open output count) Channel details (status, bytes sent/received) IIS counters High level (bytes/min, hits/min) Web service counters (current connections, method requests/sec).net CLR (SQL client connection Data, garbage collection statistics) SQL server counters Average buffer chain length, % physical I/O used by process CPU, memory, disk Custom solution templates JBoss, Oracle, Linux, WebLogic 9
Sitescope - Performance Center Link sitescope monitors in Performance Center Performance Center test resources section Add a Sitescope monitor profile Enter the appropriate Sitescope information Account details not the same as login details 10
Sitescope - Performance Center Enter Server and Port Details Click on Get Counters, Monitors Should Appear Select the Appropriate Sitescope Monitors Repeat this Process for all Desired Sitescope Monitors 11
Sitescope - Performance Center Add the monitors in Performance Center Performance test designer window Select desired monitor(s) from profile tree 12
Performance Center Monitors Monitors may be added directly in Performance Center 13
Performance Center Monitors Multiple monitor types Windows Resources, UNIX Resources, SNMP, IIS, SQL Server, Oracle 14
Diagnostics Monitors Types of Monitors.NET Probe J2EE Probe SQL Server Collector Oracle Collector WebSphere MQ Collector Steps Involved Install Probe or Collector Software Configure Probe or Collector Instrument Probe 15
Diagnostics Monitors.NET Probe Standard View (CPU, VM Heap Utilization, Latency, Throughput) 16
.NET Probe View Server Request Details Diagnostics Monitors 17
Diagnostics Monitors.NET Probe View Server Request Details 18
Diagnostics Monitors WebSphere MQ collector Queue information (depth, messages enqueued/dequeued per second) 19
Diagnostics Monitors SQL Server and Oracle collectors View SQL statements, latency, throughput 20
Diagnostics - Performance Center In Performance Center s test designer view Diagnostics tab Diagnostics can be enabled for desired % of Vusers Configure J2EE/.NET application diagnostics 21
Diagnostics - Performance Center Select the Desired Probes or Collectors 22
Analysis Methodology Start at the web layer Look deeply into front end application Examine the database layer Know the integration layer Determine the type of problem 23
Test Execution and Monitoring - Analysis File Review - Execute the monitored load test Upon completion, collate and analyze the results Using the LoadRunner analysis tool, we can View transaction response time results View other Performance Center measurements (hits per second, throughput, etc.) View sitescope monitoring results View J2EE/.NET diagnostic information Overlay graphs and correlate information Identify trends and hotspots 24
Test Execution and Monitoring - Analysis File Review - View transaction response times to identify potential problems 25
Test Execution and Monitoring - Analysis File Review - Show J2EE/.NET chain of calls 26
Test Execution and Monitoring - Analysis File Review - Overlay graphs to correlate information 27
Test Execution and Monitoring - Analysis File Review - Overlay graphs to correlate information 28
Test Execution and Monitoring - Exploratory Testing - Profile single user transactions 29
Additional Tools for Analysis HttpWatch / Fiddler / Firebug Inspect browser HTTP traffic Database Monitoring Reports SQL Server Profiler Oracle AWR Report JVM Inspection Garbage collection logs Thread dumps Heap dump analyzer 30
Q & A 31
Contact Details Justin Bruns, Performance Test Lead Fireman s Fund Insurance Company Work email jbruns@ffic.com Home email jbruns1@yahoo.com 32