Justin Bruns, Performance Test Lead, Fireman's Fund Insurance Robert May, Enterprise Software Tools Administrator Fireman's Fund Insurance

Transcription

1 Justin Bruns, Performance Test Lead, Fireman's Fund Insurance Robert May, Enterprise Software Tools Administrator Fireman's Fund Insurance Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

2 HP Tools for Software Performance Testing and Analysis Justin Bruns, Performance Test Lead - Fireman s Fund Insurance Company June 2012 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

3 Agenda Fireman s Fund Overview Problem Statement Solution (Sitescope, Performance Center, Diagnostics) Performance Analysis Methodology Test Execution and Monitoring, Analysis File Review, Exploratory Testing Additional Tools for Analysis Q & A 3

4 Fireman s Fund Overview Company profile: Fireman's Fund Insurance Company is a premier property and casualty insurer, providing personal and commercial insurance products nationwide. The company is differentiated from its competition through its product and service excellence, and exceptional claims handling. Based in: Novato, California Employees: 3,900 Company history: Founded in San Francisco in 1863, Fireman's Fund was named for its founding social mission: contributing a portion of its profits to widows and orphans of firefighters. Today the company continues its philanthropic mission, supporting firefighters for safer communities through its Fireman's Fund Heritage SM program. Parent company: Allianz SE of Munich, Germany is one of the world's largest providers of insurance and other financial services with more than $1 trillion in assets. It is rated A+ by A.M. Best. 4

5 Problem Statement In addition to performance test response times, customers want answers. Performance bottlenecks sometimes difficult to isolate and correct Challenges Encountered Multiple causes of performance slowdowns Insufficient communication between performance testing and development teams Disparate monitoring tools and techniques 5

6 Solution Establish integrated and comprehensive monitors HP Sitescope for Load Testing HP ALM Performance Center HP Diagnostics Server 9.12 HP LoadRunner Analysis Tool HP Virtual User Generator Gather and analyze results Performance Analysis Methodology Inspect information in real-time or after completing tests Make recommendations and necessary changes revalidate results 6

7 Application Architecture

8 Sitescope Monitors 8

9 Sitescope Monitors Wintel performance counters (CPU, memory, disk, network) AIX performance counters (CPU, memory, disk, network) AIX process level details (CPU, memory size, threads) WebSphere MQ counters Queue details (current depth, open input count, open output count) Channel details (status, bytes sent/received) IIS counters High level (bytes/min, hits/min) Web service counters (current connections, method requests/sec).net CLR (SQL client connection Data, garbage collection statistics) SQL server counters Average buffer chain length, % physical I/O used by process CPU, memory, disk Custom solution templates JBoss, Oracle, Linux, WebLogic 9

10 Sitescope - Performance Center Link sitescope monitors in Performance Center Performance Center test resources section Add a Sitescope monitor profile Enter the appropriate Sitescope information Account details not the same as login details 10

11 Sitescope - Performance Center Enter Server and Port Details Click on Get Counters, Monitors Should Appear Select the Appropriate Sitescope Monitors Repeat this Process for all Desired Sitescope Monitors 11

12 Sitescope - Performance Center Add the monitors in Performance Center Performance test designer window Select desired monitor(s) from profile tree 12

13 Performance Center Monitors Monitors may be added directly in Performance Center 13

14 Performance Center Monitors Multiple monitor types Windows Resources, UNIX Resources, SNMP, IIS, SQL Server, Oracle 14

15 Diagnostics Monitors Types of Monitors.NET Probe J2EE Probe SQL Server Collector Oracle Collector WebSphere MQ Collector Steps Involved Install Probe or Collector Software Configure Probe or Collector Instrument Probe 15

16 Diagnostics Monitors.NET Probe Standard View (CPU, VM Heap Utilization, Latency, Throughput) 16

17 .NET Probe View Server Request Details Diagnostics Monitors 17

18 Diagnostics Monitors.NET Probe View Server Request Details 18

19 Diagnostics Monitors WebSphere MQ collector Queue information (depth, messages enqueued/dequeued per second) 19

20 Diagnostics Monitors SQL Server and Oracle collectors View SQL statements, latency, throughput 20

21 Diagnostics - Performance Center In Performance Center s test designer view Diagnostics tab Diagnostics can be enabled for desired % of Vusers Configure J2EE/.NET application diagnostics 21

22 Diagnostics - Performance Center Select the Desired Probes or Collectors 22

23 Analysis Methodology Start at the web layer Look deeply into front end application Examine the database layer Know the integration layer Determine the type of problem 23

24 Test Execution and Monitoring - Analysis File Review - Execute the monitored load test Upon completion, collate and analyze the results Using the LoadRunner analysis tool, we can View transaction response time results View other Performance Center measurements (hits per second, throughput, etc.) View sitescope monitoring results View J2EE/.NET diagnostic information Overlay graphs and correlate information Identify trends and hotspots 24

25 Test Execution and Monitoring - Analysis File Review - View transaction response times to identify potential problems 25

26 Test Execution and Monitoring - Analysis File Review - Show J2EE/.NET chain of calls 26

27 Test Execution and Monitoring - Analysis File Review - Overlay graphs to correlate information 27

28 Test Execution and Monitoring - Analysis File Review - Overlay graphs to correlate information 28

29 Test Execution and Monitoring - Exploratory Testing - Profile single user transactions 29

30 Additional Tools for Analysis HttpWatch / Fiddler / Firebug Inspect browser HTTP traffic Database Monitoring Reports SQL Server Profiler Oracle AWR Report JVM Inspection Garbage collection logs Thread dumps Heap dump analyzer 30

31 Q & A 31

32 Contact Details Justin Bruns, Performance Test Lead Fireman s Fund Insurance Company Work jbruns@ffic.com Home jbruns1@yahoo.com 32