Performance Management from black-art to process

Size: px

Start display at page:

Download "Performance Management from black-art to process"

Luke Adams
10 years ago
Views:

1 Performance Management from black-art to process

2 What needs to be measured? CPU utilisation I/O Rate TPM level transaction stats

3 What needs to be measured???? Remember how much fun it was (is?) managing desktops? CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats

4 What needs to be measured? End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats

5 And that s just the infrastructure App 1 App n App 1 App 2 App n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats

6 And that s just the infrastructure App 1 App n App 1 App 2 App n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats

7 And that s just the infrastructure Svc 1.. App 1 App n App 1 App n Svc n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats

8 Diff rent strokes for diff rent folks Development Test Production Developers Test Centers Production, Staging Performance Engineering (Arch Validation, Profiling) Workstation Cruise Control, Apache Continuum, Quickbuild, Anthill, Bamboo, VSTS Continuous Integration & Automated Testing Load Testing Tuning, Diagnostics 24x7 Transaction Tracing, Monitoring, Diagnostics IDEs: Eclipse Visual Studio CI Build/Test Systems: Ant, NAnt MSBuild Web Srvcs SilkPerformer MS VSTS itko SOA Test Neotys Proxysniffer JMeter Push2Test WebLoad Perfmon (Windows), JMX, PMI (IBM WebSphere) and OS Performance

Load Testing Tuning, Diagnostics 24x7 Transaction Tracing, Monitoring, Diagnostics IDEs: Eclipse Visual Studio CI Build/Test Systems: Ant, NAnt

9 What s needed to diagnose application issues? Svc 1.. App 1 App n App 1 App n Svc n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats

10 dynatrace s Core Building Blocks Patented PurePath TM Technology

11 Traditional Development & Test Approach Development Test Production Software Arch. Perf. Analyst ` Oliver Operations Developer Test Sam Sys. Architect Develop locally Dev AppServer Continuous check outs, builds & functional unit tests Buildserver Int AppServer Functional Load testing & load tests Test AppServer Test AppServer Test AppServer

Architect Develop locally Dev AppServer Continuous check outs, builds & functional

12 The Project plan Delivery Date Effort ( (Mandays Mandays) Development Testing Production

13 But Delivery Date Effort ( (Mandays Mandays) Development Testing Production

14 But Delivery Date Effort ( (Mandays Mandays) IMPACT Development Testing Production

15 But Problem Resolution Effort ( (Mandays Mandays) IMPACT Delivery Delayed Development Testing Production

16 What happened?! Development Test Production Software Arch % Dev time Lost Perf. Analyst ` Oliver Operations Developer Test Sam Sys. Architect Limited opportunity for performance testing in development Difficult to reproduce error cases Many problems found in load tests could be detected earlier Testers identify the symptoms, not the cause Tedious and error prone process of documenting faults Communication of issues involves many round-trips Time and resource constraints are paramount Several testing iterations are required to solve problems identified

could be detected earlier Testers identify the symptoms, not the cause Tedious and error prone process of documenting faults Communication

17 Continuous APM through the lifecycle Development Test Production Software Arch. Perf. Analyst ` Oliver Operations Developer Test Sam Sys. Architect Early Performance Feedback Develop locally Test + Architecture Validation Dev App-Server Continuous check outs, builds & unit tests + Continuous Perf. Regression Functional & load tests + Continuous Perf. Regression Perf. Analyst Test App-Servers Buildserver Int App-Server Test Sam Sys. Architect Dev WorkStation dynatrace Dev Server dynatrace Integration Server dynatrace Testserver

Architect Early Performance Feedback Develop locally Test + Architecture Validation Dev App-Server Continuous check outs, builds &

18 Continuous APM in development Develop locally Test + Architecture Validation Dev App- Server Perf. Analyst Test Sam Sys. Architect Dev WorkStation dynatrace Dev Server

19 dynatrace in test Scenario: Take order Step: Product selection Load test harness Application dynatrace server Correlation with LoadRunner, SilkPerformer, Neoload, Facilita, JMeter etc. Functional & load tests + Continuous Perf. Regression Test App-Servers dynatrace Testserver

LoadRunner, SilkPerformer, Neoload, Facilita, JMeter etc.

20 dynatrace in continuous integration CI Server Application dynatrace server Continuous check outs, builds & unit tests + Continuous Perf. Regression Buildserver Int App-Server dynatrace Integration Server

21 Continuous APM avoids the Iceberg Effort ( (Mandays Mandays) Delivery on spec, on time Development with integrated APM Testing Production

22 Company Leading provider of software solutions for healthcare insurance industry Over 200 institutional clients managing over 30 million individual policy holders Handling 250 million EDI flows per year Client in-house deployment (packaged solution) as well as hosted SaaS models Why dynatrace? End-to-end transaction monitoring across multiple JVMs Fine granularity: Apache CXF, RMI, JDBC access / Hibernate Sharing PurePath information between teams and lifecycle integration Easy to deploy, configure and use in continuous integration APM Challenges Optimise application performance across 4-tier architecture Reduce time to detect application performance bottlenecks Improve issue tracking across all stakeholders to fast track resolution Results As performance testing becomes automated, continuous integration accelerates application feature development Greater and better application behaviour visibility and understanding improve software quality Halved MTTR root cause shared with development immediately to provide solution

23 Continuous integration at Cegedim Shared Development Server and local workstations Mainly used by the Architects and senior developers Development server dynatrace development server Processor : Intel Core2Duo or Xeon 3.16 GHz RAM : 8 GB Disk : 160 GB minimum (10k RPM) or SAN dynatrace Repository dynatrace Collectors License activation License activation

24 Example: CI with dynatrace at a professional networking site 1. Schedule test runs from Hudson, start selenium ruby scripts 2. Start session recording on dynatrace 3. Start execution scripts against Web Browsers 4. Browser sessions create load on backend 5. Measurements from backend apps are captured 6. Stop session recording 7. Retrieve performance metrics from dynatrace thru REST API as XML 8. Process XML output and generate HTML report 9. Access performance reports from Hudson

25 dynatrace on dynatrace at dynatrace... Continuous Performance Results for different test use cases 25

26 dynatrace in Action Novell: Redefining the Value of the Test Centre? instantly Tester Developers Test Run Reproduction Refine Capturing Re Run Tests Reproduction Refine Capturing 5-6 iterations Re Run Tests Reproduction Problem Analysis Results Problem Solving time Increase test throughput by 2-3x Deliver root-causeresults to dev Tester Test Run asynchronously Reproduction Developers Refine Capturing Raise productivity by reducing test cycles and async diagnostics Re Run Tests Reproduction Refine Capturing 1-2 iterations Re Run Tests Reproduction Problem Diagnostics Problem Solving time

The Evolution of Load Testing. Why Gomez 360 o Web Load Testing Is a

The Evolution of Load Testing. Why Gomez 360 o Web Load Testing Is a Technical White Paper: WEb Load Testing To perform as intended, today s mission-critical applications rely on highly available, stable and trusted software services. Load testing ensures that those criteria