Performance Management from black-art to process peter.holditch@dynatrace.com
What needs to be measured? CPU utilisation I/O Rate TPM level transaction stats
What needs to be measured???? Remember how much fun it was (is?) managing desktops? CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats
What needs to be measured? End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats
And that s just the infrastructure App 1 App n App 1 App 2 App n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats
And that s just the infrastructure App 1 App n App 1 App 2 App n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats
And that s just the infrastructure Svc 1.. App 1 App n App 1 App n Svc n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats
Diff rent strokes for diff rent folks Development Test Production Developers Test Centers Production, Staging Performance Engineering (Arch Validation, Profiling) Workstation Cruise Control, Apache Continuum, Quickbuild, Anthill, Bamboo, VSTS Continuous Integration & Automated Testing Load Testing Tuning, Diagnostics 24x7 Transaction Tracing, Monitoring, Diagnostics IDEs: Eclipse Visual Studio CI Build/Test Systems: Ant, NAnt MSBuild Web Srvcs SilkPerformer MS VSTS itko SOA Test Neotys Proxysniffer JMeter Push2Test WebLoad Perfmon (Windows), JMX, PMI (IBM WebSphere) and OS Performance
What s needed to diagnose application issues? Svc 1.. App 1 App n App 1 App n Svc n End user experience monitoring CPU utilisations I/O Rates AppServer stats Virtualisation stats CPU utilisation I/O Rate TPM level transaction stats
dynatrace s Core Building Blocks Patented PurePath TM Technology
Traditional Development & Test Approach Development Test Production Software Arch. Perf. Analyst ` Oliver Operations Developer Test Sam Sys. Architect Develop locally Dev AppServer Continuous check outs, builds & functional unit tests Buildserver Int AppServer Functional Load testing & load tests Test AppServer Test AppServer Test AppServer
The Project plan Delivery Date Effort ( (Mandays Mandays) Development Testing Production
But Delivery Date Effort ( (Mandays Mandays) Development Testing Production
But Delivery Date Effort ( (Mandays Mandays) IMPACT Development Testing Production
But Problem Resolution Effort ( (Mandays Mandays) IMPACT Delivery Delayed Development Testing Production
What happened?! Development Test Production Software Arch. 10-15% Dev time Lost Perf. Analyst ` Oliver Operations Developer Test Sam Sys. Architect Limited opportunity for performance testing in development Difficult to reproduce error cases Many problems found in load tests could be detected earlier Testers identify the symptoms, not the cause Tedious and error prone process of documenting faults Communication of issues involves many round-trips Time and resource constraints are paramount Several testing iterations are required to solve problems identified
Continuous APM through the lifecycle Development Test Production Software Arch. Perf. Analyst ` Oliver Operations Developer Test Sam Sys. Architect Early Performance Feedback Develop locally Test + Architecture Validation Dev App-Server Continuous check outs, builds & unit tests + Continuous Perf. Regression Functional & load tests + Continuous Perf. Regression Perf. Analyst Test App-Servers Buildserver Int App-Server Test Sam Sys. Architect Dev WorkStation dynatrace Dev Server dynatrace Integration Server dynatrace Testserver
Continuous APM in development Develop locally Test + Architecture Validation Dev App- Server Perf. Analyst Test Sam Sys. Architect Dev WorkStation dynatrace Dev Server
dynatrace in test Scenario: Take order Step: Product selection Load test harness Application dynatrace server Correlation with LoadRunner, SilkPerformer, Neoload, Facilita, JMeter etc. Functional & load tests + Continuous Perf. Regression Test App-Servers dynatrace Testserver
dynatrace in continuous integration CI Server Application dynatrace server Continuous check outs, builds & unit tests + Continuous Perf. Regression Buildserver Int App-Server dynatrace Integration Server
Continuous APM avoids the Iceberg Effort ( (Mandays Mandays) Delivery on spec, on time Development with integrated APM Testing Production
Company Leading provider of software solutions for healthcare insurance industry Over 200 institutional clients managing over 30 million individual policy holders Handling 250 million EDI flows per year Client in-house deployment (packaged solution) as well as hosted SaaS models Why dynatrace? End-to-end transaction monitoring across multiple JVMs Fine granularity: Apache CXF, RMI, JDBC access / Hibernate Sharing PurePath information between teams and lifecycle integration Easy to deploy, configure and use in continuous integration APM Challenges Optimise application performance across 4-tier architecture Reduce time to detect application performance bottlenecks Improve issue tracking across all stakeholders to fast track resolution Results As performance testing becomes automated, continuous integration accelerates application feature development Greater and better application behaviour visibility and understanding improve software quality Halved MTTR root cause shared with development immediately to provide solution
Continuous integration at Cegedim Shared Development Server and local workstations Mainly used by the Architects and senior developers Development server dynatrace development server Processor : Intel Core2Duo or Xeon 3.16 GHz RAM : 8 GB Disk : 160 GB minimum (10k RPM) or SAN dynatrace Repository dynatrace Collectors License activation License activation
Example: CI with dynatrace at a professional networking site 1. Schedule test runs from Hudson, start selenium ruby scripts 2. Start session recording on dynatrace 3. Start execution scripts against Web Browsers 4. Browser sessions create load on backend 5. Measurements from backend apps are captured 6. Stop session recording 7. Retrieve performance metrics from dynatrace thru REST API as XML 8. Process XML output and generate HTML report 9. Access performance reports from Hudson 9 1 3 4 2 6 7 5 8
dynatrace on dynatrace at dynatrace... Continuous Performance Results for different test use cases 25
dynatrace in Action Novell: Redefining the Value of the Test Centre? instantly Tester Developers Test Run Reproduction Refine Capturing Re Run Tests Reproduction Refine Capturing 5-6 iterations Re Run Tests Reproduction Problem Analysis Results Problem Solving time Increase test throughput by 2-3x Deliver root-causeresults to dev Tester Test Run asynchronously Reproduction Developers Refine Capturing Raise productivity by reducing test cycles and async diagnostics Re Run Tests Reproduction Refine Capturing 1-2 iterations Re Run Tests Reproduction Problem Diagnostics Problem Solving time