Mike Chyi, Micro Focus Solution Consultant May 12, 2010
Agenda Load Testing Overview, Best Practice: Performance Testing with Diagnostics Demo (?), Q&A
Load Testing Overview What is load testing? Type of load testing, Load testing architecture (lab, cloud), Web Load Testing (other environment), How to pick tools (criteria)? Best Practices.
Load Testing Overview What is load testing? Type of load testing, Load testing architecture (lab, cloud), Web Load Testing (other environment), How to pick tools (criteria)? Best Practices.
Load Testing Generic term for testing applications (whole infrastructures) under load to ensure that they work and deliver acceptable response times and throughput Catch all" term for the many different types of tests that can be performed: Load, Performance, Stress, Volume, Negative, Scalability, Endurance, etc. Performance tests emulate user activity and analyzes the effect of the real-world user environment on an application
Purpose of Load Testing? Put simply, Load Testing ensures that your application works under real-world loads before your customers find that it doesn't! Test Drive
Why is Performance Testing Important? Increase uptime of mission-critical systems Avoid project failures by predicting application behavior under large user loads Measure and monitor performance of your business infrastructure Protect IT investments by predicting scalability and performance Performance testing is the only way to verify the scalability of components working together without going into production!
Load Testing Overview What is load testing? Type of load testing, Load testing architecture (lab, cloud), Web Load Testing (other environment), How to pick tools (criteria)? Best Practices.
Types of Performance Testing Load - focuses on replicating expected user activity levels, both in terms of number of users and the volume of traffic Stress focuses on worst case scenarios that could potentially happen in production environments today or in the future Negative similar to stress testing, negative testing is performed to take applications to their breaking point to learn the limits of the system/application or environment
Load Testing Overview What is load testing? Type of load testing, Load testing architecture (lab, cloud), Web Load Testing (other environment), How to pick tools (criteria)? Best Practices.
Load Generation & Monitoring Controller Load Generator System Under Test (SUT) 200 vu s 500 vu s 300 vu s Test Scripts Server Monitoring
Micro Focus Solution - Cloud Approach and Value Easily consumable high volume load testing Easier to get machines, deploy agents, run tests and collect results Simplification of load testing scripting Test from global points of presence Capture last mile performance Mature tool set Cloud flexibility (Elasticity) Load Test Tooling Enterprise Test Agents Internet Facing Application Internal Application Using the cloud to provide flexible testing solution 12
Workflow of using SilkPerformer with Enterprise Cloud Service SilkPerformer works seamlessly in the cloud Create Agent Instances needed for Load-Test Full monitoring capabilities thru probes placed in the intranet Easy Root Cause Analysis of internet related problems and bottlenecks
Load Testing Overview What is load testing? Type of load testing, Load testing architecture (lab, cloud), Web Load Testing (other environment), How to pick tools (criteria)? Best Practices.
Web Load Testing (most popular) Classic: Capture traffic at HTTP, HTTP/S level Rich Internet Application (RIA), Ajax: Asynchronous, classic record / playback does not work (based on synchronous protocol). Need a new approach, SilkPerformer introduce Browser Driven record / playback technology SilkPerformer 2010.
Load Testing other environment Middleware / Protocol Approach ERP, CRM (SAP, PeopleSoft, Siebel, etc.), Database (Oracle, DB2, SQLServer, etc.), Citrix, Oracle Forms, Framework (source code) Approach.net java GUI driven (low-end, catch all)
Load Testing Overview What is load testing? Type of load testing, Load testing architecture (lab, cloud), Web Load Testing (other environment), How to pick tools (criteria)? Best Practices.
How to pick tools (criteria)? Product Maturity Proven Easy of Use, scripting, data-driven, modification, debugging, logging, workload creation, correlation, reporting, etc. True End-User Emulation, Accurate Measurement, Comprehensive Report,
Load Testing Overview What is load testing? Type of load testing, Load testing architecture (lab, cloud), Web Load Testing (other environment), How to pick tools (criteria)? Best Practices.
Best Practices: Load Testing Process 1. Establish performance test objectives, 2. Prepare test environment (SUT), 3. Define use cases (test scenarios), 4. Create, modify, and baseline your scripts, 5. Execute Performance test (controlled, repeatable) 6. Monitor, diagnose the impact of load on servers/databases, 7. Analyze results, 8. Tune system, 9. Restore database, 10. Repeat, 11. Deploy mission critical application with confidence. (you can sleep better!)
Performance Test Scripts Short and sweet, Normally only a handful of scripts needed (not functional test), 80/20 Rule means that the most common activities use a limited subset of the application functionality Data driven, to emulate real-life user actions, avoid caching, and exercise the backend database, Start simple, login smoke test, Non-destructive, vs. Destructive (add, mod, delete scripts), Create reusable modules (rules, functions), Run a couple of GUI-user scripts to visualize the user scenarios, also provide true end-to-end response times.
Useful Performance Matrix Response Time (percentile 90, 85, 80) Acceptable response time for end user to be productive Concurrency / Scalability / Workload distribution Maximum number of concurrent users that must be supported. Throughput Minimum transaction throughput per (?) Network Footprint Maximum allowable bandwidth utilization Server Impact Maximum server metric(s) tolerance, CPU, memory, run queues.
Root-cause analysis/diagnosis capability to pin-point performance problems TrueLog On Error Shows exactly what happened until the error occured Front-end diagnostics (VU persp.) Response time break-down Method-level diagnostics Diagnose issues from problem page 3 clicks to root cause,
Poor Application Performance Shown above are the application response time and the number of running users. Note how the response time increases as more users are added to the system!
Compare Timing Files compare multiple test runs, allowing testers to easily perceive changes in application behavior (from green to blue)
Question: Why are there still performance issues in production after numerous load testing cycles?
Too Late in the test cycle No performance requirement, Baked in performance requirement early Load test early once the application is functional ready Load test with diagnostics, flush out concurrency issues, coding issues early in the life cycle, eliminate rework later.
The Solution: SilkePerformer: Load Test with Root Cause Analysis. 30 days trial download (10 Vu) http://www.borland.com/downloads/download_silk.html
Load Testing and Root Cause Analysis Virtual Users DBMS Web Servers Application Servers JDBC-ODBC Mainframe URL JSP-ASP Method Web Service Method MQ Identify load related failures by driving anticipated production user load Correlate performance relationships between all application components Pinpoint root cause of performance bottlenecks
Questions?