Effective Performance Testing Johann du Plessis Micro to Mainframe
Objectives Knowledge sharing Presentations Papers and Articles Workshops Improved performance testing Add value to projects Performance testing more visible
Seven Rules for Effective Performance Testing 1. Follow a proper methodology 2. Do a complete assessment 3. Complete a test plan 4. Test while scripting 5. It s not all about load 6. No shortcuts with monitoring and analysis 7. No shortcuts with report and feedback
Effective Performance Testing Rule1: Follow a Proper Methodology Always implement a methodology Same methodology for all projects Compare & measure quality/standards Same deliverables for all projects Apply best practices What value are we adding? Methodology must help to answer this
What Value Are We Adding? Are We Adding Value? A Good methodology allows us to determine (measure) the value we are adding to a project Successful tests Identify problems early Recommended changes actually improve performance Deliverables Assessment report Test plan Test scripts Test scenarios Test results Result summaries Test report & presentation Visibility of performance testing efforts
Performance Test Methodology Phase 1 - Assessment Phase 2 - Planning Phase 3 - Scripting Phase 4 - Test execution Phase 5 - Results analysis Phase 6 - Report & feedback
Effective Performance Testing Rule2: Do a Complete Assessment Assessment (Can we do it? If yes, how?) Requirements Architecture / Platform How much work? Time available vs. time required Tool to be used Confirm application can be scripted Data requirements Test requirements Exclusions Manage Expectations Deliverable assessment report Expectations should be managed for the duration of the project Our most successful projects all started with a proper assessment
Assessment Report
Effective Performance Testing Rule3: Complete a Test Plan Goal Objectives Scope Business processes to test Environment to test Monitoring Requirements for test execution Responsible people Deliverable test plan The test plan can also act as a checklist with so many details needed for a successful performance test Add names of responsible people to the test plan
Performance Test Plan
Effective Performance Testing Rule4: Test While Scripting Don t loose focus of the testing Be aware of what is happening with the application and the environment as a whole Interact with developers, users, business Follow the basics when scripting Get a feel for the Best practices system Familiarise with application Understand the business processes Understand the data requirements Know the environment Scripts must exercise the whole environment Verify script execution in the database Manage test data properly Start monitoring as part of scripting Be alert while scripting Deliverable test scripts You need to be in a position where you can completely trust the tool and your scripts
Take Note / Be Alert During Scripting and Script Replay
Take Note / Be Alert During Scripting and Script Replay
Effective Performance Testing Rule5: It s Not All About Load Performance testing Baseline tests Load tests Stress tests Volume tests Soak tests Deliverables Test scenarios 85% of performance problems can be identified and solved during baseline test runs The idea is to get an almost clear run when you run the 1 st load test
Effective Performance Testing Rule6: No Shortcuts with Monitoring and Analysis Monitoring & Analysis Plan early All the relevant people involved Know monitoring requirements Test tools limited, make use of other options Full monitoring with baseline tests Deliverables Test results Results summary (between test runs) The results that you want from a test must be the goal to work for when designing performance test scenarios
Effective Performance Testing Rule7: No Shortcuts with Report and Feedback Final report & presentation Covers project as a whole Before and after scenario Deliverables Performance test report Presentation
Show The Value You Add 1 st Load Test Final Load Test
Questions?