Enterprise Performance Testing. Karuna Karnati and Ajay Kumar. Accenture Delivery Center for Technology in India

Size: px

Start display at page:

Download "Enterprise Performance Testing. Karuna Karnati and Ajay Kumar. Accenture Delivery Center for Technology in India"

Russell Rose
8 years ago
Views:

1 Enterprise Performance Testing Karuna Karnati and Ajay Kumar Accenture Delivery Center for Technology in India Accenture, its logo, and High Performance Delivered are trademarks of Accenture.

2 Enterprise Performance Testing Failure is not an Option 2

3 Agenda Why Performance Testing Types & Approaches Part of SDLC Environment Test Metrics Report Crafting 3

4 Industry Facts What it costs when your internal enterprise systems go offline. Industry Application Cost per Hour Financial Brokerage Operations $6,500,000 Financial Credit Card Sales $2,600,000 Media Pay-per-view $1,150,000 Retail Home shopping (TV) $113,000 Retail Catalog sales $90,000 Transportation Airline Reservations $89,500 For external web sites, from the same Standish Group study, the typical costs were in the middle of the range, running anywhere from $1,000 to $27,000 per minute of downtime, or $60,000 to $1,620,000 per hour. Later figures, since 1998, are much higher as companies move more of their business onto the web. Data from The Standish Group 4

(TV) $113,000 Retail Catalog sales $90,000 Transportation Airline Reservations $89,500 For external web sites, from the same Standish Group study, the typical costs

5 Industry Facts 2002 Winter Olympics, an online ticketing page was unavailable during a critical sales period placing a potential of $1.5 million dollars of revenue at risk. Application problem with the State of Minnesota s $27 million online tax filing system delayed tax refunds and diminished consumer confidence for greater than 60,000 tax filers. Enterprise systems that perform poorly, function incorrectly or fail completely cost businesses money, reputation and market share. 5

Application problem with the State of Minnesota s $27 million online tax filing system delayed tax refunds and

6 Industry Facts Performance measurement and tuning have become more rigorous over the past few years. Worldwide Web-Load Testing Market Growth Distributed Environments,

7 Why Performance Testing? Failure is not an Option 7

8 Why Performance Testing Speed Does the application respond quickly enough for the intended users? Scalability/Capacity Will the application handle the expected user load and beyond? 8

9 Why Performance Testing Stability/Robustness Is the application stable under expected and unexpected user loads? Confidence Are you sure that users will have a positive experience on go-live day? 9

10 FURPS - Performance F unctionality U sability R eliability P erformance S upportability Performance Effectiveness Scalability 10

11 Performance Testing Performance Testing is the discipline concerned with determining and reporting the current performance of a software application under various parameters Performance Testing Types Load & Stress Performance Test approaches Scalability & Stability Performance Testing terminologies Scenario /Business Flow Operational Profile Work load Performance Metrics Client side & Server side 11

Load & Stress Performance Test approaches Scalability & Stability Performance Testing terminologies

12 Performance Testing - SDLC Performance testing as an integral part of SDLC IDE tools Eclipse Rational Performance Tester Visual Studio Team Microsoft Profiling and Probes 12

13 Trends in performance testing Performance testing as an integral part of SDLC New IDE Tools Microsoft, Eclipse, etc.. Object and Resource profiling Probes Architecture Framework evaluation against performance requirements The Open Group Architecture Framework (TOGAF) ( The Architecture Tradeoff Analysis Method (ATAM) ( 13

. Object and Resource profiling Probes Architecture Framework evaluation against performance requirements

14 Performance Test Environment LoadRunner Controller Get scripts from Vugen Load Agent Load Balancer Simulate virtual users from Load Agent Virtual User Generators Web Servers Application Servers System Under Test Database Server 14

virtual users from Load Agent Virtual User Generators Web

15 What is a Counter? A Counter is a unit that measures and gathers performance-relevant events of the system Client Side Metrics Response Time Throughput Hits Per Second Pass/Fail Statistics Server Side Counters Memory Processor Disk Network 15

16 Categories of Counters Memory Cache RAM HDD Processor Disk Seek Time RPM Platter Network 16

17 Performance Test Tool Tool LoadRunner SilkPerformer WebLoad Jmeter Vendor Hp-Mercury Borland RadView OpenSource Language Java/C BDL Javascript Record& Play 17

18 Report Crafting Provide test results in tabular format and graphs Include all issues faced during testing Document all the findings & observations (performance testing is close to research) Load and especially stress would sometime reflect the bad side of an application. It throws errors, capture all of them for future analysis Any deviations/workarounds used should be mentioned Conclusions should be substantiated with proper analysis Raw data can be provided as an appendix 18

19 Where to go for more information (My site) (QA Blog Collection) (Huge QA Forum) (Good articles and links) m (Good articles and statistics) (Good articles and statistics) 19

loadtester.com (Good articles and links) http://www.segue.com/html/s_solutions/papers/s_wp_info.

20 Final Message Performance Testing is Insurance to the Business Perf-surance 20

21 Questions?.. 21

22 Thank You 22

Performance Testing of a Large Wealth Management Product

Performance Testing of a Large Wealth Management Product Meherphani Nori & Global Head Quality Assurance Krishna Kankipati & Vice President Mohan Pujari & Product Specialist Broadridge Financial Solutions