TESTING AND OPTIMIZING WEB APPLICATION S PERFORMANCE AQA CASE STUDY

Size: px

Start display at page:

Download "TESTING AND OPTIMIZING WEB APPLICATION S PERFORMANCE AQA CASE STUDY"

Paula Lee
8 years ago
Views:

1 TESTING AND OPTIMIZING WEB APPLICATION S PERFORMANCE AQA CASE STUDY

3 What is Load Testing? Load testing generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the program's services concurrently. As such, this testing is most relevant for multi-user systems, often one built using a client/server model, such as web applications. Copyright 2009 TEST4LOAD Software Load Test Experts 3

program by simulating multiple users accessing the program's services concurrently.

4 Why Load Test Your Application? High cost of mission-critical application failure Simulate real-life workload conditions Find boundaries of your system Locate potential problems before your customers do Reduce development cycle and time-to-market Predict and reduce infrastructure costs Copyright 2009 TEST4LOAD Software Load Test Experts 4

conditions Find boundaries of your system Locate potential problems before your

5 Load Testing Goals Performance Ensure your application s performance under basic workload (baseline) Stress Find break points and boundaries of your application Reliability How stable is your application under extended period of load Scalability Determine how your application scales for the workload growth Copyright 2009 TEST4LOAD Software Load Test Experts 5

stable is your application under extended period of load Scalability Determine how your

Load Testing and Product Life Cycle Requirements Design Development Testing Production Define performance requirements Design your system based on

6 Load Testing and Product Life Cycle Requirements Design Development Testing Production Define performance requirements Design your system based on performance requirements Load test separate components Load test a whole system Monitor your system s performance Copyright 2009 TEST4LOAD Software Load Test Experts 6

performance requirements Load test separate components Load test a whole

Background AQA The Assessment and Qualifications Alliance (AQA) is the largest of the three English exam boards. Awarding 49% of full course GCSEs and 42% of A- levels nationally.

9 Background AQA The Assessment and Qualifications Alliance (AQA) is the largest of the three English exam boards. Awarding 49% of full course GCSEs and 42% of A- levels nationally. Processing over 3.5 million exams each year. Providing online services to teachers, exams officers, examiners and students. Copyright 2009 TEST4LOAD Software Load Test Experts 9

Awarding 49% of full course GCSEs and 42% of A- levels nationally. Processing over 3.

10 SUT ERA (Enhanced Results Analysis) ERA is a free on-line tool that gives teachers in UK an instant analysis of exam results. ERA s key features: Grades reports by centre and subject Cross-centre and years results comparison Individual students grades analysis Question by question analysis Copyright 2009 TEST4LOAD Software Load Test Experts 10

ERA s key features: Grades reports by centre and subject Cross-centre and years

The Challenge Database size over 100 million records 10 million active records 70,000 users 3000 2500 2000 1500 1000 500 0 22/08/2007 - Requests per 10 minutes 00:00 01:00 02:00 03:00 04:00 05:00

11 The Challenge Database size over 100 million records 10 million active records 70,000 users /08/ Requests per 10 minutes 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 14/08/ Requests per 10 minutes Copyright 2009 TEST4LOAD Software Load Test Experts :00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 Workload of less than 10 pages/min at usual day Pick of 300 pages/min at results day

Requests per 10 minutes Copyright 2009 TEST4LOAD Software Load Test Experts 11 3000 2500 2000 1500 1000 500 0 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00

13 Test Architecture Server Oracle Web Cache Oracle AS Oracle RDBMS Configuration Microsoft Windows Server 32bit Oracle Web Cache RedHat Linux ES4 U6 Oracle Application Server ChartFX 6.2 RedHat Linux AS4 U7 Oracle RDBMS Copyright 2009 TEST4LOAD Software Load Test Experts 13

14 Identifying Test Scenarios 14 Pick 5 most common scenarios in your system Don t add scenarios which operate similar back-end flows Identify problematic back-end flows ahead and make sure scenarios cover them. Copyright 2009 TEST4LOAD Software Load Test Experts

flows Identify problematic back-end flows ahead and make sure

15 Identifying Potential Bottlenecks Search Usually sensitive to various input values. Simple or not implemented paging can cause to slow response time Search Results If paging is not implemented, result page can be very large Back to search results page from item details can execute search again External and 3rd Party Components 3rd party components can cause to deadlocks or poor performance OSSO server can perform slower than the rest of the system Export to Excel Usually returns large result set. Can cause to slow performance. Copyright 2009 TEST4LOAD Software Load Test Experts 15

to search results page from item details can execute search again External and 3rd Party Components 3rd party components can cause to deadlocks or

16 Execution Plan Test the system with light input parameters first Get through the obvious bottlenecks, constantly increasing the workload Run nightly tests every night to ensure reliability and stability of the system At level of 500 users, apply heavy input parameters Test system for randomly distributed workload Copyright 2009 TEST4LOAD Software Load Test Experts 16

reliability and stability of the system At level of 500 users, apply heavy input parameters

17 10 Users Conn. Pool Starvation Measurement Avg./Sum Max. Page Time 1.6 sec 34 sec HTTP 200 3,348 HTTP 304 2,190 HTTP HTTP % 80% 60% 40% 20% 0% Connection Pool Connections Copyright 2009 TEST4LOAD Software Load Test Experts 17

6 sec 34 sec HTTP 200 3,348 HTTP 304 2,190 HTTP 302 410 HTTP

100 Users Apache Shows Its Limit Measurement Avg./Sum Max. Load Size 100 Page Time 5.

500 Users Slow DB & OSSO Measurement Avg/Sum Max. Load Size 1000 Page Time 24 sec 2.

5 min Pages per Second 7 19 HTTP 500 368 HTTP

1000 Users Search & Large Content Measurement Average/ Sum Max.

Concluding 1000 Users 8 hrs Test Measurement Average/ Sum Max.

23 Achieved Results Supported concurrent sessions/users: From 10 to 1000 (x100) Reliability From 90% failure to ~100% reliable system Response Time at 1000 VU From 1 min to less than a second Pages per Minute From 13 PPM to 166 PPM Copyright 2009 TEST4LOAD Software Load Test Experts 23

25 Optimizing Search Mechanism Use existing solutions. Use free text search only as the last option. Limit user input to reasonable values. Implement paging on both search and result levels. Improve Back to search results mechanism. Copyright 2009 TEST4LOAD Software Load Test Experts 25

26 GZip Compression Effectiveness Pros: Reduce size of large text pages by 80%-90% Improve response time by 50% Decrease throughput Cons: Increased CPU utilization by 35% Tip: Apply GZip selectively to text-based static resources and large dynamic pages Do not apply GZip to every response! Copyright 2009 TEST4LOAD Software Load Test Experts 26

27 Database vs Web Servers Costs Additional database setup and maintenance costs are significantly higher than of web server. Make DB work easier by making web server work harder. Memory costs less than CPU or new hardware rely more on memory, but keep in mind that everything has its limits. Copyright 2009 TEST4LOAD Software Load Test Experts 27

28 Developers and Testers Tips Get to know your user. Evaluate his system usage behavior or try to predict the common flow. Know the costly operations of your system. Try to build the flow that omits recurrent usage of the operations. Do not be reluctant to cache data, still don t overuse the cache. Remember that shared data should be stored once for all users. Copyright 2009 TEST4LOAD Software Load Test Experts 28

29 Project Manager Tips Make sure all involved parties are available at the moment of test execution. Share test findings among all project members. Log all changes to the test environment. You will need to reapply them later to the production environment. Make sure the final results meet the performance requirements and business user expectations. Copyright 2009 TEST4LOAD Software Load Test Experts 29

30 Team Synergy Synergy and cooperation between the following teams is crucial for the success of the test: Database Team Application/Development Team System Team Performance Team or Specialist Copyright 2009 TEST4LOAD Software Load Test Experts 30

Questions? Contact Info Nachum (Nick) Dimer Owner & Chief Consultant IL: +972.54.

31 Questions? Contact Info Nachum (Nick) Dimer Owner & Chief Consultant IL: UK: PL:

Bernie Velivis President, Performax Inc

Performax provides software load testing and performance engineering services to help our clients build, market, and deploy highly scalable applications. Bernie Velivis President, Performax Inc Load ing