Performance Testing of a Cloud Service

Transcription

1 Performance Testing of a Cloud Service Trilesh Bhurtun, Junior Consultant, Capacitas Ltd Capacitas Introduction Objectives Environment Tests and Results Issues Summary Agenda Capacitas

2 Customer Over 99% of the company's revenue is earned via the ecommerce service Highly Competitive Market Business Critical Service 20% year on year growth Frequently introducing new products to keep up with the competition and to increase revenue Large ICT infrastructure mirrored across 2 data centres Capacitas Introduction Look at benefits arising from migrating the web and logic infrastructure that currently supports the ecommerce service to a cloud service Proposed a private cloud service with virtual machines (VMs) Some initial sizing of the number of virtual machines (VMs) required to meet business demand had been delivered by the service provider Capacitas

3 Introduction This sizing had been achieved using assumptions around the capacity of the VM environment relative to the current production environment Customer wanted to conduct a proof of concept to assess whether the cloud service is fit for purpose from a capacity and performance perspective Capacitas Introduction Capacitas offers performance testing as a managed service to this customer amongst other services This includes Risk Assessment Writing Test Plan Run Performance Tests Produce Test Report Capacitas

4 System Architecture Three tier layer Interfacing to various B2B and other 3 rd parties Capacitas Software Development Agile Development Process Sprint Planning Large number of developers working towards a monthly release Pre release activities include: Functional Testing Performance testing Capacitas

5 Objectives The main objective for the customer was to understand the capacity and performance characteristics of the ecommerce application on the cloud offering compared to the existing physical infrastructure In order to do this, Capacitas planned to carry out a series of tests to measure the resource cost per transaction on both virtual servers in the cloud and the customer s physical infrastructure Capacitas Objectives In addition, the virtual machines were running on Windows 2008, Capacitas attempted to see if the current Windows 2003 limitation of maximum process size of 1.2 GB is removed in Windows 2008 Customer had opted to use the existing Load Runner load generation capabilities in the Test environment for this Cloud Proof of Concept project Capacitas

6 Objectives Objective Description 1 Prove that the ecommerce web application is stable when running within a VM instance 2 Quantify the capacity throughput per web server VM in business terms 3 Quantify the capacity throughput of the web server VM, relative to the current production web server 4 Quantify the capacity throughput per Logic server VM in business terms 5 Quantify the capacity throughput of the Logic server VM, relative to the current production Logic server 6 Demonstrate that the known web server memory constraint which exists in production is alleviated on a web VM 7 Demonstrate that the CPU cost per business transaction is constant under increasing load 8 Demonstrate that the web server application memory leaks are no worse than those on the current production service, as far as can be reproduced in test 9 Prove that the operational management tools are fit for purpose from a capacity management perspective Capacitas Environment Cloud servers were connected to the existing physical performance testing environment with load generation capability. Capacitas

7 Environment The Test environment needed to have the following configured: SSL turned on at the web server Compression turned off at the web server The virtual environment needed to have the following configured: SSL turned off at the load balancer Compression turned off at the load balancer Caching turned off at the load balancer SSL turned on at the web server Compression turned off at the web server Capacitas Environment Service Provider will be able to move the virtual servers over the two physical boxes according to the requirements of the test plan The underlying architecture was based on two clustered HP DL585- G7 4 CPU * 2.1 GHz AMD Opteron Twelve-Core, 128GB RAM servers The servers were running vsphere v4.1 Capacitas

8 Limitations This approach had the following limitations because of the limited load generation capabilities: The logical layer was not loaded realistically during web tests There was no testing of the multiple machine logical memcached configurations under significant load however this issue exists with existing test environment There was no significant load on the underlying hardware that supported the virtual machines Capacitas Tests The tests consisted of the following transactions: Homepage Search Product Selection Additional Options Login Purchase Order Confirmation Order History Capacitas

9 Tests Test LoadRunner Scripts Against Cloud Servers Setup Performance Counters Prepare Test Scenarios Capacitas Test 1 4 vcpu WEB Service times at the WEB server were much higher for some steps when measured on the cloud server compared to the physical server Capacitas

10 Test 1 4 vcpu WEB Page sizes from the cloud tests were half than those measured from the baseline This indicated that compression could be turned on Capacitas Test 1 4 vcpu WEB Compression was enabled both at the WEB and Logic servers Capacitas

11 Test 2 4 vcpu WEB (Compression Off) Turning Compression off did not improve the performance at the WEB server by much Capacitas Test 2 4 vcpu WEB (Compression Off) Page sizes were similar to the ones measured on the baseline Capacitas

12 Test 3 8 vcpu WEB The 8 Cores WEB Server was performing worse than the 4 Cores WEB server Capacitas Test 3 8 vcpu WEB Further investigation into these issues showed that the web site was running in debug mode Capacitas

13 Test 4 4 vcpu WEB (Debug Off) Service times decreased considerably but were still higher than those measured on the physical servers Capacitas Test 5 8 vcpu WEB (Debug Off) The 8 cores web server was still performing worse than the 4 cores one even with debugging off Capacitas

14 Logic CPU Service times on the cloud Logic server was nearly similar to the ones measured on the physical Logic server except for the purchase step Capacitas VMware Counters VMware Performance Counters Collected Capacitas

15 VMware Counters Capacitas VMware Counters Capacitas

16 Transactions Per Second Response Time (s) Response Time Response Times Home Search Purchase Physical Cores cores The increase in response time had a negative impact on throughput as well Response times were worse across the main steps on the cloud servers when compared to physical servers Transactions Per Second Home Search Purchase Physical Cores cores Capacitas Setup Network Connectivity Configuration inconsistency Firewall Incompatible Code Issues Capacitas

17 Conclusion Moving the ecommerce service to the private cloud was not a technically and from the discussions economical feasible proposition They would need 288 and 64 physical cores (3.0 GHz) at the web and logic server respectively, if they used 4 VCPU servers 2.1 GHz AMD Opteron) then they would need 480 and 80 cores at the web and logic servers respectively Capacitas Questions? Trilesh Bhurtun Junior Consultant Capacitas [email protected] Capacitas