Productized Services: Cloud Performance Testing Presented By Rupesh Garg, Wipro Technologies. 1
All About Cloud What is Performance Testing Cloud Testing Approach Cloud Service Providers Any Questions 2
Productized Services - Definition 3
Benefits of Productized services Allow the market to self-service their final deliverables by means of standardized packages with key parameters and KPIs, which dramatically shortens sales cycles and reduces cost of sales. Help in ensuring repeatability and consistency, and also reduce the amount of customer intervention in the process, which in turn improves profitability. Allow service providers to achieve efficiencies as deliverables are pre-defined, and enable them to offer differentiated and top-up services at a premium, thereby increasing profitability. 4
All About Cloud
Cloud? Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. 6
Cloud? Hybrid Clouds Deployment Models Private Cloud Community Cloud Public Cloud Service Models Software as a Service (SaaS) Platform as a Service (PaaS) Infrastructure as a Service (IaaS) Essential Characteristics On Demand Self-Service Broad Network Access Rapid Elasticity Resource Pooling Measured Service Common Characteristics Massive Scale Homogeneity Virtualization Low Cost Software Resilient Computing Geographic Distribution Service Orientation Advanced Security 7
Performance Testing 8
Why Performance Testing? Performance Testing is used to evaluate following qualities in a system Reliability Scalability Interoperability 9
Understand Performance Testing Causes of inadequate performance testing Lack of requirements Lack of understand of the fundamentals of performance testing Misinformation provided my tool and simulator vendors Limited budget or funding Enormity of the task 10
Evolution of Performance Testing Load Generator 11
Evolution of Performance Testing Load Generator Firewalls Load balancers Routers Switches Appliances 12
Evolution of Performance Testing Load Generator Firewalls Load balancers Routers Switches Appliances 13
Evolution of Performance Testing Load Generator Geographic Locations Firewalls Load balancers Routers Switches Appliances 14
Evolution of Performance Testing Load Generator Geographic Locations Network Providers Firewalls Load balancers Routers Switches Appliances 15
Evolution of Performance Testing Load Generator Geographic Locations Network Providers Firewalls Load balancers Routers Switches Appliances 16
Evolution of Performance Testing Load Generator Geographic Locations Network Providers Firewalls Load balancers Routers Switches Appliances Routes Latency 17
Evolution of Performance Testing Load Generator Geographic Locations Network Providers Firewalls Load balancers Routers Switches Appliances End-User Response Time Routes Latency 18
SaaS: Performance Testing Approach Performance Testing Cloud Infrastructure Data Business Logic End-user Experience Load Testing Performance Analysis Performance Management 19
Why use the cloud for load testing? The cloud brings substantial benefits to load testing, whether the testing is carried out by a large corporation with an in-house performance center or by a start-up. o More flexibility: multiple generators, multiple internal resources etc. o More savings: you only pay for the power you use, when you really need it. o More realistic: you can test more realistic scenarios with load being input from different parts of the globe. 20 o No limits: you can progressively test heavier and heavier loads (the power available 2012 WIPRO is almost LTD WWW.WIPRO.COM limitless) INTERNAL to find the real limits of
Using the Cloud for Load Testing Ability to correlate the load with end-user response times and the impact on the systems We can borrow particular environment as per needs rather than going through the licensing and setup worries. Ability to control the distribution of the load Ability to use the same systems and configuration for subsequent testing With the help of Virtualization, it enables us to test multiple configurations, which was not possible few years back. Perfect coordination of the load injectors Know and understand the origination of the load (IP, geography) 21
Traditional Performance Testing Approach Controller LG LG SUT LG LG LG 22
Cloud World 23
Performance Testing From Cloud 24
Performance Testing Approach Automation Test Scripts Preparation Test Execution & Monitoring Web Tire App Tire DB Tire Test Results Analysis Requirement Gathering 25
Typical Performance Testing Gather Plan Design Execute Analyze POC Environment Setup costs Tool Setup costs Test Data Management Load Injectors Capacity Planning Performance Modeling 26
Cloud Performance Testing Gather Plan Design Execute Analyze Tool/Environment/ POC setup on the cloud IAAS/PAAS/SAAS Load injectors/test Data /Performance Modeling/Modeling Capacity on cloud IAAS/SAAS/PAAS 27
Cloud Performance Testing Setup Performance Tester F i r e w a ll Performance Testing and Monitoring Tool Installed (SAAS) Data Storage (PAAS) App Under Test F i r e w a ll Load Injectors from Cloud ( IAAS) 28
Solution Overview Wipro Performance Team @ Atlanta Client System Under Test Environment IaaS Provider Performance Test Tool SaaS Provider Wipro Performance Team @ Flex Delivery Center Offshore 29
Controlled Cloud Testing Private Cloud Know and understand the origination of the load (IP, geography) Ability to control the distribution of the load Ability to use the same systems and configuration for subsequent testing Perfect coordination of the load injectors Ability to correlate the load with end-user response times and the impact on the systems 30
Wipro Cloud load tests 31
Cloud Based Testing Services NEOTYS App Perfect Gomez Platform Lab Soasta Keynote Browser Mob Load Impact Load Strom HP - Beta sauce LABS PushToTest 32
Load Storm LoadStorm is a cloud load testing tool that generates web traffic for you. Please sign-up for a free account or take a quick tour of the tool. 33
http://www.soasta.com/cloudtest/ SOASTA, Inc. provides services to test websites and web applications. [2][3] It is the leading provider of cloud-based testing services, and created the industry's first browser-based website testing product. Website tests include load testing, performance testing, functional testing and user interface testing. SOASTA provides cloud website testing with their product CloudTest, which simulates thousands of users visiting a website simultaneously using the Amazon Elastic Compute Cloud (EC2) service. SOASTA allows customers to use predefined tests or create customized tests to automatically test their web. "There was no way we could ever have simulated 500,000 users against our site in an internal test lab," said Christopher Roe, CTO, Qtrax. "By cloud testing we were able to identify connection issues immediately and fix the site prior to production. 34
http://www.keynote.com/products/ 35
Gomez Gomez: Test from where your end-users are located by selecting from over 100,000+ load generating consumer-grade desktops 500+ Combinations of Browsers, Operating Systems and Screen Resolutions. Whether you need to test on Windows XP and Internet Explorer 6, Mac OS X 10.5 and Firefox 2.0 or the iphone OS and Safari 2.0, Gomez s comprehensive, up-to-date virtual test bed has you covered with more than 500 combinations of browsers, operating systems and screen resolutions plus more than 5,000 supported mobile devices. 36
http://saucelabs.com( Automation) Cross browser testing with Selenium on Cloud 37
http://acutest.co.uk/acutest/load-testing SAP performance testing Cloud load testing service: where the load comes from the cloud Stress testing cloud services: where the system under test is in the cloud Load testing CRM and ERP implemenations Infrastructure performance monitoring and tuning (including the network) Performance test tools evaluation Load testing web applications, websites and web services Performance testing multi-tiered solutions that include a combination of applications, IT systems and infrastructure IT systems volume testing (including embedded systems) 38
Productized Costing Space - 1 39
Limitations with Cloud Computing "The cloud [test environment] doesn't reflect production environments -- applications run on specific [physical] servers, specific virtualization technology, specific networking and bandwidth, and that is hard to replicate, Forrester Research Inc. "Then there's the issue of cloud providers not supporting the technology that you use at all, like a specific HP blade server. Integration, in particular, is causing a lot of headaches 40
Any Questions 41
References Basics about Cloud Computing: http://en.wikipedia.org/wiki/cloud_computing SOASTA Performs 500th Test in the Cloud http://www.reuters.com/article/pressrelease/idus118692+27-jan-2009+mw20090127 Will Cloud Computing Be the Tipping Point for Change in the Test Industry?http://www.breakingpointsystems.com/community/blog/will-cloud-computingbe-the-tipping-point-for-change-in-the-test-industry Tapping the cloud as a software testing service http://searchciomidmarket.techtarget.com/news/article/0,289142,sid183_gci1356175,00.html Preparing for testing applications in the cloud http://searchsoftwarequality.techtarget.com/tip/0,289483,sid92_gci1357743_mem1,00. html The First Cloud-Based Global Test Platform http://www.soasta.com/blog/?p=142 The Greatest Risk to Cloud Computing? The Vista Effect http://etherealmind.com/2009/03/22/risk-to-cloud-computing/ 42
43