Cloud Testing Production Applications CloudTest Strategy and Approach

Transcription

1 White Paper Cloud Testing Production Applications CloudTest Strategy and Approach Mobile Performance Testing Web Performance Testing

2 Table of Contents Executive Summary Intended Audience The Past and Present of Performance Engineering Product and Methodology - Working Together in Harmony Select Topics From SOASTA s CloudTest Methodology Security Test Data in Production Potential Live Customer Impact Customer Case Studies - Testing in Production Security Case Studies Conclusion About CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 2

3 Executive Summary Web sites are the most accessible consumer-facing outlet for most companies. The brand impact from poor performance or an outage is far reaching and usually garners visibility at the highest levels. Anyone involved in web site performance and operations of consumer-facing applications. Online application performance is critical - no one would challenge this statement. Yet the dirty little secret in the web world is that applications are rarely performance tested. When performance for web applications has delayed the introduction of load testing methodologies for production testing in production. CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 3

4 Applications are increasingly event driven, needing to serve a larger than normal customer base for perhaps only a few weeks out of the year. We are 1. Intended Audience 2. The Past and Present of Performance Engineering CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 4

5 There are certain pieces of critical functionality that are required in a testing tool to be successful at production testing. CloudTest is a purpose-built production performance testing tool. It has the ability to scale up to generate true concurrent user loads up into the millions of users while still delivering the analytics in real time. The most common test tool in use today for web application testing was never designed for this purpose. 3. Product and Methodology - Working Together in Harmony CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 5

6 The methodology SOASTA has developed is a full lifecycle system. It starts with strategy and planning, and then proceeds into implementation details. The third pillar is execution. Finally, a full iteration of the process ends with measurement of success and changes in the processes to improve the system, ultimately looping back to begin again. 4. Select Topics From SOASTA s CloudTest Methodology CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 6

7 Lower left: Tests are created and controlled via web browser from anywhere. Center: Load is generated from one or more cloud environments around the world. directed at the target application wherever it resides (private datacenter, cloud, etc) over standard HTTP and HTTPS ports just like real users. 4.1 Security Sensitive Data Applications Non-Disclosure Agreements CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 7

8 Sensitive Data Sites Pre-Release and Early Access Sites IP Address Restriction Cookies and Headers Transaction tagging CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 8

9 CloudTest dashboards combine data from the test controller perspective with data from the Another data stream that might be present during a running cloud test is the monitoring data (CPU, Memory, etc). Whether the data is coming from SOASTA s monitoring agents themselves or from 3rd party monitoring systems, the data is always relayed through SOASTA s agent called and then it uses push technology to send data to the CloudTest main instance. While this data is not generally considered sensitive, as it is only related to the health of systems under test, it can be encrypted for further security and transferred over HTTPS. the servers being monitored and the outside world. This creates a secure relay point for the monitoring data to travel through. leaves the datacenter. Unfortunately, by choosing this option, customers lose the ability to see results. Test data, in the context of this paper, refers to data that resides in an environment (production, in this case) to enable testing. It can also refer to data that was created in the production environment as a result of testing, such as test orders, test accounts, and things of that nature. Test data is sometimes used loosely to refer to data collected during a test by the CloudTest load generators, such as result reports, dashboards, monitoring data, etc. In some cases, added security to companies whose performance and benchmarking are critical to their business, such as platform performance information about their software. CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 9

10 4.2 Test Data in Production on the coverage Using Test Data in Production User Accounts query plan to optimize for database fragmentation Handling Other Types of Test Data inserted, updated or deleted CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 10

11 A common performance problem that databases encounter over time is fragmentation. If you delete large chunks of data from a database, there will be gaps in the data and the individual pages that store data will become noncontiguous. This causes SQL statements to take much longer to process. Test Data Cleanup database fragmentation Coding for Testability CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 11

12 alternate code paths 4.3 Potential Live Customer Impact Three Requirements for Successful Live Testing kill switch 5. Customer Case Studies Testing in Production 5.1 Banking and Financial Data Security test data and alternate code paths CLOUD TESTING PRODUCTION APPLICATIONS WHITE PAPER 12

13 IP address restriction Conclusion About Headquarters Castro St, Fourth Floor Mountain View, CA FAX: EMEA +44 (0) Thames Court 1 Victoria Street Windsor, Berkshire SL4 1YB To learn more visit: 2012 SOASTA. All rights reserved. SOASTA, the SOASTA logo, and SOASTA CloudTest are trademarks of SOASTA. All other product or company names may be trademarks and/ or registered trademarks of their respective owners. EST Connect with us: