Why Performance Matters. Cloud Testing with SOASTA CloudTest. Hexaware Introduction. Best Practices for Performance Testing in Production.

Transcription

1 Performance Testing in Production Using the Cloud to your advantage Presented by: Rob Holcomb VP Performance Engineering SOASTA Chris Cho Director, BTO Division Hexaware Technologies 1 Agenda Why Performance Matters Cloud Testing with SOASTA CloudTest Hexaware Introduction Best Practices for Performance Testing in Production Summary Questions 2 1

2 Using the Cloud to Test Your Site More Today Than Ever Recent Headlines 2

3 More Today Than Ever as It Is Measured in Lost Sales Recent Headlines Recent Reports Google loses 20% Traffic with.5 second of Latency USERS APPLICATION PLATFORM NETWORK 3

4 Quantity Type Location Browser Servers Settings Load Balancer Firewalls Databases Memory Connections USERS PLATFORM PERFORMANCE MANAGEMENT APPLICATION NETWORK Code Content Architecture External Com API CDNs Location Traffic Load Generation Inside a Test Lab 4

5 Load Generation Public Cloud Load Generation: Private & Public Clouds 5

6 Global Test Cloud New York UK San Francisco Hong Kong Target Application: Behind a Firewall Managed Service Provider Cloud Platform 6

7 Net 9/30/2009 The Leading Cloud Service For Performance Testing FIRST Cloud Testing Company (2007) Provisioned Over 200,000 Cloud Servers AWARD Winning Technology: Top Ten New Cloud Services (IW) Best Cloud Application (IT Mgmt) Visionary Leader MQ (Gartner) Only Integrated Cloud Based Test Solution Millions of efilers each TAX year Several Performance issues in Past Internal Test Lab (4K User Limit) TEST TOOLS: Silk-Keynote-Wily CLOUDTEST LIVE PRODUCTION SITE Tested over (33) Days, (7) Hours Per Day Slowly Ramping up User Traffic Ramp-Monitor-Detect-Fix-Ramp R Peak, Reached 300,000 concurrent users Peak, provisioned 2,200 Cloud server cores Detected (27) critical performance issues 7

8 Net 9/30/2009 Cost: Using the Cloud as the NEW Test Lab is affordable Agility: Scalable Test Lab at your Fingertips Scale: Simulate hundreds up to millions of Users Hexaware s Wide Range of Enterprise Services Vertical Competency Travel, Transportation, Hospitality & Logistics Banking, Financial Services & Insurance Emerging g Verticals Horizontal Competency Enterprise Solutions Human Resources BTO & Testing Solutions Business Process Outsourcing Business Intelligence & Analytics Application Management 16 8

9 Global Presence 17 Hexaware Quality Management End-to-end strategic testing services with: Strategic APT Methodology Our Approach, People, Technology (APT) methodology honed over hundreds of engagements Proven Center of Excellence approach for Quality Assurance Best practice approach to lifecycle QA management Customized Solutions OneSource Business Solutions Business Process Testing (BPT) Accelerators CloudTest On-Demand 18 9

10 Hexaware Quality Management End-to-end strategic testing services with: Global Presence Worldwide development centers Flexible options for onsite, near-shore, off-shore, and outsourcing capabilities Deep-Testing Competencies Spanning industries including financial services, insurance, telecommunications, healthcare, travel, transportation, manufacturing, hi-tech, and hospitality Formal partnerships with HP, IBM, Oracle, SAP, SOASTA 19 5 Key Stages of a Performance Test APPROACH Strategy, Discovery & Design Construction Execution Analysis Reduce Risks Reduce Failures Reduce Costs Optimization & Defect Management Define Test Requirements & Objectives, Build Test Plan, Design Test Steps Build Script Modules, Begin Data-Creation Activities Execute Performance Tests; Analyze Results & Perform Root-Cause Defect Analysis Optimize Application Performance, Prioritize Defects PEOPLE Global Resources BTO Professionals Independent QA Team Industry Certifications SDLC QA Methodology &Tools Experience Cloud and Web-based Application Expertise Deploy Plan & Discovery Train Design SDLC Software Development Lifecycle UAT Develop Unit System Develop Iteration TECHNOLOGY CloudTest On-Demand Real-Time Infrastructure Monitoring CloudTest Conductor Post-Production Monitoring & and CloudTest Certification PPM Design & Implementation Requirements Design & Validation Center of Excellence (CoE) Strategy & Implementation Test Strategy Development & Implementation BPT Accelerators for ERP, CRM, & Web-based Applications Functional Verification Performance Validation & Optimization 20 10

11 Why Performance Testing is Critical Agile Architecture such as SOA allows performance issues to mask themselves at many levels Additional Hardware & Software without Root-Cause Analysis can compound the problem The Old Mantra: Reduce Risk, Reduce Defects, Reduce Cost Today s Paradigm: Increase performance, produce a positive business outcome 21 When to Performance Test Technical Drivers Anytime Change or Release Management is Invoked Business Drivers Seasonal Surge Special Events Marketing Campaigns ROI Analysis Beginning i & End of Projects Agile Project Checkpoints end of sprints 22 11

12 Staging and Production Testing Staging Production Staging Attack the low-hanging fruit Optimize application parameter settings Tune based on initial sizing estimates Conduct preliminary interface volume testing If available, use production data source 1-1 staging to production extrapolation not always possible Take into consideration fact that staging may be inside the LAN Production Optimize for throughput Identify data volume errors Tune infrastructure components Identify capacity limits of production environment Verify that boundary systems do not trigger unintended Transact. Take proper care with setting customer expectations Test data usage in production must be tagged or handled appropriately 23 Production Testing Best Practices Address the Taboo of Production Testing Bend but don t break Focused, Real-Time Monitoring Controlled Load Injection Operational support is key Understand data and its impact Data Management Data Security Workflow Consequences Focus on Results 24 12

13 Typical Performance Test Types Performance Test Endurance Test Load Load Time Time Stress Test Elasticity Test Load Load Time Time 25 Data and Its Role In Performance Testing Data Management Data use and consumption play a key role in performance test planning. Data Security When sensitive or production data items are used, special care needs to be taken to minimize risks. Data Misuse Using data correctly can add realism to your performance testing but can also be disastrous if not well planned out. DO: Create a data management spreadsheet to calculate your data needs that includes information on virtual users, rates/hour, reusable data, single-use data Plan for data creation if your data management spreadsheet reflects that you do not have sufficient data DO: Identify potential data risks from data management spreadsheet Masquerade potentially sensitive information (SS#, credit card numbers, account numbers, names) Check with your internal security/governance/audit group for any risky data that will be used in the test DO NOT: Assume data is harmless without consulting appropriate stakeholders DO: Obtain approvals for data management spreadsheet Obtain approval from Information Security group for sensitive data use Take into consideration how boundary systems react to test data especially if testing in production DO NOT: Assume that communication or approvals are in place better safe than sorry, no harm in having a final checklist to verify all risks are mitigated 26 13

14 Operational Support & Monitoring Enterprise Monitoring leads to Real-Time Optimization Business and IT owners are both involved Open line of communication via War Room and Webinar; all have real-time access to the results Change Control and Release Management must keep pace with performance optimization 27 Deployment Options On-Demand CloudTest Services External, web-scale load testing Hexaware develops Cloud Tests with your test plans and involvement Hexaware consultants deploy/execute on CloudTest Global Test Platform Collaborative analysis and tuning using real-time analytics CloudTest Virtual Appliance Internal Testing Solution (Build Private Test Clouds) Hexaware Maintains and Updates (lease model) 28 14

15 CloudTest Summary No hardware or software to buy Hexaware will provision the hardware from leading cloud IaaS providers (Amazon EC2, GoGrid, Rackspace, Savvis, Terremark), design and execute tests, and deliver analysis in a fraction of the time and cost traditional performance vendors can deliver Load is driven from the Web using the Web via Cloud Computing Real-time analytics allows on-the-spot configuration changes to your system while the test is still running true Agile testing Removal of serious functional flaws related to data volume and synchronization issues not identified by the functional team 29 Questions Contact: t Chris Cho Director, BTO Division Hexaware Technologies