Software Testing in the Cloud. Tauhida Parveen, PhD

Transcription

1 Software Testing in the Cloud Tauhida Parveen, PhD

2 About Me University Department Chair, Software Engineering, Keiser University PhD, CS, Florida Institute of Technology MBA, University of Central Florida ISTQB Certified Tester (Foundation, Manager, Agile) Independent consultant and trainer Software Quality Engineer/ Tester Yahoo!, Sabre, Wikimedia,

3 Vice President & COO About Me

4 Motivation Cloud computing is here to stay Changing the way we engineer software Leverage the power of cloud in testing

5 Objectives What is the cloud about? How is it different from traditional computing? What is testing in the cloud? What does it mean for testers? Where to start?

6 What is Cloud Computing? An old idea whose time has finally come If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility. The computer utility could become the basis of a new and important industry. John McCarthy, 1961

7 What is Cloud Computing? Virtualized hardware SOA, On-demand, utility model Rapid elasticity Resource abstraction techniques Multi-tenancy Self-healing Not in your control (mostly) / things are taken care of

8 Cloud Services

9 Cloud Services

10 Cloud Services

11 Cloud Delivery Models Public Services offered by a provider that are publicly available, can be used with a contractual agreement Private Virtualized resources available as a service owned by individual organization Client managed, customization, high efficiency, privacy and security policies Hybrid Mix of public and private Extend private cloud via VPN Good candidate for incremental migration

12 Cloud Benefits Faster, cheaper, better Flexible Multiple test beds for multiple release testing Instant setup, tear down Scalable Virtualized environments Scalable resources On-demand Pay-per-use

13

14 What Does it Mean for Testers? How would you test this?

15 Software Testing in the Cloud Lies at the intersection of three key areas Software Testing Cloud Computing System Migration STITC (Software Testing in the Cloud) Testing in the cloud Testing of the cloud Migrating testing to the cloud

16 Traditional Testing Testing in the Cloud Low asset utilization Scalability: Long time to increase capacity Long time to build datacenters Difficult to manage Duplicate test systems Improved asset utilization Less time (instantaneous) increase and reduction in capacity Purchased as a service from cloud providers Better management and increased productivity Aggregated system Creates unnecessary waste Cleaner, greener testing

17 Case Studies Case Study #1: Concurrent execution of JUnit test cases Case Study #2: Concurrent execution of GUI testing of ios applications It can take a large number of test cases to cover all the variations that users could experience Different configurations OS versions (ios?) Device Orientations (Portrait, Landscape) Screen sizes (3.5", 4") Form Factors (iphone, ipad, Universal)

18 Case Study #2

19 Software Testing in the Cloud

20 Software Testing in the Cloud Management Process Technology People Cloud testing strategy Can t be achieved as an execution-only activity Roles and responsibilities

21 Management: Process Strengths Faster Cheaper Better Weakness Network connectivity Security Standards Availability Performance Project management Opportunities Faster execution Faster feedback Efficient testing process Value proposition Threats Complexity Hype Vendor lock Tool chain Education Training SWOT ANAYSIS

22 Management: Process Business Drivers Faster Cheaper Better Migration Strategy MAYBE Desirable YES NO Technical Factors Where are you now? Where do you go next? MAYBE Types of Applications Types of Testing Feasible Test Execution Environment NO YES SMART-T: A decision framework to migrate testing to the cloud MAYBE Stakeholders Workshop Operational Results Trial Migration Pilot Study Acceptable YES Evaluation Results NO Actual Migration Document Guidelines Adjust Estimates Implement Migration Plan

23 Management: Process Evaluate current overlapping activities Agile development and testing Test early, test often TDD, Unit testing JUnit in the cloud Google Test in the cloud HadoopUnit Faster feedback Automation Regression testing

24 Management: Process Cost model How much does it cost you to test now? Do you have a test lab? Machines for every project? How often do you configure the machines? How often do you upgrade them? ABC: Activity Based Cost model Identify and put $$ to all the big and small activities

25 Management: Technology

26 Management: Technology Testing as a Service Distributed processing technologies Test execution environments

27 Management: Technology Amazon Web Services (AWS)

28 Management: People Will your testers be able to adapt to cloud testing? Testing at many levels User Interface, Services, Governance, Process, Security, Integration Programmers? Beyond unit testing System administrators? Cloud service providers Training Continuing education Testing skills, programming skills Communication skills

29 Testing in the Cloud: Strategy Does testing in the change the foundation of testing? Does it involve new technique, technology, and domain? How is it going to effect your current tasks? (good/bad) Do you need new skills? Are there different types of testing? Are there tools to help you? Risks associated with cloud testing Assumptions, constraints

30 Testing in the Cloud: Strategy Critical Success Factors (CSF) What are the CSF for testing in the cloud? Automation? Skilled testers? Test lab/environment? How do your measure them? Post release evaluation? Number of bugs for the release Test lab setup and tear down?

31 Testing in the Cloud: Strategy

32 Roles and Responsibilities Developers: Unit testing, Integration testing Test Analyst: Implement testing in the cloud strategy Technical Test Analyst: Testers Customers: Acceptance testing IT department: POC to cloud provider, monitoring, availability Cloud providers: SLA, services testing

33 Lessons Learned

34 Lessons Learned Testing Environment is a legacy system Acceptability of partial migrations Selecting/Preparing test execution environment Configuration in the cloud is challenging The nature of the test cases False negatives

35 What Does it Mean for Testers? How would you test this?

36 Key Points Software testing is challenging activity SOA, virtualization, computational grid, and cloud computing bring efficiency in development process/consumer need Leverage these technologies in software testing Plan, start early, know your goals Understand the holistic system architecture Understand the migrated test data Establish communications with the vendor

37 Key Points Cloud computing is not hype!!! Provides the infrastructure to process Big Data It s changing the testing landscape It s changing the testing profession We have had similar shifts in the past Cloud computing is here to stay Brace for change in testing!!!

38 Food for thought!!! WSJ (Nov 6, 2015)

39

40 Thank You References: Jamsa: Cloud Computing: SaaS, PaaS, IaaS, Virtualization, Business Models, Mobile, Security Hausman: Cloud Essentials