Software Quality Analysis with Cloud Challenges and Approaches. Progress Software, Hyderabad, India

Transcription

1 Software Quality Analysis with Cloud Challenges and Approaches Dr Ganesh Iyer, PhD, Santhosh Kumar Soma, Kiran Kumar Angara, Principal QA Engineer Senior QA Engineer QA Engineer, Progress Software, Hyderabad, India

2 Masters & Ph.D. from National University of Singapore Interests: Kathakali, Traveling, Photography, Cooking GANESHNIYER Master of Computer Applications from JNTU, Hyderabad Hands-on experience: functional/non-functional testing with Cloud Interests: Cricket, Blogging, Listening Music SanthoshSoma Masters in Artificial Intelligence from University of Hyderabad Working on Rollbase OpenEdge Integration for last two years Interests: Dance, Traveling, Surfing KiranAngara

3 Agenda What is Cloud? How Software Testing got affected by Cloud? Additional Test Dimensions by Cloud Conclusions 3

4 ENJOY SERIOUSLY FAST DEV DEPLOY ON YOUR TERMS LIBERATE IT GO MOBILE CODE YOUR APPS OR NOT MODERNIZE LEGACY APPS 4

5 Cloud Delivery Models SaaS PaaS IaaS 5

6 Software Test Challenges With Cloud Paradigm Shift Web-driven SaaS applications Everything in a few mouse clicks Everything is Agile Frequent releases / Frequent changes No SRS documents Availability High Availability and Resiliency are key Seamless upgrades Everything is shared Publically shared resources Multi-tenant penetration testing 6

7 7 Cloud Test Dimensions

8 Cloud Test Dimensions in a Nut-shell Security Integration Performance Cloud Test Dimensions API & Connectivity Resilience Compatibility 8

9 Security Security Automation: Traversal vulnerability Websecurify User Access / Roles Web UI Security User access/ roles Identity federation management ZAP tool Scripts Vulnerabilities Multi- Tenant DB Multi-tenant penetration 9

10 API Invocation API Invocation Connectivity and invocation API load API security Multi-tenancy Automation: SoapUI Node.js PySys In-house Frameworks 10

11 Performance Performance Automation: Time to deploy Multi-tenancy Latency Telerik Test Studio NeoLoad Stress Reliability Deployment Performance Connectivity and reliability with 3rd parties JMeter Load Performance Testing 24 x 7 Reliability and availability 11

12 24 x 7 Testing Key highlights Tests Could be Existing tests Batch Script Decides frequency Ant Script Decides parallelism Reporting Kendo UI controls 12

13 Resilience and Failover Ability of a server, network, storage system, or an entire data center, to continue operating even when there has been an equipment failure, power outage or other disruption Software error Hardware error Human error Natural Disaster 13

14 Resilience and Failover Resilience IaaS Making one instance down Making multiple instances down Application Interface (UI / API) ELB HA Instance 1 Disaster Recovery Logs Stubs for 3 rd Parties HA Instance 2 14

15 Compatibility Compatibility Accessibility Globalization and localization Compatibility under different situations Automation: Telerik Test Studio Telerik DeviceCloud Sahi Selenium 15

16 Testing Mobile Apps in Cloud Telerik DeviceCloud >300 Devices 29 Manufacturers OS versions

17 Integration Integration P2P Integration System Integration End-to-End Integration SSO Customer Scenarios 17

18 A Typical Integration Scenario in Progress Pacific OData REST Service Telerik Mobile OData REST Service QA Dashboard App OpenEdge DB DMZ REST Wrapper 18

19 Typical End-to-End Scenario LDAP Registration Land on product dashboard Community Export Compliance, Marketing Click on subscription details Choose a plan Billing Marketing, Billing, Support Checkout process Generate invoice Billing, Invoicing Metering Billing Product usage Log-out LDAP 19

20 Integration Test Challenges and Approaches Multiple systems behave differently No automated deletion of test data in some systems Unpredictable delays in updating various systems Different types of environments for testing 20

21 Integration Test Automation With Cloud Test Logger Product 1 Product 2 Product 3 Test Manager Run Time Test Framework UI Framework Stubs for 3 rd Parties API Calls Test Repository Test suite Libraries (APIs) Output Repository Telerik Test Studio as the UI framework, C# as the Runtime framework 21

22 22 Additional Test Dimensions with IaaS Repercussions on SaaS and PaaS

23 Cloud Test Dimensions in a Nut Shell Security Performance Resilience & Failover Compatibility API Connectivity Integration Traversal vulnerability Time to deploy Making one instance down Accessibility Connectivity and invocation P2P Integration User access/ roles Multi-tenancy Making multiple instances down Globalization and localization API load System Integration Identity federation management Latency Disaster Recovery Compatibility under different situations API security End-to-End Integration Multi-tenant penetration Connectivity and reliability with 3rd parties Multi-tenancy SSO Reliability and availability Customer Scenarios 23

24 Automation Frameworks Functional (Web/Mobile UI) Performance API Invocation Integration Security Telerik Test Studio, Telerik DeviceCloud Sahi / Selenium scripts Telerik Test Studio for Load / Performance Testing Neoload, JMeter SOAP UI / Node.js / PySys Telerik Test Studio (with C#) TestNG + Java + Sahi combined approach Websecurify ZAP Tool 24

25 Conclusion With Cloud, QA has Unique opportunities & Challenges New test dimensions Security, API Invocation, Integration, etc. Faster test execution Agile Leveraging automation tools and develop new automation frameworks 25

26 26 Cloud & Beyond

27 Backend Services Mobile M2M Activity SMAC Social Marketing Social Activity Voice of Customer Capacity 27

28 28 IoT or IoE?

29 29 Feedback Time

30