Accelerate. Innovate. Automate. Model Based Test Automation Avoiding the maintenance trap Hans Planken Vice President Alliances EMEA January 2015 2014 Tricentis GmbH. All rights reserved. 1
Agenda Automation level below 25% even after 30 years of software testing. Why? Do we need a different approach? 2014 Tricentis GmbH. All rights reserved. 2
Evolution of Complexity - Testing Challenges Complexity Mainframe Client/Server Web Mobile 2014 Tricentis GmbH. All rights reserved. COM_B18 3
What s the problem? A typical test project. Total Cost and Effort Risk Coverage 100% Test Case Specification If you don t know how much test coverage you have, you ll end up with far too many test cases! Not effective enough 50% Manual Testing (Execution) Automated Testing (Setup, Maintenance and Execution)? Automation levels are low because of it s difficult to automate! Not efficient enough 0% Source: TRICENTIS customer projects 2014 Tricentis GmbH. All rights reserved. 4
Tricentis Tosca Testsuite Target Applications Reporting & Dashboard Optimize Manage Automate Risk Coverage Optimizer Test Data Provisioning Model-Based Test Automation Virtualize Orchestrated Service Virtualization Technology Integration 2014 Tricentis GmbH. All rights reserved. 5
Tricentis Tosca Testsuite Reporting & Dashboard Optimize Manage Automate Risk Coverage Optimizer Test Data Provisioning Model-Based Test Automation Virtualize Service Virtualization & Orchestration Aligns business priorities to test planning & execution Generates the fewest test cases to maximize risk coverage Shifts focus from Test Coverage to Risk Coverage 2014 Tricentis GmbH. All rights reserved. 6
Synthetic Test Data Engine Total effort/costs 100% 75% Test Case Specification 50% Manual Test Execution Execution of Test Cases 25% insufficient control of test data Finding and Preparing Test Data 50-75% of manual execution costs! 0% Automation Source: Customer Projects TRICENTIS 2014 Tricentis GmbH. All rights reserved. 7
Tricentis Tosca Testsuite Reporting & Dashboard Optimize Manage Automate Risk Coverage Optimizer Test Data Provisioning Model-Based Test Automation Virtualize Service Virtualization & Orchestration Provides tailored test data basis for test execution Reduces effort in manual test execution by 40 50 % Stateful test data management is key for high automation rates 2014 Tricentis GmbH. All rights reserved. 8
Tricentis Tosca Testsuite Reporting & Dashboard Optimize Manage Automate Risk Coverage Optimizer Test Data Provisioning Model-Based Test Automation Virtualize Service Virtualization & Orchestration Enables extreme automation (rates > 90%) Covers UI and nonui test automation Enables seamless transition from manual to automated testing 2014 Tricentis GmbH. All rights reserved. 9
Tosca Testsuite Robust, Cost-Effective, Scalable Risk Management Engine: Higher Coverage, Fewer Tests. What makes it different? Model-Based Test Automation: Robust Cost-Effective Scalable 2014 Tricentis GmbH. All rights reserved. 10
Model-Based Automation Model-Based Test Automation Robust, Cost-Effective, Scalable 2014 Tricentis GmbH. All rights reserved. 11
What is the Problem? What keeps us from higher automation levels Efficiency: Why are automation levels so low? Prohibitive Maintenance of Test Scripts Missing or inadequate Object Recognition GUI and nongui tests require different teams/skills Autom. Tests require dynamic and thorough Test Data 2014 Tricentis GmbH. All rights reserved. MBT_B0 12 1
The Evolution of Test Automation 1 st Generation Record/ Replay FRAGILE Robust Cost Effective Scalable 2014 Tricentis GmbH. All rights reserved. 13
The Maintenance Problem with Capture/Replay Example: select invoice with the car registration number VIC-3456 Checkbox Name Reg Number Invoice John Smith NSW-1234 90035560 Jim Courier NSW-2345 90035561 John Cook VIC-3456 90035562 Tim Rebhorn VIC-4567 90035563 Dale Howard WA-5678 90035564 Frank Rose WA-6789 90035565 CheckBox Click, /usr/cntlcontainer/ shellcont/ shell[2]/chbx[1,3] Does your business team understand this command? Is this command stable? Conclusion: Capture/Replay does not work! 2014 Tricentis GmbH. All rights reserved. 14
The Evolution of Test Automation Script-Based Model-Based 1 st Generation 2 nd Generation Next Generation Record/ Replay FRAGILE Test Automation Frameworks EXPENSIVE Model-Based Robust Cost Effective Scalable 2014 Tricentis GmbH. All rights reserved. 15
Cognitive Test Cases: business language - not scripts! Requires only business SME skills Highly stable from a technical as well as from a business point of view Independent from technologies, HTML, Java,.NET, SAP, Manual execution, 2014 Tricentis GmbH. All rights reserved. 16
Example: Do you understand the business instruction? Do you remember it? CheckBox Click, /usr/cntlcontainer/she llcont/ shell[2]/chbx[1,3] select invoice with the car registration number VIC-3456 2014 Tricentis GmbH. All rights reserved. 17
Model Based Test Automation How does it work? 2014 Tricentis GmbH. All rights reserved. 18
Model-Based Automation Action Action Test Cases System Under Test UI Business Technology decoupling non UI Model(s) 2014 Tricentis GmbH. All rights reserved. 19
OneView: UI/nonUI; Robustness over Change CT ST SIT (E2E) UAT Test Cases in plain English Model SCAN & SYNCHRONIZE W/S HTML SQL JAVA.NET SWIFT XML HOST Systems CHANGE 2014 Tricentis GmbH. All rights reserved. 20
GWT HTML Web Services SWIFT 500 HP Petrol Diesel Gas Consistent User Experience & On-Demand Staffing TOSCA User Interface DB Test Logic Technology Base Engine decouples 2014 Tricentis GmbH. All rights reserved. 21
Technologies xbrowser Java Centura/Gupta Win32, C++, VB PowerBuilder, Delphi 3270, Emulators Word, XL, Outlook PDF-Documents SAP Siebel, Oracle Mobile.NET, WPF Databases WebServices, XML REST JSON Avaloq Adapter Adapter Adapter Adapter Broad coverage or technologies & applications Model Based Test Automation > 30 technology specific adapters Customerspecific extensions TOSCA allows for easy tests across any technology and/or platform! 2014 Tricentis GmbH. All rights reserved. 22
Traditional Automation Traditional automation from a manual to an automated Test Case Manual (Prose) Automated (Code) 2014 Tricentis GmbH. All rights reserved. 23
Only Tosca offers seamless test automation Model You decide! Traditional testing: two sets; typically out of sync! 2014 Tricentis GmbH. All rights reserved. 24
The Bottom Line Total costs Pre Tosca Cost 100% Test Case Specification 75% 50% Manual Test Execution Execution of Test Cases Post Tosca Cost Risk-Based Test Structure 25% Automation Finding and Preparing Test Data Poor automation coverage due to (i) maintenance trap (ii) insufficient control of test data Test Case Spec. Manual Testing Test Automation Test Case Design 0% Source: Customer Projects Tricentis 2014 Tricentis GmbH. All rights reserved. 25
Thank You! 2014 Tricentis GmbH. All rights reserved. 26