Test automation as-a-service

Transcription

1 Test automation as-a-service A smart way to make automated testing more efficient Dr. György Réthy

2 About myself testing related aspects Graduated as Ph.D. in 1991 Hungarian Telecom (now a T-Com company), ISDN acceptance tests International end-to-end ISDN tests Ericsson R&D in Hungary, Coordinating conformance test development for 3G switching centers Technical product manager of Ericsson Test Competence Center Representing Ericsson in ETSI TB MTS Leader of the ETSI team developing the TTCN-3 testing language Test Automation as-as-service HUSTEF Page 2 (by )

3 Content Where we are today? The ICT domain goes virtualized TAaaS: exploiting the changes TAaaS insights Conclusions Test Automation as-as-service HUSTEF Page 3

4 The starting point Typical test automation activities This is the scope of this presentation Build & continuous Integration components TC implementation Installing environment Test Automation workspace Network configuration TC validation function tests, maintenance TC execution (regression) Test Automation as-as-service HUSTEF Page 4 SUT Note: SUT may be just SW running in a simulated environment, centralized servers etc., but it doesn t have significance at the moment

5 The starting point Typical structure of TA Solutions Test cases TA tool (IDE, execution, logging/test evaluation) System-specific test framework (SUT-specific libraries) Common part (interface & protocol support, common utilities etc.) User domain Tool provider domain 3 rd party tools/ dependencies Typically, there are at least 3 different players Test Automation as-as-service HUSTEF Page 5 OS HW IT domain

6 Cloudified SUTs SUTv-s can be deployed and orchestrated automatically Test tools are typically not designed for automatic deployment and configuration Remains a time consuming manual work Installing environment Test Automation workspace SUTv Deploy ment Config uration Test Automation as-as-service HUSTEF Page 6

7 Cloudifying the test environment Deploying and configuring the test environment Test Automation workspace SUTv Deploy ment Config uration Test Automation as-as-service HUSTEF Page 7

8 Calculates workflow TaaaS from the users perspective requests creation shares suspends, resumes deletes, etc. logs in & uses VM OS acces s tools environment settings Test Automation as-as-service HUSTEF Page 8

9 Test automation as-a-service A Opening Software the as a box Service (SaaS) platform for configuring and deploying test workspaces in the Cloud. Functional View Identity module authorization authentication Ericsson corporate directory Web UI User sets goal, orders workflow execution & follows status, manages workspaces, tools etc. Knowledge management framework Knowledge base Datastore Workflow generator Dependency resolution + user goals => generates workflow Tool dependency graph, Triplestore for users, workflows, data & domain descriptions etc. Openstack IaaS VM provisioning, LDAP access & authentication Tool information, logs, tool binaries * Green Color: Non-TAaaS components Test Automation as-as-service HUSTEF Page 9

10 Creating a workspace Test Automation as-as-service HUSTEF Page 10

11 Accessing the workspace Test Automation as-as-service HUSTEF Page 12

12 Managing workspaces Test Automation as-as-service HUSTEF Page 13

13 Tool database graph Test Automation as-as-service HUSTEF Page 14

14 Conclusions == Advantages Installation time of test automation environments TA toolset 1 TA toolset 2 Manual installation 4-7 hours ~1 hour TAaaS min. ~6,5 min. Stability of the test environment Efficiency Eliminates human errors, like missing/wrong version of dependencies Easy to add/remove tools, workspaces, change configurations Easy to share the environment (workflow) within team/project Reuse workflow components between organizations Saving HW/SW resources by creating workspaces on-line & storing Provides interfaces for continuous integration Can be used for development workspaces as well Test Automation as-as-service HUSTEF Page 16

15 Questions? Test Automation as-as-service HUSTEF Page 17