Test automation as-a-service A smart way to make automated testing more efficient Dr. György Réthy
About myself testing related aspects Graduated as Ph.D. in 1991 Hungarian Telecom (now a T-Com company), 1993-1999 ISDN acceptance tests International end-to-end ISDN tests Ericsson R&D in Hungary, 1999- Coordinating conformance test development for 3G switching centers Technical product manager of Ericsson Test Competence Center Representing Ericsson in ETSI TB MTS Leader of the ETSI team developing the TTCN-3 testing language Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 2 (by http://visitfinland.com )
Content Where we are today? The ICT domain goes virtualized TAaaS: exploiting the changes TAaaS insights Conclusions Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 3
The starting point Typical test automation activities This is the scope of this presentation Build & continuous Integration components TC implementation Installing environment Test Automation workspace Network configuration TC validation function tests, maintenance TC execution (regression) Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 4 SUT Note: SUT may be just SW running in a simulated environment, centralized servers etc., but it doesn t have significance at the moment
The starting point Typical structure of TA Solutions Test cases TA tool (IDE, execution, logging/test evaluation) System-specific test framework (SUT-specific libraries) Common part (interface & protocol support, common utilities etc.) User domain Tool provider domain 3 rd party tools/ dependencies Typically, there are at least 3 different players Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 5 OS HW IT domain
Cloudified SUTs SUTv-s can be deployed and orchestrated automatically Test tools are typically not designed for automatic deployment and configuration Remains a time consuming manual work Installing environment Test Automation workspace SUTv Deploy ment Config uration Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 6
Cloudifying the test environment Deploying and configuring the test environment Test Automation workspace SUTv Deploy ment Config uration Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 7
Calculates workflow TaaaS from the users perspective requests creation shares suspends, resumes deletes, etc. logs in & uses VM OS acces s tools environment settings Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 8
Test automation as-a-service A Opening Software the as a box Service (SaaS) platform for configuring and deploying test workspaces in the Cloud. Functional View Identity module authorization authentication Ericsson corporate directory Web UI User sets goal, orders workflow execution & follows status, manages workspaces, tools etc. Knowledge management framework Knowledge base Datastore Workflow generator Dependency resolution + user goals => generates workflow Tool dependency graph, Triplestore for users, workflows, data & domain descriptions etc. Openstack IaaS VM provisioning, LDAP access & authentication Tool information, logs, tool binaries * Green Color: Non-TAaaS components Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 9
Creating a workspace Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 10
Accessing the workspace Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 12
Managing workspaces Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 13
Tool database graph Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 14
Conclusions == Advantages Installation time of test automation environments TA toolset 1 TA toolset 2 Manual installation 4-7 hours ~1 hour TAaaS 12-22 min. ~6,5 min. Stability of the test environment Efficiency Eliminates human errors, like missing/wrong version of dependencies Easy to add/remove tools, workspaces, change configurations Easy to share the environment (workflow) within team/project Reuse workflow components between organizations Saving HW/SW resources by creating workspaces on-line & storing Provides interfaces for continuous integration Can be used for development workspaces as well Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 16
Questions? Test Automation as-as-service HUSTEF 2015 2015-11-19 Page 17