The Hitchhiker's Guide to Mobile Apps Test Automation Galaxy

Transcription

1 The Hitchhiker's Guide to Mobile Apps Test Automation Galaxy TenKod EZ TestApp Technology Sales office TenKod Ltd.

2 Table of Contents Abstract... 3 Test Automation for Mobile Apps Challenges and PAINS... 3 TenKod EZ TestApp Architecture - Guidelines - Tips... 5 About TenKod Ltd References Sales office TenKod Ltd.

3 Abstract The field of mobile test automation is complex and challenging by definition. Mobile technologies are changing and evolving on daily basis. However there are tools available to help businesses test their applications, making sure they are ready for the marketplace; one such answer for these organizations is the TenKod EZ TestApp solution for testing Android and ios applications. In order to understand the philosophy behind the EZ TestApp solution and unique values offered to businesses, this report will address the challenges of mobile test automation, as faced at any stage of the corresponding life- cycle. We will see how they are tackled by the underlying software architecture as defined by TenKod. Test Automation for Mobile Apps Challenges and PAINS Most of the development processes, which are part of the mobile test automation, could be perceived as formal and repetitive. It is well illustrated by the principal mobile Test Automation life- cycle (as depicted on the diagram below). It will also help us better understand the numerous technical considerations, challenges and issues relevant for each and every phase. Image 1. Mobile Test Automation principle life- cycle Sales office TenKod Ltd.

4 Once you plan your test cases that are related to your mobile application, you will need to implement them. At this point in the process, you will most likely face the first major challenge. What kind of tools do you need in order to automate your scenarios? Commercial? Open source? All of them have pros and cons and none provide an easy, intuitive automation experience from start to finish. For example, most of these solutions don t answer the question: where would be the most appropriate place for to start test implementation process? If you are experienced with Web test automation, you will probably find the approach of using browser- based tools familiar. You will soon realize that this is - to the least of it - inconvenient for mobile test automation, though. You may then switch to some standalone desktop applications, but you will sooner or later need to modify your tests and you will figure it out that again this is not what you are looking for. Even, if you manage to automate your scenarios in some way, you will still be missing the important capability of having a well- known and standard testing framework to facilitate your test development, execution and reporting. Unless you pick one and make your scenario implementation fit well with it. Of course you can always trust many of the commercial offerings and their proprietary testing frameworks, however, option this takes you another step closer towards vendor- lock in and soon you will be lost. The reality is that the test implementation process may become a hard, tedious and time- consuming assignment, unless hundreds of person days are available at your own disposal. However, worldwide deadlines tend to become tighter and delivery dates get more and more pressing, while mobile applications tend to become more and more complex. Once you manage to implement your tests you will need to run them regularly. Here is the next challenge that you will need to overcome. Eventually you will conclude that Continuous Integration (CI) will save the day and your project, but in most cases you will be again on your own to figure it out what to do next. Sometimes you may be so locked in with proprietary testing solutions that such an approach might already be a lost cause for you. Sales office TenKod Ltd.

5 And what is a test run without decent reporting? Without proper documentation, any of your tests performed are a pure waste of system resources and CPU time? Okay this might be an exaggeration, but you will need to take care of this part of the process somehow. We have not even discussed some of the common pitfalls that huge number of mobile testing solutions will get you in, if you are not careful and cautious enough. Beware of jailbroken physical devices used for testing and solutions that promise miracles, if you just agree to instrument the application you would like to test. These are huge threats to you and your testing strategy as they seriously alter the runtime environment of the tested application, which often invalidates findings based on test execution. There are many other challenges and difficulties, which you will probably face until you get to a stable and reliable mobile test suite that is worthy investing in. Last but not least, you will face a whole new class of challenges once you try to automate the same application for both Android and ios, for example. You will soon find out that the diversity of mobile platforms also means there are a huge number of tools, concepts and technical issues that you will face. TenKod EZ TestApp Architecture - Guidelines - Tips The TenKod EZ TestApp takes a unique approach in order to tackle all these challenges. It uses cutting edge open source components as a technical basis (refer to the TenKod EZ TestApp software architecture below) and builds unique and homogenous end- to- end test automation experience independent of the mobile platform used underneath, thus leaving you with more resources to define and implement value- added test cases. Sales office TenKod Ltd.

6 Image2. TenKod EZ TestApp high- level architecture 1. Easy and intuitive UI test creation Test automation is effective in terms of regression avoidance and adds value when automated tests are executed as close as possible to the developed and tested software solution. This basic principle also applies to applications for smart mobile devices. In order to make the best of it we deliver the TenKod Studio as part of the TenKod EZ TestApp - the IDE for mobile Test Automation. Based on the industry- standard open source Eclipse IDE, the TenKod Studio provides a rich and intuitive user experience for test implementation and local test execution. Creating a test project for automating either ios or Android UI tests is a breeze. Using the Eclipse Maven integration layer and the TenKod Repository, one quickly creates Maven- enabled projects from the most up- to- date archetypes for all of the supported mobile platforms. Apache Maven - - being one of the most Sales office TenKod Ltd.

7 robust build automation systems - - guarantees a fluent integration with any Continuous Integration (CI) system on the market. Every test project can be arranged in minutes and each contains the TenKod Expense Tracker mobile app (implemented for the mobile platform) and a set of sample tests; this allows experienced mobile test automation engineers to easily ramp up and proceed with producing mobile tests using the outstanding refactoring capabilities of the Java Development Tools in the TenKod Studio. Any source code created by TenKod, as part of a test project, is covered by a variety of JUnit tests so that you can build up the best mobile testing suite, satisfying your own needs in a safe and regression- free manner. Implementing and running mobile tests in the TenKod Studio is easy and intuitive as JUnit is used as a test execution framework and Selenium WebDriver for mobile (Appium) ensures there is proper interaction with the tested application. The TenKod EZ TestApp is based on well- known and established test automation concepts and open source projects with high adoption rates, so that its learning curve is as smooth as possible. In addition, the TenKod EZ TestApp uses some advanced object mapping techniques, which make it possible for you to produce much more maintainable and durable test implementation processes. For example the, TenKod EZ TestApp can create testing solutions using image recognition for object mapping purposes, for example... But what if you are not a Java expert yet? Or you do not know Selenium? Or what if you are not aware of the tested mobile application to the extent necessary for implementing automated tests on your own you don t know a lot about its structure, internal component identifiers, names and so on and so forth? TenKod Studio features make the process easy for all levels of testing experience with its award- winning TenKod Snooper tool. The Snooping is a mobile application that reveals all these details and will also help you implement new tests in no time. Forget about long test automation cycles. What used to take months now takes minutes! Sales office TenKod Ltd.

8 Snooping, the TenKod Expense Tracker application is a good starting point. Once you get the hang of it you could just put your own mobile application in a test project created in the TenKod Studio and use Snooper. When TenKod Snooper is started, you are presented with a visual graphical representation of the tested application. This graphical representation allows you to interact with your application. TenKod Snooper uses the Selenium WebDriver under the hood and this makes it possible for you to interact with any tested mobile application in a highly unified manner - independent of mobile platform; this means you are able to click, tap, swipe, rotate and what not either the whole application or just some of the available UI controls. Each and every interaction is recorded by default, so that you can easily define your own testing scenarios. And what is a test scenario without some added- value testing verifications? Using the rich modeling capabilities of TenKod Snooper makes it possible to easily add assertions while recording scenarios. Once you are ready with a recorded testing scenario you could easily export it as a mobile automated test, similar to the sample tests delivered with the used test project. 2. Easy and reliable execution of test cases: Ok, now that you have your mobile automated tests implemented how can you run them? Being standard JUnit tests you can execute them in the TenKod Studio with a couple of clicks. However, if you have some experience with Eclipse and JUnit, you may already know how to do this. By default tests are executed using the emulator/simulator of the corresponding mobile platform. You may need to do this using physical devices. You just need to configure your tests by altering a single line and you are ready to go. Although this kind of configuration can be done without source code modification, test parameterization techniques are available out- of- the- box with the TenKod EZ TestApp. This makes your tests configurable from the outside without any source code modifications, which certainly improves their maintainability and reduces lots of efforts on your side. Sales office TenKod Ltd.

9 If you have experience with other solutions for mobile test automation, you may ask some questions, such as: Whether the tested mobile application is instrumented prior to test execution? Whether the physical devices need to be rooted/jail- broken? Thanks to the software architecture presented above the answer to these questions is: no. Simply put, your tests will be more reliable, relevant and last but not least - accepted by any development or testing organization. Of course, having your mobile tests as part of Maven- enabled test projects makes it possible to run them easily outside the TenKod Studio. 3. Easy integration in mobile application development lifecycle (CI): When exiting the test implementation phase, a test automation engineer is faced with a new challenge of achieving end- to- end test automation - - the test execution phase. During this phase, the typical requirements are to execute the automated tests as often as possible in an automated manner and bring it as close as possible to the development lifecycle of the application; TenKod EZ TestApp is able to execute automation testing on each and every change submitted in the codebase of the application under development and thus enriching its Continuous Integration (CI). Usually the automated tests are bundled in a test suite and physically hosted in a test project that depends on many other components comprising the complex test environment needed for the successful execution of these tests. So you need an advanced dependency management system to take care of all this stuff in a transparent and easy way. You can configure the automated tests to be executed with a single simple command that will can manage the complex test environment from scratch, ensuring the necessary isolation level for the successful, free of infrastructure issues or polluted test environment, automated test execution. Maven This wonderful piece of software was created by the Apache Software Foundation in 2002 and has been perfected for more than 12 years. It is written in Java, therefore it is capable of running on many different operating systems, Sales office TenKod Ltd.

10 and distributed under Apache License 2.0 license type. With its plugin based architecture and a large number of productive maven plugins you can use and customize it in the way you want. It is well established and heavily used in leading large software development companies and successfully takes care for automating the build, packaging, automated test execution of complex software solutions. It is also integrated and can be directly used in Eclipse and in one other important component, which we will shortly present. So having our best of breed component called Maven which takes care to successfully execute our test automation in a simple and easy way, the next thing on your agenda is to model your test execution schedule or even better integrate it with TenKod EZ TestApp s mobile application under test development lifecycle, aka Continuous Integration server (CI). A very bright example of such an implementation is called: Jenkins The development of this open source component started in 2004, and has been perfected over the last 10 years, and supported by 567 project members. It is written in Java and distributed under MIT License type, which makes it, actually a free, open source software. It claims to have 70% of the market share of CI servers. As well as Maven, Jenkins is also with plugin- based architecture with a huge library of plugins ready to be installed and run. It can be installed and run in a matter of seconds. From then on the only thing left is, with several clicks and nice looking web interface to model the test execution automation, execute it execution on regular basis, or taking advantage of the capability to execute test automation on each commit in the version control system of the mobile application under test, achieving with easiness our test automation execution and integration in mobile application CI. 4. Standard and easy to comprehend reporting of test execution result: Once the test automation engineer is done with the test execution phase, to consider it a job done they must create a report of results of the automated test execution. In developing TenKod EZ TestApp, we picked the right technologies when coding and executing our test automation this is now a very easy part to achieve and Sales office TenKod Ltd.

11 more or less comes as a reward to the choice we ve already made. We based our test automation implementation on JUnit test automation framework which already takes care of producing fine grained reports for each and every executable test we have implemented. The fine grained report contains stack traces and user readable messages which are necessary to understand, in case of a test failure, what exactly has gone wrong and where in our test implementation. Combining this with the out- of- the- box capability of TenKod EZ TestApp s mobile Testing Runtime capabilities, it is very easy to attach a failing test a screenshot of the mobile screen taken at the moment the test is failing - - priceless information isn t it? Last but not least the detailed reporting can be aggregated and presented in a very comprehensible way in a single screen. For this feature is a benefit because of the fact that we use Jenkins as our test automation executor. With Jenkins it is very easy to see the total number of executed tests: Image 3. Test reporting in Jenkins Dashboard Sales office TenKod Ltd.

12 With one quick look you can see which tests are failing, how much time it took for the test execution, etc. This as a top level of the aggregation, then using the drill down capabilities one can dig deep in the details: Image 4. Drilled down test reporting in Jenkins Dashboard You are able to go over each and every individual failure, take a look at the screenshot, read the human readable message specifying the concrete reason for the test to fail. This can save a lot of time, removes any chance of error prone test reporting, and helps the test automation engineer cope with a large test automation collection, as needed to fully cover the mobile application(s) under test. Sales office TenKod Ltd.

13 About TenKod Ltd. TenKod s vision is to make mobile apps developers and testers life better! TenKod was founded in 2013 because of the ongoing challenges within the heterogeneous and complex continuous integration development and testing landscape in the mobile apps arena. TenKod addresses these challenges by providing a simple solution, executable on all operating systems and on every application layer. TenKod s flagship product, EZ TestApp, enables vast and fundamental integration between development and testing teams providing them an intuitive, cost effective solution, which covers the testing lifecycle. Sales office TenKod Ltd.

14 References 1. Apache Maven 2. Apache Maven Project 3. Apache Maven Project Intro Jenkins 6. Jenkins Documentation 7. Apache Software 8. Apache 9. Apache Maven Plugins 10. Maven in Eclipse 11. Eclipse 12. MIT 13. Jenkins plugins JUnit Sales office TenKod Ltd.