Challenges and Pains in Mobile Apps Testing

Transcription

1 Challenges and Pains in Mobile Apps Testing Sales office

2 Table of Contents Abstract... 3 Mobile Test Automation... 3 Challenges & Pains... 4 EZ TestApp Concept and Elements... 5 About TenKod Ltd Sales office

3 Abstract Smartphone applications have the power to change our world on a daily basis. This relatively new sector is positioned to propel online industries to new heights and be a significant engine of growth in today s global economy. However, the hurdles and barriers software developers and testers face are extremely challenging. The online economy demands its community of developers create high quality applications in as short a time as possible, in order to shorten the rout to market; at the same time the complexity of these new applications is evolving. The diversity of mobile platforms, combined with ongoing demand for quality, generates continuous requirements for Test Automation Processes (TAP) - - solutions that will improve a business productivity and the overall performance of its organization. Currently there is not a single offering out there, which completely satisfies the mobile Apps test automation process from start to finish. Installing and configuring the open source products is a tedious task; commercial tools often rely on proprietary techniques, resulting in an increased number of instabilities for the automated tests produced. One should be extremely careful when looking or the right tools for mobile Apps testing, as often there are hidden pitfalls. Mobile Test Automation Every day new and exciting software testing trends emerge to assist QA teams to tackle the challenges of mobile Apps testing (e.g. Crowd testing, device sharing, etc.), and demonstrate a significant improvement compared to earlier years. Having said that the cost associated with resolving challenges and issues has increased dramatically in terms of time and route to market, thus only critical issues are being tested while the rest is being neglected; unfortunately, as a result, these apps are not maximized and in many cases even generates a negative impact for a business. The TAP provides a comprehensive coverage for the complete Testing Life Cycle (TLC), enabling QA teams to: improve the quality of the deliverables, rise to the challenge of rapid and frequent development cycles and play a key role in the Sales office

4 success of their organizations. Automated testing for mobile apps does not cancel exploratory testing procedures; on the contrary, such processes are required and should focus on processes not covered by TAP and function as a complementary component to the overall testing strategy. Challenges & Pains A bit of research about the testing process quickly reveals the existence of a number of open source and commercial solutions. However this sector is - - to the least of it - - complex and does not provide comprehensive and focused solutions to the challenges and pains, which are being experienced on an ongoing basis by an organization s apps development and testing teams. 1. The diversity of mobile platforms, the ongoing demand for quality and the increasing complexity of mobile applications generates continuous need for long- term TAP solutions. 2. Essentially test automation is programming, so basic knowledge in common programming languages, such as Java, C#, JavaScript, etc. is required. This ultimately raises the entry barrier for newcomers 3. The test automation tools should be similar to (if not identical as) the tools used by the development teams producing the mobile applications to be tested. However, this is not the case as most of the mobile test automation tools currently on the market are either designed to run in web browsers or as standalone desktop applications. 4. The diversity of mobile platforms already implies the same variety in terms of development and testing infrastructure. 5. The existing commercial offerings, coping with the complicated and compound Integrated Development Environment (IDE), provide proprietary ideas and concepts that are inapplicable in any other context. This approach makes it even harder for test automation engineers to ramp up and use these tools effectively. 6. Proven Agile development and testing practices mandate that automated tests are most beneficial to organizations, when updated and run regularly as close as possible to the involved development teams. This allows for issues to be resolved in a timely manner, at the lowest possible cost. Unfortunately, the testing solutions out there still encourage mostly post- production testing cycles. Continuous Integration (CI) is a chimera. Open source test automation Sales office

5 tools do not consider this approach and commercial offerings fail to provide easy and intuitive approach - - mostly due to the proprietary concepts implemented as part of such kind of software. 7. Multi- level testing is another important component needed to create an effective mobile test automation process. This is basically a logical (and often technological) separation among tests, ensuring that a number of non- functional requirements are consistently met by a tested mobile application. In this sense, depending on the specific requirements for any given application, one may need to pay special attention to Security, Performance, Accessibility, API, etc. to create the necessary tests. The multi- level testing concept increases reliability and maturity of tests. It also gives them sharper focus and makes it easier for both development and testing teams to better understand and react to test failures, thus increasing the quality of the product. Although extremely useful, most of the existing mobile test automation tools and frameworks do not use this technique at all. EZ TestApp Concept and Elements TenKod EZ TestApp fits seamlessly within an organizations existing development and testing landscapes and makes tedious automation tasks a breeze, while practices, such as vendor lock- in, device jail- breaking or application instrumentation, are strictly forbidden. How is this done? Technically the product is based on cutting edge, industry- standard open source components, which guarantee for the right degree of pluggability, extensibility and standard compliance. The ultimate goal of EZ TestApp is to create a flawless and simple mobile testing experience from start to finish, independent from the mobile platform. The solution is easy to install on Mac OS and Microsoft Windows. It is just a matter of using a one- shot, wizard- based, installation procedure and it is ready to use. The entry point towards test implementation is the TenKod Studio an IDE based on the award- winning Eclipse IDE. Creating a test project is just a matter of few clicks. Every test project is seamlessly integrated with Apache Maven, for robust dependency management and Continuous Integration (CI). Running tests Sales office

6 using the market leading CI Platform, Jenkins, for example, requires little effort to create the corresponding CI job in a fully standard manner. It is also quite easy to reuse any given test in various contexts because the EZ TestApp provides test parameterization support out of the box. Parallel test execution needed? Jenkins offers a number of approaches towards achieving parallelization under any circumstances. The usage of a source control system is mandatory? Well, welcome to CI - - now everything is possible. EZ TestApp tackles the expensive and ineffective post- production test cycles by building up a safety net for the sake of regression avoidance throughout the whole development and testing phases. Developing a test implementation process using TenKod Snooper has never been easier. Snooper is the unique visual test- recording tool, which facilitates mobile testers with a better understanding of the structure of the tested applications. Snooper deploys the application under a test on the real device, simulator or emulator, and provides a visual representation of this application in the TenKod Studio. From there on, one is able to interact with the application as is normally done on the real device, e.g. use gestures like tap and swipe, sending text to search fields, selecting values from lists, etc., which are all recorded. When a given application scenario is executed using Snooper, a thorough Selenium WebDriver compatible test code is automatically generated. In other words, EZ TestApp generates a completely compatible WebDriver test code with zero development efforts by the test developer. If needed, the generated test code can later be changed, according to the test developer s requirements. The recorded scenarios can also be replayed by just running the respective WebDriver test implementations as JUnit tests. They follow well- established industry prescriptions for increased maintainability, reusability and extensibility of automated tests. Generic Page Objects are used in order to hide the complexity of interacting with mobile applications. The EZ TestApp test runtime, based on Selenium for mobile, takes screenshots automatically, whenever a test case fails to execute correctly, allowing the investigation process to start immediately. As mentioned, it is also capable of executing parameterized tests and generating thorough test execution reports. These can be reviewed both in the TenKod Studio and Jenkins. Sales office

7 Image 1. TenKod Studio running Snooper The generated code is self- explanatory simple to read, understand and modify. The sample test bellow, illustrates an example of an EZ TestApp Test Case. Note the clean code, lack of any abnormal, non- test related constructs in the automatically generated test code. Image 2. EZ TestApp Test Code automatically generated by TenKod Snooper Sales office

8 About TenKod Ltd. TenKod s vision is to make mobile apps developers and testers life better! TenKod was founded in 2013 because of the ongoing challenges within the heterogeneous and complex continuous integration development and testing landscape in the mobile apps arena. TenKod addresses these challenges by providing a simple solution, executable on all operating systems and on every application layer. TenKod s flagship product, EZ TestApp, enables vast and fundamental integration between development and testing teams providing them an intuitive, cost effective solution, which covers the testing lifecycle. Sales office