STeP-IN SUMMIT 2014 11 th International Conference on Software Testing June 2014 at Bangalore, Hyderabad, Pune - INDIA Mobile Performance Testing by Sahadevaiah Kola, Senior Test Lead and Sachin Goyal Senior Testing Engineer, NOUS Infosystems Copyright: STeP-IN Forum Published with permission for restricted use in STeP-IN SUMMIT 2014 in agreement with full copyrights from owner(s) / author(s) of material. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without the prior consent of the owner(s) / author(s). This edition is manufactured in India and is authorized for distribution only during STeP-IN SUMMIT 2014 as per the applicable conditions. Practices Experience Knowledge Automation Produced By Hosted By www.stepinforum.org www.qsitglobal.com
MOBILE PERFORMANCE TESTING Sahadevaiah Kola Sachin Kumar Goyal Table of Contents Introduction 2 Mobile Trends 3 Few Facts: Why Mobile Performance Testing? 4 Challenges in Mobile Performance Testing: 4 Mobile Performance Testing vs Types of Applications: 5 Mobile Application Performance Drivers 5 The Mobile Performance Testing Ways: 6 The Mobile Performance Testing Execution: An Alternative Approach 9 Recommendations: Mobile Apps Performance Optimization Techniques 10 Conclusion 12 List of Figures Figure 1: Mobile Phone Internet Usage 3 Figure 2: Estimated Mobile Phone users by 2017 3 Figure 3: Effect of Mobile users on the Performance of Application 4 Figure 4: Key Components Governing Mobile App Performance 5 Figure 5: Recording in Proxy Mode 6 Figure 6: Recording in NeoLoad Tunnel Mode 7 Figure 7: NeoLoad Settings for Tunnel Mode 7 Figure 8: Snapshot of HP LoadRunner Mobile Recorder app on Google play 8 Figure 9: Recording using Desktop Browser 8 Figure 10: WAN Emulation during performance testing 9 Figure 11: An alternate approach to Mobile Performance Testing 9 Figure 12: HP LoadRunner settings to use Perfecto script 10 Figure 13: YSlow Statistics for a mobile website 11 1
Introduction In a world of fast pace, it is essential to be in contact with the costumers even when they are on move. The mobile usage is increasing and more and more people are using mobile for browsing sites and buying products. According to a study, the total mobile users are expected to reach 5 billion by 2017 and the mobile internet usages will surpass the desktop internet usage for personal use. In light of these statistics, we should expect that in order to survive, every business will have to reach out to mobile. But mobile phones have their limitations - the display size varies from mobile to mobile; market is shared across multiple OS; RAM and CPU constraints; network used has much lower bandwidth. The challenge arises when mobile users don t approve of these limitations and expect the faster or same response from the mobile websites. Hence the need for performance testing on mobile device becomes even more critical than the desktop. The performance testing on mobile should not just concentrate on backend performance but the frond-end performance as well. Here, in this paper, we see the performance challenges for different types of mobile apps; tools & techniques that can be used to find performance issues and recommendations. Mobile Trends The Internet usage over mobile phones has increased drastically over last couple of years. According to a study, Internet usage over Mobile Phones, in July 2013, accounted for 17% of global web usage. Another study estimates the total no. of mobile phone users to reach 5 billion by 2017. Figure 1: Mobile Phone Internet Usage Figure 2: Estimated Mobile Phone users by 2017 2
Few Facts: Why Mobile Performance Testing? With growing no. of mobile web users, the expectations go up as well. There are few such studies mentioned below: Mobile users will wait for max 5 seconds for a website or application to load 74% will leave a mobile website if it doesn t load within 5 seconds 50% will exit an app if it doesn t load within 5 seconds 71% of global mobile web users expect websites to load almost as quickly or faster on their mobile compared to desktop 1/3 rd of the mobile users will go to competitor s site if the website or application is slow 80% of mobile web users don t want to try a website or application more than 2 times Challenges in Mobile Performance Testing: Innumerous no. of devices with different configuration Bandwidth variation Mobile specific sites (m.example.com) Frequent updates in mobile apps Mobile users keep the connection open Responsive web design Figure 3: Effect of Mobile users on the Performance of Application Mobile Performance Testing vs Types of Applications: When it comes to Mobile Performance Testing, the approach needs to be tailored based on the application type. We need to effectively target the processes and components involved in these different types of Applications. 1. Browser Based Apps The mobile browser based applications might perform differently when compared to the same application accessed over a desktop. There are bandwidth limitations, CPU, RAM and many other constraints which degrade the performance of the application. Again, to serve the mobile users better, many a times the mobile website architecture differs from the regular desktop website. To make the mobile website work faster the unnecessary elements and components are removed. Few websites are even launched with dedicated mobile sites which start with m-dot (m.example.com). Hence, it is equally important to replicate the mobile web users while load testing a website. In addition, it is important to measure the Web page rendering time as well on the target device. 2. Native Apps Native applications are written specially for a device/platform and won t work for another device/platform. Native apps might or might not use Internet. These applications can be accesses quickly. This calls for the thorough performance testing on the platform the application is built for as well as on the devices in the target platform. 3. Hybrid Apps Hybrid apps are just web apps with native wrapper. The application is developed using Web technologies and then wrapped in a platform-specific shell that allows the app to be installed just like a native app. This reduces the effort needed to develop the app for each platform and device. Only different wrapper needs to be developed for the target devices. Mobile Application Performance Drivers The Performance test of mobile applications should target every component in the application architecture. The key components for a generic mobile application are represented in the figure below. Figure 4: Key Components Governing Mobile App Performance 3
The performance of a typical mobile application depends on three key factors: Back-end processing time: This is the total time taken by the request to reach the server, response generation time by server and the time to deliver the response back to the client. This time is affected by the total load on the server, network bandwidth, data transferred and the external component (if any). Network Emulation: Network varies on a wide range when it comes to mobile internet usage and this makes network emulation important for mobile performance testing. We need to effectively emulate the network condition to replicate real-time traffic during performance testing. Content rendering time at device: The time it takes to display the response delivered to the client on the device is measured here. The platform type, configuration of device and other hardware configurations affects this time. This time should be handled using on-device monitoring tool. The Mobile Performance Testing Ways: Recording: Based on the increasing importance, various Performance tools have started supporting Mobile Performance testing. Mobile applications can be recorded in many ways: Directly from the device itself a. Using proxy mode b. Using tunnel mode c. Using Native app (recording application) HP LoadRunner VuGen mobile app From an emulator From a desktop web browser (for pure HTML applications) a. Proxy Mode: NeoLoad and JMeter support recording user actions directly from the mobile. We need to set up proxy of the recording server (NeoLoad/JMeter machine) into the device and it records all the activities done on the device afterwards. Figure 5: Recording in Proxy Mode b. Recording in Tunnel Mode: What if your application cannot be used via proxy or the mobile device doesn t support proxy? NeoLoad has come up with recording capabilities in tunnel mode. On one side, the Tunnel simulates the web server for the browser, and on the other side, it processes the requests and responses with the web server, allowing the Recorder to catch the transactions for the test scenario. This mode makes it possible to record any application, either web or native, standard or customized, for any platform in any version. Figure 6: Recording in NeoLoad Tunnel Mode Figure 7: NeoLoad Settings for Tunnel Mode 4
c. Recording using Native Recorder: HP LoadRunner has launched its own mobile application, HP LoadRunner Mobile Recorder, to record directly on the mobile. The device must be rooted to record. Figure 8: Snapshot of HP LoadRunner Mobile Recorder app on Google play Recording using Desktop Browser: With NeoLoad, we have an option to emulate desktop browser as a browser on a specific mobile device Figure 9: Recording using Desktop Browser WAN Emulation: While performance testing mobile applications, we need to simulate different networks (e.g. 2G, 3G, Edge, WiFi etc.) to reach the real-time traffic. We can enter the real time bandwidth, latency and packet loss values using WAN Emulation. Figure 10: WAN Emulation during performance testing The Mobile Performance Testing Execution: An Alternative Approach The alternative approach of Performance testing of mobile application is as follows: Load will be generated on server In parallel, the mobiles performance will be checked with single user Figure 11: An alternate approach to Mobile Performance Testing 5
Using Perfecto mobile with HP LoadRunner provides the above solution at one platform. Perfecto mobile solutions provides us an option to use a real device to record the script and the same script can be used for the load generation on the server using HP LoadRunner loading the server single user test on the device under test Figure 12: HP LoadRunner settings to use Perfecto script Recommendations: Mobile Apps Performance Optimization Techniques Here are few performance optimization techniques for the mobile apps: Use less no. of HTTP request between the client and server; this will greatly improve the page load. Use same HTML code regardless of device; to change the user interface for different devices use CSS media queries to specify which CSS rules should apply for the browser displaying the page Although browser caching is not very efficient for mobile application but HTML5 web storage is an alternative of browser caching. In-lining resources instead of using them as linked references can greatly increase the page loading time. Bandwidth consumption can be decreased by compressing and reducing page resources to make it faster. By resizing the images according to mobile screen can not only reduce the page size but also improve the processing power. Figure 13: YSlow Statistics for a mobile website 6
Conclusion Mobile Applications Usage is increasing exponentially day-by-day and so the need for performance testing of Mobile applications also needs to be taken care to meet competitors. Simultaneously better user experience will remain most important factor to attract and retain mobile users. The basics of load testing of mobile apps are not very different from traditional desktop apps. However performance engineers need to understand all types of mobile apps architecture, their fundamental differences completely (like bandwidth, processing power, screen size etc.) from desktop apps to test all the application parameters successfully. 7