Testing Mobile Applications Quality Assurance of Applications for Cell-phones

Transcription

1 Testing Mobile Applications Quality Assurance of Applications for Cell-phones

2 Revision Sheet Date Author Version Reviewer 08 th April 10 Aravind Kunikullaya & Tejas Shinde 1.0 Mandar Joshi AFour Technologies Confidential Document Page 2 of 12

3 Contents 1. Overview Testing of Mobile Application....6 a. Necessity for Mobile Application Testing..6 b. Types of Mobile Application Testing Skills of a Mobile Application Test Engineer Case Study. 9 a. Android....9 b. Blackberry Conclusion References..12 AFour Technologies Confidential Document Page 3 of 12

4 Overview Cell-phones are an important part of life, people use it to look at company s presentations, reports, attend meetings apart from send out tweets, update their status on social networking sites and stay in touch with others. Cell-phones have paved way for advancement in technologies; the need of people has made companies to develop phones with unique and different features. The cell phones have decreased in size and increased in numbers over the years. As technology evolved so did the features available with phones E.g: Bluetooth, GPS, IR, GPRS, 3G, shake sensors, motion detectors and gravity sensors. Fact Wireless Revenues were more than 250 times greater in 2009 than in 2000(courtesy: cellphones.org) With evolving technology and features came a league of small and big developers with their homebrewed as well as professional applications. Some of them are very useful (like weather-bug for iphone); while some of them are just for entertainment. Initially applications were developed keeping the businessmen in mind, but recently they have been more global targeting a wide range of customers. As people started using these, it became necessary that they should provide some quality to its end-users. Even if an application has an important feature which customer needs, he/she won t use it, if it does not provide the expected quality. The website androlib.com estimates that there will be 7151 new applications and games in android market for end-users in the month of March All these applications and games need to be rigorously tested before they are released into the market. Nokia, Samsung, LG, Motorola, Sony Ericsson account for over 80% of cell phones manufactured AFour Technologies Confidential Document Page 4 of 12

5 Figure: Graph on estimated number of applications for Android phone. (Courtesy: androlib.com) AFour Technologies Confidential Document Page 5 of 12

6 Testing Mobile Applications: Necessity for Mobile Application Testing: There is a rapid growth in the mobile device market and hence in development of various mobile applications. Research work in development strategies resulted into many new ideas and technologies to develop high quality mobile applications. These strategies are bringing up new methodologies which have minimized the time of application development. Mobile applications are often outlined by an array of restrictions such as short development lifecycle to gain a significant advantage over others and difficulties to update once released. Hence, rigorous testing such as compatibility testing, performance testing, functional testing and usability testing is needed in order to ensure the quality of the application before releasing into the market. Types of Mobile Application Testing: 1. Compatibility Testing: While testing mobile application, major concern is compatibility. Mobile application mainly depends on OS and platform used by the handset such as Android, Symbian, Windows Mobile, J2ME etc, different UI used by different manufacturer, difference between graphical and physical keyboards, uncooperative drivers for graphics etc. So while testing mobile applications, all these compatibility issues must be covered to assure similar working of the application on different environments. Android itself has more 2. Performance Testing: than 5 OS Versions. Handheld devices are often subjected to the limited resources such as small processor and limited amount of memory so Performance Testing is important for Mobile Applications. Performance testing for Mobile applications mainly involves stress testing. Mobile applications may contain hidden exceptions, memory leakage and deadlocks that go undetected during the normal course of testing. Performance testing addresses all these problems. 3. Functional Testing: Functional Testing consists of validating the application against requirements. Functional testing mainly involves validating a set of inputs against expected results (i.e. set of outputs). Mobile applications are with less number of features, but they perform major and complex operations, so functional testing is vital in Mobile Application. It involves testing for integrity, validation and verification of data, inter and intra modular communication testing etc. The overall behavior of application is tested in this type of testing. AFour Technologies Confidential Document Page 6 of 12

7 4. Usability Testing: User interface plays a major role in customer satisfaction; more friendly and simple the application is, more it appeals and attracts the customers. Usability testing mainly focuses on exterior part the user sees. User Interface and the navigational conditions are tested in this type of testing. Thus, the UI testing is important for mobile application. 5. Comparison Testing: Comparison testing involves comparing the output of API with that of UI. In case of mobile applications, total business logic is on the server side and cell phones are just idle clients. So it is essential to compare the output of the API and UI, in order to maintain integrity. 6. Regression Testing: Mobile applications have short development lifecycle and so does testing. In order to make applications more stable, regression testing is necessary in Mobile Application. Regression testing helps the MATE to ensure the stability to the product even in short development lifecycle. 7. Security Testing: Some Mobile Applications ask the users to authorize themselves for using it, or in some other applications, user has to send some private data over network. The database is located at the server side of application, so when the application sends data through network, there might be a case when an intruder tries to gain access to sensitive information. If the application is sending data in plain text format, it is more vulnerable to the disclosure. When the application undergoes Security Testing, these problems are underlined. 8. Automation: Automation in mobile application testing enables testers to record their own test scenarios resulting into saving time, money and effort, keeping same or even increased accuracy and reliability in the quality of the application. While testing the Mobile Applications, automation helps in following areas: In reducing the man power requirement Test accuracy is increased due to minimization of manual errors Duration for the test lifecycle is minimized as it allows the tester to run scheduled tests in less time. Application coverage is more when a test pass is conducted. Different tools for Mobile Application Testing: QTP for Mobile 9.2 with M-eux add-on for Functional and UI testing on Windows Mobile and Symbian OS. TestComplete for Automated Functional and Regression testing on Windows Mobile TestQuest Countdown on Android, Blackberry, Brew MP, Symbian and Windows Phone. Device Anywhere for Functional testing on various platforms. AFour Technologies Confidential Document Page 7 of 12

8 Skills of a MATE The skills of a Mobile Application Test Engineer (MATE) can be broadly classified into three parts:- 1. Basic QA knowledge 2. Domain knowledge 3. Platform knowledge A MATE should have basic knowledge of QA, right from composing test specification document to the execution of test passes and logging bugs into bug tracking system. MATE should be able to ensure efficiency in the entire testing lifecycle, conduct testing tasks associated with development and maintenance of Mobile application. A MATE should have ample knowledge of the Mobile domain he/she is using. MATE should know the internal details of the domain, why that domain is useful or what is the purpose behind selecting it (e.g.: The tester should not try to install a Symbian application on a non Symbian phone, he/she should have enough knowledge that Symbian phone is platform dependent). Apart from having the domain knowledge, MATE should also know about platform he/she is required to test; Platform knowledge can be broadly split into two parts, the software part and the hardware part. Software: A MATE should have sufficient information about the details of the platform and the software A MATE should be well-versed with the different terminologies in the respected platform and should know the intrinsic details of the different platforms. Hardware: A MATE should have enough information about the hardware requirements when he/she is testing a Mobile application. He/she should have knowledge of the different requirements for compatibility of the application and test it on the respected hardware. Challenges faced by MATE Due to the different hardware and software changes a MATE has to conduct thorough testing on the application for its usability and functionality on all the different configurations. Some applications are network based; hence an all-inclusive test needs to be done on the application. Sometimes the hardware requirements are not met by the DUT, hence these create a challenge to a MATE. AFour Technologies Confidential Document Page 8 of 12

9 Case Studies 1) Android Platform: Overview In March-2010, a client came to AFOUR Technologies with a request to test his mobile application. The main user of this application would be a salesman who would go around with it installed on his phone and find a customer who is near to his location using the GPS service. The customer would be registered into a database which the application uses. The application would help a user in increasing his productivity. Testing This application was for an Android phone for platform 2.0 and above. AFOUR Technologies started off by quickly assimilating the concepts of this field towards testing of the application, the work environment was setup, which included downloading and setting up the OS(Ubuntu 9.10), Android SDK, the emulator for the cell phone and eclipse IDE. AFOUR conducted functional, performance, compatibility and usability testing on this application. Conclusion A fair number of bugs on the application were logged into the bug tracking tool. The difficulty faced was with the IDE which had a bug in it. AFour Technologies Confidential Document Page 9 of 12

10 2) Blackberry Platform: Overview In the year 2009, one of the AFOUR s clients had to perform functional testing for their product on Blackberry hand-held devices. The product is based upon certain key matrices pre-defined along-with specific formulae to calculate and display the key performance of stores in the form of reports. The recommended models to run the application were 9000 Verizon, 8330 Sprint, 9530, 8300 and The application was tested using the Blackberry Device Simulators that have the Mobile Data Service (MDS) feature, which provides Blackberry users the ability to access enterprise applications and existing corporate data stores. Testing The application consists of different reports named Ranking, Pie-chart, 4-Block & Trend. There are pre-defined matrices designed as per the client s requirements which are utilized in calculating the performance for stores and its higher units. The data from stores were an input to the system and calculations were done for the stores based on pre-defined formulae, this helped the corporate level managers to predict the performance of the stores and set-up certain sales targets to the stores. Also, the calculations for higher units like corporate, division, region and district are done based upon the rollup applied on the store level data. Conclusion The challenges faced in testing this application on a hand-held device were keeping a watch on the memory consumed for each application screen, the time required to load it, the calculations done and the alignment with reference to the available screen area. AFour Technologies Confidential Document Page 10 of 12

11 Conclusion According to International Telecommunication Union (ITU) reports there are going to be 5 billion cell phones all over the world by 2010 and all these phones have its unique features and different applications which will help the customer perform certain tasks. These applications should provide a better quality. The application needs to undergo rigorous testing procedures to provide better quality before they can be released into the market. An application which consumes all the resources of a cell phone or the one which drains the battery of a cell phone is of no use to the customer. A Mobile Application Tester should have enough knowledge about the domain and the platform apart from the basic testing knowledge. Today Mobile Application Development is facing multitude challenges due to the increased complexity of the application across different handsets, platforms, configurations. To test all of these criteria would require a very skilled and dedicated QA Team this is what AFOUR builds. AFour Technologies Confidential Document Page 11 of 12

12 References AFour Technologies Confidential Document Page 12 of 12