Mobile Application Testing

Transcription

1 Mobile Application Testing Mobile applications are booming,so mobile apps are big business, but with this hyper-growth comes the challenge of releasing apps that are stable on multiple portable devices and platforms. Unfortunately, when it comes to mobile apps, quality assurance can be a complex, time-consuming and costly task. This can be especially true if you rely only on manual testing or on traditional tools for testing. Mobile Application testing is a process in which Mobile applications are tested for its functionality,usability and consistency. Mobile devices differs from screen sizes,input methods with different hardware capabilities.there are different Mobile Operating Systems in the market. The major ones are Android, IOS, BREW, BREWMP, Symbian, Windows 7.5 Mango, and Blackberry (RIM). Each operating system has its own limitations. Testing a single application across multiple devices running on the same platform and every platform poses a unique challenge for testers. There are over 400 mobile network operators in the world;out of which some are CDMA(Code division multiple access), some GSM(Global system for Mobile communications).each network operator uses a different kind network infrastructure and this limits the flow of information. As devices differ in keystrokes, input methods, menu structure and display properties a single script doesn t function on every device.

2 Types of Mobile Application Testing Functional Testing: Functional testing ensures that the application is working as per the requirements. Laboratory Testing: Laboratory testing, usually carried out by network carriers, is done by simulating the complete wireless network. This test is performed to find out any glitches when a mobile application uses voice and/or data connection to perform some functions. Performance Testing: This testing process is undertaken to check the performance and behavior of the application under certain conditions such as low battery, bad network coverage, low available memory, simultaneous access to application s server by several users and other conditions. Performance is checked on both client and server side. Memory Leakage Testing:Memory leakage happens when a computer program or application is unable to manage the memory it is allocated resulting in poor performance of the application and the overall slowdown of the system. As mobile devices have significant constraints of available memory, memory leakage testing is crucial for the proper functioning of an application. Interrupt Testing: An application while functioning may face several interruptions like incoming calls or network coverage outage and recovery. The different types of interruptions are: Incoming and Outgoing SMS and MMS Incoming and Outgoing calls Incoming Notifications Battery Removal Cable Insertion and Removal for data transfer Network outage and recovery Media Player on/off Device Power cycle

3 Usability testing: It is carried out to verify if the application is achieving its goals and getting a favorable response from users as the usability of an application is its key to commercial success. Installation testing: Certain mobile applications come pre-installed on the device whereas others have to be installed from the store. Installation testing verifies that the installation process goes smoothly without the user having to face any difficulty. This testing process covers installation, update and uninstallation of an application. Certification Testing: To get compliance certificate each mobile device needs to be tested against the guidelines set by different mobile platforms. 1. eggplant: Mobile Application Testing Tools eggplant is a GUI(Graphical User Interface) based automated test tool delivering a robotic solution to mobile testing across all operating systems and devices. eggplant delivers a robotic solution to mobile application testing across all operating systems and devices. Using RealVNC, generic VNC or emulators, eggplant deploys patented intelligent image recognition technology to see any device display. This approach to test automation is easy to use, non-invasive and requires no special hardware. Write one script and use this script to test across multiple devices and emulators. eggplant can be used in a continuous integration environment, for regression testing and provide statistics in a load tested environment. eggplant provides a scalable and flexible solution for all functional testing and data driven testing objectives. With eggplant mobile application testing solution QA test automation engineers can : 1. Test on real mobile devices 2. Test across mobile OS with a single script

4 3. Test within your firewall As a result, testing applications with TestPlant is much faster than using traditional test tools, meaning robust mobile applications can be brought to market very quickly. In addition, it brings great manual test productivity, so costs can be dramatically reduced. Link for eggplant testing Smart phone: Cross Platform Testing. eggplant scripts work regardless of the environment you are testing. Supported platforms include: a. Windows 95, 98, 2000, NT, XP, Vista, and Windows 7 b. Mac OS X c. LINUX d. Solaris e. HP-UX f. Even BIOS and other hardware devices Here is a link for eggplant in action: Rich Internet application testing. I. HTML5:Its is the new RIA and growing rapidly. With growing adoption in mobile app development HTML5 is set for great growth. But testing it remains a challenge. But with eggplant you can test it as any other RIA application. II. Adobe Flex:Because eggplant doesn t rely on code hooks for application testing, it s perfect for GUI testing graphic rich applications designed and built in Flash where no code-level tool integration is possible. III. AJAX:As eggplant does not rely on objects, testing AJAX functionality is

5 simple and easy as well as allow for performance metrics to be used. IV. Silverlight:With eggplant, scripting a Silverlight application is no different from scripting any other application. V. Java:As GUI s are developed using Java with Swing or SWT. Testing them for functional validation becomes more important.with eggplant you can conduct diversified GUI testing. For more information on how it works,click on the following link: Performance Testing using eggplant i. Application Performance Testing is conducted to determine if an application, system or component meets such metrics as responding in an appropriate amount of time or handling an acceptable number of connections. ii. Stress or Load tests are one type of Performance Testing that can often be automated with a test tool that simulates multiple users. Common deficiencies encountered during this phase of performance testing are related to things like slow execution cycles, data corruption, or complete system failure. iii.there are a number of different methods one can use to create a performance test with eggplant. iv. eggplant application performance testing tool can time individual events or sequences of events, and it can be used either to generate small loads or to enhance an existing load test scenario by simulating real user interactions while another tool creates an artificial load on the application. v. Performance testing example GIMP is the GNU Image Manipulation Program. It is freely distributed and used for such tasks as photo retouching, image composition, and image authoring. Objectives: To determine the performance impact of having multiple images open for GIMP. To determine the number of open images that the system can

6 handle the threshold that the system will allow. Test Specs: Operating System: Linux RedHat 7.1 Kernel: Desktop: Gnome Application: GIMP Test Details Our test example will use one jpeg image with a file size of bytes. The sample image must reside in the home directory of the current user and will be named eggplant.jpg. For consistency, no other UI applications will be allowed to run concurrently with GIMP. The following settings for GIMP will be used: Tip of the day will be disabled. All other windows, with the exception of GIMP s main window, will be hidden from view.

7 Test execution will consist of the following: 1. Launch GIMP from the Gnome menu bar. 2. Open a new image using the Control-o keyboard shortcut. 3. Wait for the Load Image window to appear. 4. Wait for the Load Image window to appear. 5. Proceed by selecting eggplant.jpg. 6. Press Enter to open the image. 7. Measure the amount of time that it takes for GIMP to load the image. The time limit for an image to open will be set to 10 seconds. Once the threshold is reached end the test. 8. Log the time. 9. Repeat steps 2 through 8. Integration and validation with IBM Rational Quality Manager 4.0. eggplant s integration with IBM Rational Quality Manager, eggplant s market-leading automated GUI and screen testing is available to IBM Business Partners and customers as part of the Ready for IBM Rational program. This opens up opportunities for new users to harness eggplant s capability for consistently delivering powerful business-driven GUI automation and software testing results across legacy, desktop and mobile environments.the eggplant integration allows you to call eggplant scripts running on any platform (Mac, Windows, Linux or Mobile) and return the results to your RQM server.

8 v=7yniujh8hww&feature=player_embedded Benefits Non-invasive does not load to test devices or interact with underlying code Transparent processes, results and metrics vital for ensuring that application development teams, Quality Assurance teams and business teams are all on the same page Works with Windows 95, 98, 2000, NT, XP, Vista and Windows 7; Mac OS X; LINUX; Solaris; HP-UX; plus BIOS and other hardware devices Supports HTML 5; Adobe Flash/Flex; Silverlight; AJAX and Java

9 2. DeviceAnywhere :A cloud based platform for carrying out testing and monitoring the functionality, usability, performance and availability of mobile applications. 3. elusivestars:a crowd sourced Mobile Application service where Developers have their Apps tested by real users with real devices. 4. Cucumber-Functional Automation tool Cucumber is a functional test automation tool for lean and agile teams. It supports behavior-driven development, specification by example and agile Acceptance testing. You can use it to automate functional validation in a Form that is easily readable and understandable to business users, developers and testers. This helps teams create executable specifications that are also a goal for development, acceptance criteria and functional regression checks for future changes. Cucumber is one of the rare tools that try very hard to stay out of your way, to let you do your work without making you worry about the tool itself too much. It helps teams automate their specifications efficiently in several ways: It is relatively easy to set up. It supports multiple report formats, including HTML and PDF. It supports writing specifications in about 30 spoken languages, making it easy for teams outside of English-speaking territories or those working on internationally targeted software to engage their business users better. It supports different ways of describing executable specifications including story-like prose, lists and tabular data. It allows scripting, abstraction and component reuse on several levels, allowing both technical and non-technical users to efficiently describe specifications. It generates the tricky parts of the code so that you don't have to write most of the boiler-plate automation or make mistakes doing it. It integrates nicely with the rest of the development ecosystem. It does not try to impose a version control system, but works off plain-text files that can be stored in any version control system. For continuous

10 build integration, it emulates JUnit (and everything else in the world is already integrated with Junit). Although it is a Ruby tool, people who work on other platforms do not have to learn Ruby to use it. You can use Cucumber with.net or JVM languages almost natively. It's integrated with all the most popular web testing libraries. It allows you to mark tests with tags so that you can quickly run a group of related tests (eg quick tests, slow tests, integration tests, accounting tests). 5. MobileCloud-Interactive by Perfecto Mobile The MobileCloud-Interactive service enables users to test mobile applications, websites or services on a multitude of real handsets and tablets via the web and control them as if holding them in their own hands. Using MobileCloud-Interactive, you'll be able to streamline development, testing and support processes, perform more tests and ensure optimal compatibility of your product or service, on more handsets, networks and locations. 6. Robotium-Its like a Selenium but for Android