Mobile App Testing is not something special Simon Peter Schrijver TesT-PRO @simonsaysnomore p.schrijver@test-pro.nl simonsaysnomore.wordpress.com
My career in Mobile (App) Testing Between 2006 and 2014 Test Manager Did a lot of test execution myself Projects Introduction of new MVNO Upgrade of network systems HLR IN CDR processing CMS Billing Introduction of iphone (2008) Data Roaming
My career in Mobile (App) Testing Between January and October 2014 Senior Telecom Application Tester Projects Building HLR AuC HLR IN Automation SOAP UI
My career in Mobile (App) Testing From October 2014 up until now Senior Tester Mobile App Projects Jumbo ABN-AMRO
A Mobile Device is special As Steve Jobs said during the introduction of the ipod I want people to carry 1000 songs in their pocket * He later added the Internet, Touch and Phone to it * Source : http://genius.com/steve-jobs-ipod-introduction-annotated
A Mobile Device is special You carry a lot of things with you on one device It is a part of everybody's live A whole new industry is created
A Mobile Device is special Because it is moving Brings some new things to test for Movement Edge of network Portrait/Landscape Touch Notifications Speed of data Battery of device Waiting time
A Mobile Device is special Network interfaces Bluetooth Near Field Communication Global Positioning System Wifi Cellular networks Sensors
Unique devices 24093 (increase of 28%) Versions Kitkat (4.4) runs on 37.8% of those devices 1294 Android manufacturers 1.4 billion active user per month The first device went on sale in October, 2008 Fragmentation Android in 2015 Source : https://androidhub.intel.com/en/posts/peachpellen/infographic_android_in_2015.html
Fragmentation by device Source : http://opensignal.com/reports/2015/08/android-fragmentation/
Fragmentation by brand Source : http://opensignal.com/reports/2015/08/android-fragmentation/
What is a Mobile App? A mobile application (or mobile app) is a software application designed to run on smartphones, tablet computers and other mobile devices. Source: Wikipedia
Top Consumer Mobile Applications Money Transfer Location-Based Services Mobile Search Mobile Browsing Mobile Health Monitoring Mobile Payment Mobile Music Mobile Advertising Mobile Instant Messaging
Source : http://www.nielsen.com/us/en/insights/news/2015/tops-of-2015-digital.html
Type of Mobile App? There are primarily three techniques for creating Mobile Apps: Native applications Mobile web applications Hybrid mobile applications Web-to-App applications Web-and-App applications
Type of Mobile App? Native Installed applications can use all mobile or tablet device features such as camera, calendar, and contacts. Support the richest user experience (for example, gaming applications) Built using each phone's native SDK Source: http://www.worklight.com
Type of Mobile App? Web Accessible over the internet without installing software Use device browsers to provide native-looking applications Built using standard web technologies such as HTML5, CSS and JavaScript Source: http://www.worklight.com
Type of Mobile App? Hybrid Leverage web technology and native device features Installed applications that use the device browser to display webbased user interface Able to use native device features Built using standard web technologies (HTML5, CSS, JS) and phone's native SDK. Source: http://www.worklight.com
Native vs. Web vs. Hybrid Source: http://www.worklight.com
What is Mobile App Testing? Software on a mobile device Interact with an user Send data over an interface To a backend system Wait for a response And process this response
What is Mobile App Testing? Fragmentation Devices Brand Operating system Features on the device Native, Hybrid, Web Features of the App
Mobile App Testing - Fragmentation Devices Brands
Mobile App Testing - Fragmentation Android 4.0 4.1, 4.2, 4.3 4.4 5.0, 5.1 6.0 ios 7.0, 7.1 8.0-8.4 9.0 9.3 9.4? 10.0 (sept 16)
Mobile App Testing - Fragmentation How to assess fragmentation? Define target groups to narrow your scope Choose between simulators (emulators) and real devices (rental devices) Assess the risks you are facing Test smart
Mobile App Testing - Fragmentation Test approach (manual) Integration/regression test on 1 real Android and ios device, complete test set 3-4 different Android/iOS devices with reduced test set based on assessment Time left over, test with other devices Start automation as soon as possible with native automation tools (Espresso for Android, XCTest for ios)
Mobile App Testing - Features on the device Network Camera Battery Touch screen Voice recording Location
Mobile App Testing - Native, Hybrid, Web Native Installation test Interface with API Touch Different OS Interaction with device attributes, like camera
Mobile App Testing - Native, Hybrid, Web Web Reached by navigating to a website by way of an URL All code executed from a server, resulting in network-dependent performance Have to test different mobile browsers. Responsive web design, responds automatically to whichever size device a user is on
Mobile App Testing - Native, Hybrid, Web Hybrid Best from the Native world and the Web world. Access to device-specific features
Mobile App Testing - Question Which one of these four expressions does not belong here? Usability Mobile Performance Security Mobile is made up as a specialism by the commercial people Guess what? You can become a Certified Mobile Tester (via ASTQB)
Mobile App Testing - Features of the App Make sure the user has an awesome experience That means, the functionality of the app is in order, the security measures are in place, the performance of the app is within accepted limits and the usability of the app is user friendly. These 4 quality attributes must meet the expected requirements
Skills for a Mobile App tester A mobile tester must possess certain skills who will make him/ her valuable. Improve your skills by learning from other Apps. Watch how other people use their Apps Communication Curiosity Critical thinking Tenacity Constant Learner Creativity Customer focus Programming and technical skills
Skills for a Mobile App tester - Communication Listen carefully. Don t interrupt other people while they re speaking. Speak clearly and precisely. Don t get personal when communicating with other people. Be able to communicate on different levels, ranging from technical to nontechnical audiences. Improve your vocabulary by reading books, blogs, and newspapers. Daniel Knott - Hands-On Mobile App Testing
Skills for a Mobile App tester - Curiosity A curious software tester explores the software to get as much information as possible out of the system to identify potential problems and raise interesting questions about the product Source : Daniel Knott - Hands-On Mobile App Testing
Skills for a Mobile App tester - Critical Thinking Critical thinking is thinking about thinking with the aim of not getting fooled. Michael Bolton Gain a deep understanding of the product Focus on the right testing work Daniel Knott - Hands-On Mobile App Testing
Skills for a Mobile App tester - Tenacity Software testers must be tenacious and explain or raise bugs over and over again until the software quality standards have been met. But be careful with this as you may end up being considered a nuisance. Here it s important to rely on your strong communication skills. Daniel Knott - Hands-On Mobile App Testing
Skills for a Mobile App tester - Constant Learner Software testers need to take note of changes taking place around them in order to adapt and learn new approaches, methods and technologies. To keep pace and learn new techniques and tools, software testers can read blogs or books and attend conferences and training courses. Daniel Knott - Hands-On Mobile App Testing
Skills for a Mobile App tester - Creativity The creativity process starts with generating the test idea and collecting the test data. Be creative in order to use mobile devices in different ways by paying attention to all the interfaces, sensors, and locations. Daniel Knott - Hands-On Mobile App Testing
Skills for a Mobile App tester - Customer Focus Try to think like a customer in order to determine whether the software being tested meets customer needs Requires you to be a product and field expert within your team Talk to customers to determine their needs and problems Use the knowledge to every phase of the software development process Daniel Knott - Hands-On Mobile App Testing
Skills for a Mobile App tester - Programming skills Mobile testers with no programming skills need to train themselves by reading a book about programming languages or patterns following a programming tutorial on the Internet attending a programming course. It s also possible to ask a developer to train the mobile tester within a project or company. Daniel Knott - Hands-On Mobile App Testing
Mobile App Test Coverage Model Heuristic : LONG FUN CUP Location Orientation Network Gesture Function User Scenarios Notification Dhanasekar Subramaniam Test coverage model focussed on testing of Mobile App Communication Updates Platform https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Location App should be smart enough to track your current location and provided suggestions surrounding your locality App asks for user permission before tracking the location Options to mock your locations for testing the app Do not test an App sitting at your desk https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Orientation Change the orientation, app should be able to provide a consistent user experience Test all your screens, pop-ups, toast messages, forms in all supported orientations Instance where the filled in form data disappears when you change the orientation https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Network Mobile devices supports both cellular and wi-fi. They can automatically switch between any available networks. How does the app behave when the device switches between networks? Test for signal strength, network availability and reliability https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Gesture Does your app supports all standard gestures? Is it consistent across the app? If it uses any new gestures, is it easy for the users to understand? https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Function Any function that defines or distinguishes the product or fulfills core requirements. Test for interactions, error handling, starting and closing of the app, file access, navigation, multimedia and sync. https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP User Scenarios Any scenario should have a story that is credible, motivating, complex and easy to evaluate Try to think about disfavored users and how they might try to exploit the app. Compare with competitor or web interface to get more real life scenarios https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Notifications Notifications enable an application to inform its users that it has something for them. Test for all available types of visual notifications, sound and vibration. https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Communication The primary purpose for using mobile phones is voice calls. How does the app behave after interruptions by an incoming call? What about text message, Skype video call or FaceTime? Test for such interruptions from voice communication and all necessary functions of your app. https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Updates Mobile app gets frequent updates. Not only app updates but OS updates as well. What changes, which new features are in the latest OS update Analyse if the app needs any modification because of the OS updates. https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONG FUN CUP - Platform Have a good understanding of the popular mobile platforms especially Apple and Android Be aware of the history and also the latest trends in the mobile platform. What tools are available for testing What kind of test-ability layers are provided by the platform https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
Summertime bug Via the browser I entered my birthday, on the mobile app I displayed my birthday. It was displayed wrong https://simonsaysnomore.wordpress.com/2016/03/08/the-summertime-bug/
Next year
Useful information http://katrinatester.blogspot.co.nz/2015/08/mobile-testing-pathway.html http://adventuresinqa.com/2016/05/03/10-mobile-app-testing-mistakes-to-avoid/ http://adventuresinqa.com/2015/06/22/top-10-best-practices-for-mobile-apptesting/
Mobile App testing is not something special Mobile device is a channel. The user gains access via an App to the backend where all the information is stored. Mobile App testing is based on four quality attributes. Functionality is the most important one. When you as a tester already have some test experience, you can do Mobile App testing
I m a Tester 50 years old Tester since 1997 Test Mobile Apps since November 2014 Got the Mobile Apps testing job via my network Based on my reputation as a tester My job interview (there was none) Learn to test Mobile Apps just by myself
Simon Says Thank You Do you have any questions? @simonsaysnomore p.schrijver@test-pro.nl simonsaysnomore.wordpress.com