WWW.QUALTECHCONFERENCES.COM Europe s Premier Software Testing Event World Forum Convention Centre, The Hague, Netherlands The Future of Software Testing Mobile Software Testing Facing the Future Challenges Ian Smith & Robert Campbell, Sopra Group Ltd., UK
Mobile Software Testing Facing The Future Challenges Ian F. Smith Principal Consultant ismith@uk.sopragroup.com Sopra Group Ltd. Tel: +44(0)131 332 3311 Robert Campbell Senior Developer rcampbell@uk.sopragroup.com Sopra Group Ltd. Tel: +44(0)1698 574 200 T A L E N T E D T O G E T H E R Unissons nos Talents Mobile Software Testing Facing The Future Challenges - EuroSTAR 2008 (R1) 2
Overview Introductions Mobile Systems Trends Challenges/Problems Approaches Testing Mobile Systems Example/Case Study Summary 3
Introductions Ian F. Smith Principal Consultant Testing Division Over 28 years experience in software and system development and testing Strong background in telecoms and real-time/embedded systems Robert Campbell Senior Developer Testing Division 28 years experience in embedded development, manufacturing and test Focussed on testing telecoms monitoring equipment for the last 6 years Sopra Group Sopra Group, a major European IT Services Company Total of 11,000 staff and group 2007 Revenue of 1 Billion Offices throughout UK and Europe + and a fully integrated offshore Development and Testing Centre in Noida, Delhi Services: Application Outsourcing, Systems Integration, Testing + Specialist services (Consulting, Information Security, Business Intelligence) Key markets: Finance, government, retail / manufacturing, energy & utilities, health, telecoms + media 4
Mobile Systems User Mobile Handset Mobile Network Internet Corporate Application Server ISP Mobile Application Server User Mobile Device Content/Data Server What s a typical mobile system? Various types of mobile system User operates mobile devices Supports access/operation in various locations and can often connect by various methods Key Characteristics of Mobile Systems Elements: Mobile Devices Wireless/Wired connection Server + Network Infrastructure Mobile applications Can run on either device or server, depending on architecture and device capability More internet based apps/services are being now provided for mobile use Increasingly, core business systems are providing mobile application interfaces 5
Mobile Systems - Examples Navigation/Route Planning Mobile Application Server: Navigation/Route Planner Handset software: Navigation Handset Client Content: Route information Access Methods: GSM/3G Email Mobile Application Server: Email Server Handset software: Email Client Software Content: Email information Access Methods: Desktop, Laptop, GSM, WiFi, Voip Service Mobile Application Server: Voip Network System Handset software: Handset Voip Features + Agent software Content: Voip calls data + SMS Access Methods: WiFi, 3G 6
Trends Some key stats 1.28 billion mobile handsets to be sold in 2008 [Source: Gartner] 32.2 million smartphones sold in Q1 2008 [Source: Gartner, May 2008] Connectivity 10G Better support for data (networks and devices) 1G 100M LANs 3G data services WiFi Increasing power and sophistication of mobile devices More powerful processor and storage Software capabilities Connectivity Audio/Camera/Video Sensors/Accelerometers GPS 10M 1M 100K 10k Application Capability 1980s Fixed Access Mobile Access 1990s GSM 3G 2000s Future 2010+ Application trends Desktop Converged Devices Web 2.0 applications Increasing expectations Desktop applications on mobile devices Laptop Mobile Devices Device capabilities e.g. LBS 1980s 1990s 2000s Future 2010+ 7
Trends (Cont.) Handsets gaining laptop/pda capabilities Processing, storage and software Mobile OSs Established players Symbian, Windows Mobile, RIM BREW, Linux Mobile, New Players Apple, OHA/Android, LiMo, Landscape is changing Symbian Foundation Open source Possible consolidation?? Development Environments Varied environments Open source Proprietary IDEs/SDKs Improving (but still with limitations) Server application development 8
Challenges/Problems Cellular network services can vary significantly Data offerings Performance Mobile device platforms Many different software models + SDKs/IDEs/Toolsets Changing platform landscape (Symbian Foundation, Android, iphone, ) Device specific firmware issues Application complexity Meeting expectations for current desktop based applications on mobile Application/content access Access Diversity Ubiquitous access GSM/UMTS WiFi/WiMax Wired Security Trust models and signing Increasing amount of sensitive data on devices 9
Approaches Manage Complexity Application and system architecture Component partitioning Divide and conquer Multi-level testing (unit, component, integration, system levels) Get value from unit testing Layer UI to simplify testing and support automation Maximise code reuse Important for multi-platform developments Different platform models can limit this Consider signing testing or manufacturer specific testing Signing can be expensive and time consuming 10
Approaches (Cont.) Using emulation Doesn t address all testing needs (telephony, performance) Combine with GUI based test automation tools Device Testing Confirm compatibility (device firmware issues) Cover device specific features Security Testing Threat models change constantly, so review security testing regularly Consider full operational lifecycle Deployment, operation, upgrade, removal 11
Building Mobile Systems Choosing platforms Implications System Architecture DfT Design for Test Create test support features Handset software Engine (portable) + (thin) UI layer Different software models Application signing Consider application efficiency Power efficiency/management is a major factor for mobile systems Development environments SDKs are getting better, but building and testing will still be challenging Design for offline use Connectivity is improving but devices won t always be connected Also, some OSs don t support application concurrency, so consider this 12
Testing Mobile Systems Planning testing Server and core system tests Mobile device software testing Using devices in system testing Design testing to cover different handset platform needs Aim for reuse, where practical Server testing Existing web or other test tools may be used as mobile device client capabilities approach those of desktop client systems. Network & Infrastructure testing Tools Understand provider capabilities and limitations Consider whether additional support tools can be procured/created to assist in testing (set-up, reporting, diagnosis, etc). Emulators Can support application level testing Not good at emulating telephony or real-time operations Combine with GUI test tools 13
Automating Testing Quick wins Unit test can be well automated Continuous Integration tools Even some handset code!! Automating device testing This can be tough different environments, user interfaces, etc. Design UI to support automation (allow virtual user actions driven by tools) Test Automation Tools Performance Testing Skills Automation is essential to effective performance testing Consider this in design-for-test activities. 14
Example Webraska Navigation System Handset based navigation system Route information from server GPRS connection Bluetooth GPS Handset software testing project Evaluate handset software Quality Performance Usability 15
Case Study Network Under Test Test Operator (1) Test Control Computer Test Handset GSM/UMTS Radio Access Network Core Mobile Network Test Operator (M) Test Control Computer Quality Center Server Mobile Handset Controller Test Handset Billing System Mobile Call Generator Project Develop proof of concept test call generation system Key project objectives: Automated control of voice calls, messaging and data functions Scalable solution Remote control RemCall System Architecture Software Handset Server(s) Development Challenges Achievements Limitations 16
Case Study (Cont.) Multi-level Testing Unit, Component and System level test Handset Agent Software Piloted automation for unit testing Introduced CI with CruiseControl Handset Controller PC Software Quality Center Server Software 17
Summary Mobile systems offer many major opportunities but developing and testing them pose significant challenges. The pressure to supply mobile access to existing business applications and systems is growing as device capabilities improve to support these. New and existing businesses are bring innovative mobile systems to market. The handset landscape is changing rapidly, with established players evolving and new entrants joining. This change will continue. Consider platforms carefully development implications and future market. Architecture and component partitioning can have a marked effect on system development and testability. 18
Questions? Thank You www.sopragroup.co.uk 19