Ramakrishnan R Associate Vice President Banking Practice, SQS India BFSI Limited

Whitepaper Bank on Mobile sqs.com Authors: Ramakrishnan Venkateswaran Test Lead, SQS India BFSI Limited Ramakrishnan R Associate Vice President Banking Practice, SQS India BFSI Limited Published: August 2015 SQS Thought Leadership Book 2015 This whitepaper is an excerpt of this year s Thought Leadership Book. It brings you the latest thinking in current challenges and solutions facing modern businesses today. If you wish to receive a copy of the book, please visit our booth no. 45 in the EuroSTAR Conference Exhibition. We wish you an inspiring reading experience. SQS the world s leading specialist in software quality

Bank on Mobile 2 Ramakrishnan Venkateswaran Test Lead ramakrishnan.venkateswaran@thinksoftglobal.com Ramakrishnan Venkateswaran, Test Lead holds a bachelor s degree in Electrical and Electronic Engineering and has been with SQS for over 9 years. He has successfully led many testing assignments for global clients across banking domains. His core competencies include functional analysis, test, automated test suite and automated test case creation. Recent developments in Mobile Testing have inspired him to get involved in Mobile Testing and Mobile Automation. Ramakrishnan R Associate Vice President Banking Practice ramakrishnan.r@sqs.com Ramakrishnan R is a Chartered Accountant with over 18 years of experience in IT Services. He has been with SQS for over 12 years and has in-depth knowledge and experience in Core Banking, Cards, Payments, Trade Finance Operations, Forex and Domestic Deposits. He has successfully managed many manual and automated testing projects across the Cards and Banking domains. He has been instrumental in establishing and stabilising Managed Services assignments for many global clients.

Bank on Mobile 3 Contents Management summary...4 Introduction....5 Mobile banking mania a paradigm shift from tradition...5 Mobile testing requirements...6 Mobile testing market cry...7 SQS mobile testing solution....8 SQS mobile automation framework...10 Conclusion...15 Benefits of SQS mobile test methodology...15 Benefits of SQS mobile test automation...15 References...16

Bank on Mobile 4 Management summary Mobile devices or hand-held devices have become the most powerful tool in the 21st century and have transitioned from being the cornerstone for communication, to constituting a multipurpose personal gadget and indispensable companion. People use mobile for anything and everything in this modern era and the banking business is no exception to this technological advancement. There is a sharp collision between tradition and modern technology, with a clear shift from the traditional banking channels to newer service delivery channels and mobile. The ING International Survey 2015 (Bye-bye cash) also stated that banking and shopping are increasingly going the mobile way. Mobile and everything through mobile is the current thriving business area, where big investments are made with the aim of attracting young and tech-savvy customers. Consumers are embracing mobile in their day to day lives and are more likely to leave their wallet at home than their mobile phone. With all this energy and momentum around mobile, there are some areas of concern regarding the quality of mobile apps introduced by every bank. A key challenge is the customer s aversion to the mobile app, perceived as lacking usability, reliability and security. Banks need to ensure that every application meets customers expectations and offers the highest quality standards in order to prevent repercussions on brand reputation and loss of customers resulting in a drop in business and revenue. A comprehensive mobile application testing strategy is essential for getting applications to market on time and with optimum quality. Testing mobile applications is different from and more complex than testing traditional applications. Mobile applications need to be tested on a variety of OS versions and on multiple hardware devices. Moreover, the rapid pace of mobile OS updates, the frequent introduction of new devices, and customer expectations of quick upgrades require multiple test cycles. Adding to the complexity is the need to integrate various third-party apps into mobile banking applications. This requires the almost continuous performance of a robust QA procedure. This whitepaper explores the different facets of automated mobile testing and the challenges around it, gained from actual experience in the field.

Bank on Mobile 5 Introduction Mobile banking mania a paradigm shift from tradition The preference for digital is now pervasive across all customer segments. An effective mobile banking strategy is required to offer user-friendly mobile banking channels which are easy for consumers to use, flexible enough for financial institutions to adapt to, and scalable to include more services. The recent ING International Survey 2015 [1] quoted, The Netherlands is the most developed mobile banking market in the world. The United States is in second place in the survey, closely followed by the United Kingdom. Overall, uptake in Europe is lower than in the United States and Australia, but the share in Europe in the next 12 months is expected to be higher (Figure 1). The survey goes on further quoting Most customers say mobile banking leads to better money. Mobile banking is perhaps most often lauded for its convenience. However, there appear to be many other benefits. The vast majority of people who use mobile banking indicate that their money has improved since using the technology, for example, they feel more in control of finances and confident of not missing payments due, etc. on account of lack of trust in the security procedures (Figure 2). Belgium Italy European Consumer Australia United Kingdom USA Netherlands 9 % 8 % 9 % 13 % 15 % 14 % 20 % 35 % 36 % 41 % 44 % 49 % 50 % Expect to use mobile banking Already use mobile banking 58 % USA Australia Italy United Kingdom Belgium Netherlands European Consumer 89 % 75 % 90 % 84 % 79 % 74% 85 % Figure 1: Use of mobile banking Figure 2: Improvement in money

Bank on Mobile 6 Mobile testing requirements Mobile testing is different from traditional application testing and, as such, depends for its success on a strong test methodology and a comprehensive testing strategy. From a testing perspective, some methodologies push for testing input early in the development lifecycle, while others wait until a working model of the system is ready. Defining the test strategy Which software methodology to choose? As mobile applications become more and more complex and intertwined with a number of different OS platforms and models, it is imperative to have a robust testing methodology to make sure the product is rolled out with optimum quality and able to operate successfully in all anticipated environments with the required usability and security. Selecting the right test methodologies depends on many factors such as the nature of the project, client requirements, region of the customer base, project schedule, etc. A comprehensive mobile testing strategy that includes device and network infrastructure, optimised selection of target devices, and an effective combination of manual and automated testing tools to cover both functional and non-functional testing is essential. The factors in Table 1 should be taken into consideration for building a strong mobile test strategy that gets mobile applications to market on time and on budget. Mobile App Test object Type of the App (Native App, Hybrid App) Platform (ios, Android, Windows, BlackBerry) Platform Version Device Type (Smartphone, Tablet) Device Resolution Service Provider Integration (standalone, with backend) Mobile Internet Test object Type of Browser (Firefox, IE, Chrome, Safari) Operating System Operating System Version Browser Version Device Resolution Service Provider Integration (standalone, with backend) Table 1: Factors to be considered in a strong mobile test strategy

Bank on Mobile 7 Mobile testing market cry As mobiles increase in popularity, more and more banks are moving into today s multi-channel system architecture to attract customers. The key cry areas in order to achieve quicker time to market with reduced costs and optimum quality are: 1. Continuous Integration: The IT industry is fast-moving and adopting agile and DevOps methodologies. Mobile development is also slowly shifting focus towards these practices in order to roll out products as early as possible. Here, development and QA have to be in tandem and this is achieved through automating testing (Continuous Integration) to the maximum extent. 2. Script once and run multiple times: 2015 has seen many entrants in the mobile device sector flooding the market with their devices; not to mention the frequent updates on the OS platforms Android and ios. With these rapid changes comes the need to have a mature mobile testing system with a working mobile automation framework that runs the same script across varying device types and OS platforms. This way the Dev/QA teams can be confident they are taking the best app to end users. 3. Multi-lingual testing: As Banks emerge as global players with footprints across the globe, their mobile applications need to be in unison and should be available in multiple languages. To reduce the QA cost and ensure maximum re-use, the automation framework should be robust enough to run the same test in multiple languages dynamically. 4. Real device testing: As businesses aim to bring a more realistic and lifelike approach to QA, testing must be performed on real devices rather than emulators. The key challenge in this approach is: a. Not being able to run tests continuously as the real device tends to break down beyond a point b. Device and cost Management becomes cumbersome 5. Parallel testing: To reduce the test cycle or elapse time and attain better ROI, the testing needs to be performed on as many parallel devices as possible.

Bank on Mobile 8 SQS mobile testing solution SQS - Test organisation has evolved as a proven methodology following years of industrial experience gained. It comprises a work breakdown structure (WBS) combined with a test strategy as per ISTQB standards and identifies the responsibility for each test stage. It is a key instrument in customising projects for multi-channel mobile or Internet-based systems and ensures that the test process is: Manageable (planning, control and ) Effective and efficient (completeness, avoidance of redundancy) Traceable (coverage) and Transparent (reporting) SQS adopts smart planning and strategy to maximise test coverage and ensure better device and cost savings. Various market surveys on OS and device distribution [e.g. 2] conducted by reputable institutions are constantly monitored, and an optimum combination of right OS and devices for different geographies has been ascertained. This matrix is quite useful in a project in order to arrive at an optimised mix of physical devices to be tested in conjunction with a set of emulators. IDC s Worldwide smartphone operating system market share projection [3] shows clearly that Android is the current Market leader. People are also tending to change their mobiles more frequently which has resulted in Apple and Samsung losing some of their market share. Figure 3 shows the ecosystem of the mobile world. Multiple Distribution Channels Dozens of Target Platforms Hundreds of Devices Vendor Figure 3: Diversity of platforms, versions and devices in the mobile world

Bank on Mobile 9 Mobile testing the big picture Test Management Functional Testing (ISO25010) Non-Functional Testing (ISO25010) Environment & Tools Test strategy and planning Operational TM in companies and projects Set up centres of excellence Test asset optimisation (TAO) Test health checks and improvement Component Component integration Function Application Interface Business domain Full E2E User Acceptance (UAT) Compatibility Load & performance Portability Reliability Security Usability QA for architecture, design and code Operational acceptance testing (OAT) Infrastructure testing Test automation Test data Test environment Test tool Figure 4: Mobile testing the big picture Various components are to be strategised for an end to end mobile testing covering test, functional, non-functional and tools (cf. Figure 4). The functionalities of mobile banking which need to be tested can be broadly classified into: Enquiry: Includes enquiries for balance and transactions, loan and credit information, ATM and branch locators, etc. Transactional: Includes fund transfers, bill payments, repayments on loans/credit cards, e-commerce transactions, etc. Services, Marketing & Acquisition: Includes SMS alerts, help information and other alerts on promotional products, renewals, etc. The mobile testing landscape with its various testing needs is depicted in Figure 5.

Bank on Mobile 10 Aspect Functionality Usability Areas/Types of testing User interaction testing Functional testing User experience Network Network type (Wi-Fi, 2G, 3G, 4G) Impact of connectivity issues Performance UI responsiveness Transaction completion time(s) Peak load performance Longevity Security Data retention on device Transmission security Conformance Marketplace guidelines compliance (e.g. Apple App Store policies) Enterprise policy compliance (e.g. prohibited content) Compatibility Mobile platform compatibility (e.g. ios 6, ios 7, ios 8 Adroid ICS, KitKat, Lollipop Device model compatibility Backward compatibility (with previous app version) Figure 5: Mobile testing landscape SQS mobile automation framework The SQS mobile automation framework is a solution for the shorter test cycle window and caters to the extremely dynamic, complex and fragmented mobile market. This framework has been developed based on the tried and tested process which has helped to deliver projects successfully for various global customers. The SQS mobile automated testing framework is a solution for applications developed using Android, ios and Windows platforms. The framework also helps to ensure enhanced coverage across multiple mobile configurations and multiple languages. This unique in-house-developed automation framework boosts testing using physical devices in a geographical spread, or emulators or cloud-based solutions, or a combination of all three depending on the bank s requirement. SQS has conducted research into the mobile automation tools available in the market, and based on the tool feasibility study, SeeTestAutomation [4} is suggested for its features and cost effectiveness. The outcome of the market study is shown in Table 2.

Bank on Mobile 11 Category Feature Perfecto SeeTest App Types OS Support Native Hybrid HTML-5 Web ios Android Blackberry Windows Phone Integration HP ALM HP UFT Access Cloud Overall Suitability Stability Maturity Coverage Price Table 2: Mobile automation tools The framework uses a cost effective automated solution with the SeeTest tool in combination with existing automation tools such as UFT, Selenium, RFT, MS TFS, etc. The platform ensures single time scripting that works cross-platform and thus guarantees maximum re-use of scripts. The framework is built with high re-usability and easy extendibility. It also comes with flexible adaptors to support the agile development methodology. A blue print of the SQS mobile lab & architecture can be referred in Figure 6 and framework components can be referred in Figure 7. Automation tools for mobile application testing use one of the two major techniques; object-based/ image-based. There are commercial tools which use objectbased recognition (SeeTest, Jamo Solutions) to achieve automation by mapping elements on the device screen into objects. This approach is independent of screen size or resolution and provides a high degree of script re-usability. This method allows testing to be performed on real devices and has high usability across OS platforms. Image-based or bitmap-based tools (such as Perfecto Mobile and Device Anywhere) create automation scripts based on the screen coordinates of elements. While this approach is agnostic to the type of application (native, web or hybrid) and provides useful image matching capabilities, the automation testing is not so reliable when using multiple device types or manufacturers. The SQS automation framework is developed in such a way that it can be integrated with any tool to support a hybrid approach combining the best of both worlds.

Bank on Mobile 12 Test Stack Mobile Test Lab Technology Test Manual execution Software Cloud (remote content) Lab Manual test interface Automation interface Hardware Rack-based cabinet USB WLAN PC-based server Mobile devices Test automation Services Test Test basis Test organisation Risk evaluation Test planning Defect Manage & report Test Closure Functional & non-functional testing Test analysis Test design Test implementation Test execution Test automation Tool chain set up / Frameworks for mobile testing SQS-TEST /Professional HP ALM/UFT Selenium Mobile drivers Mobile lab operation model Incident Change Event monitoring Asset Configuration Quality People Order Availability Financial Mobile lab operation services Release mgmt. and deployment Build & transition services Continuous improvement Continuous service improvement Figure 6: SQS - Principal Technical Setup and Services Blueprint

Bank on Mobile 13 Designed to support modular test automation Used with HP UFT, SQS-TEST /Professional, HP BPT, HP ALM Test reports can be viewed directly in a browser Can be tailored to fit project-specific requirements Automatic code document generation Use of programming patterns for HP UFT and VBS Keyword handling Reporting Core libraries Default keyword implementation Keywords for: Data handling Data transfer Utility libraries Technology depending String operations File operations Check and compare Web functions SAP functions (...) Figure 7: Components of SQS UFT Framework The 7 best practices adopted in the SQS automation framework Describe test cases step-wise to maximise re-usability of automation building blocks Setup and maintain an action key word library for different platforms Create automation blocks to cover pre-processing, execution, evaluation and repetitive activities across the complete Test Process Automation (TPA) Schedule, control and report automated tests from a central driver of third-party tools, scripting, programming languages, OS and DBMS Apply overall code generation principles to minimise maintenance effort by putting this effort into a low number of action words and having minimal changes to scripts. Support distributed test execution on different platforms The framework provides standardised feature to integrate third-party tools and ensure ease of interfacing Combining the above principles and best practices, SQS has built an exemplary mixture of infrastructure types for a mobile test lab (Figure 8).

Bank on Mobile 14 Emulation Private cloud Public cloud Unit testing Unit integration testing Developer workspace Functional testing Regression testing Reliability testing Usability testing Security testing Test lab Functional testing Acceptance testing Cloud provider Test tools Figure 8: Exemplary mixture of infrastructure types

Bank on Mobile 15 Conclusion Benefits of SQS mobile test methodology Implementing test organisation offers the following benefits: Divides the given project scope into manageable pieces Allows transparent omission of test activities on a large scale Provides the structure to develop and maintain re-usable assets Quality & test Adaptation to additional quality aspects, test objects and test tasks Managing target diversity through multichannel testing Intelligent of test efforts through risk-based approach Integration with agile and lean software development approaches Functional & non-functional testing Early testing to reduce test effort and cost of error correction Mix of suitable non-functional test techniques to address performance and security requirements Test environments & test infrastructure Benefits of SQS mobile test automation Despite the challenges in mobile application testing, careful selection of target devices using device matrices, connectivity options and tools that maximise ROI ensures a cost effective mobile testing rollout. Using a mix of simulators and physical devices can maximise test coverage without the need to test every feature on each available device model The use of Wi-Fi networks for the majority of testing in combination with network simulation tools can reduce the cost and complexity of testing on various cellular networks Maximising automation is an effective way of expediting the testing process and reducing long-term testing costs Hardware independence The SQS framework is flexible to support multiple devices of different screen resolution, size (phone, phablet, tablet), etc. Re-usability Scripts captured in one OS can be re-used for another OS/version (scripts created for ios can be re-used for Android, scripts created for ios 7 can be re-used for ios 8, etc.) The SQS automated mobile testing framework is the right solution to achieve a successful mobile channel rollout in a short timeframe with better ROI. Systematic to maximise efficiency of test environment use for the various test tasks Strong support for automation across test environments and test tasks

Bank on Mobile 16 References [1] ING International Survey 2015 [2] Chetan Charma (2015). Mobile industry prediction survey. http://www.chetansharma.com/mobilepredictions2015.htm [3] IDC (2015). Worldwide smartphone operating system market share. [4] https://experitest.com/automation/ SQS Software Quality Systems AG, Cologne 2015. All rights, in particular the rights to distribution, duplication, translation, reprint and reproduction by photomechanical or similar means, by photocopy, microfilm or other electronic processes, as well as the storage in data processing systems, even in the form of extracts, are reserved to SQS Software Quality Systems AG. Irrespective of the care taken in preparing the text, graphics and programming sequences, no responsibility is taken for the correctness of the information in this publication. All liability of the contributors, the editors, the editorial office or the publisher for any possible inaccuracies and their consequences is expressly excluded. The common names, trade names, goods descriptions etc. mentioned in this publication may be registered brands or trademarks, even if this is not specifically stated, and as such may be subject to statutory provisions. SQS Software Quality Systems AG Phone: +49 2203 9154-0 Fax: +49 2203 9154-55 info@sqs.com www.sqs.com