View Point. Developing a successful Point-of-Sale (POS) test automation strategy. Abstract. - Sujith George

Transcription

1 View Point Developing a successful Point-of-Sale (POS) test automation strategy - Sujith George Abstract While Test Automation has been around for a while, QA teams in the retail industry are still struggling to extend it successfully to test POS systems because of certain inherent characteristics of these systems. The following document attempts to help QA teams understand the challenges involved in automation of POS testing and how they can be overcome by developing effective comprehensive strategy.

2 Organizations view test automation as a strategy to reduce timelines, cut costs and improve quality. While this is true, it is also important to note that before we reap the benefits of automation we need to make significant investments and not everything can be automated. ROI from automation depends on Automation Coverage, i.e., percentage of test scenarios that have been automated. With POS Systems accounting for a major portion of a Retailer s IT spends, automating POS testing is something that cannot be ignored. It is something that most Retailers want to undertake, but hesitantly, owing to the ambiguities surrounding POS Test Automation. Testing of POS applications is unlike other types of testing. While most applications can be tested using a desktop, testing of POS applications requires a Cash- Register which usually comes with a set of other peripheral devices such as receipt-printers, bar-code scanners, cash-drawers, pin-pad devices, swipe-readers etc.. Usage of these peripherals requires human intervention and this necessitates the physical presence of the tester. The POS devices come in a widerange of shapes and sizes: legacy POS devices with limited/no-display, to modern POS devices with touchscreen UI; stationary terminals to wireless handheld devices; Store- POS to Restaurant-POS. It, therefore, naturally follows that automation of POS testing too, is different from traditional automation. Based on our extensive experience from engagements involving testing of POS applications, we have identified a few factors which make POS testing a challenge Human and Peripheral Device Interaction The majority of the POS scenarios such as scanning a bar-code, swiping cards, pin-pad-entry, opening and closing of cashdrawer etc. involve peripheral devices, which require human intervention. Such scenarios are typically difficult to automate using a tool and hence the extent of automation is lowered to 15-20% of the test scenarios. Consequently realizing ROI takes much longer. Non-Standard UI Objects In order to provide users with a vastly improved user experience, today s POS systems come with thick-client applications and touch-screen UI. The systems are further customized extensively through use of non-standard objects. In fact certain objects are created specifically for the application, e.g. screen-shots of receipts. However, the flip-side is that most of these nonstandard objects cannot be recognized by standard automation tools, thereby making it difficult to automate the POS system. Dynamic UI POS software, whether custom-built or bought off-the-shelf, invariably ends up being customized to suit the retailer s specific needs. The UI is often highly dynamic to allow it to cater to ever-changing business needs. Further business processes are frequently modified and the cost and time required to maintain an automated regression test suite increases steeply and in some cases makes it infeasible to maintain one. Multiple Interfaces and Complex Configurations POS applications typically interface with a number of external systems, both front-end and back-end (such as CRM, Merchandising systems, Sales Audit, E-Commerce etc.). These external systems in turn could also be third-party systems built on a wide range of platforms. Most POS vendors end up having multiple versions/formats of POS hardware and software. Maintaining multiple versions of automated scripts for incremental changes in code and configuration is infeasible. 2 Infosys View Point

3 Strategy to build a Robust, Scalable and Re-usable POS Test Automation Suite Feasibility Analysis TECHNICAL FEASIBILITY Carrying out a Proof of Concept (PoC) on the application architecture under consideration helps determine the feasibility of automation. The PoC should focus on the following aspects: Technical architecture of the POS Application: understanding the technical architecture of the POS application is crucial at the PoC and tool selection stage. The applicability of an automation tool and even the choice of tool adapters and add-ins depends on the underlying technical architecture and communication protocols of the application under test. Evaluation and comparative analysis of automation tools: Tools differ in their features, advantages and disadvantages. If time permits a comparative analysis of two or more tools should be carried out to evaluate the tools for fitment. A scorecard based comparative analysis ensures short-listed tools are quantitatively evaluated against criteria which carry specific weightages. POS systems, by virtue of being thick-client applications, are typically customized a lot. Therefore the evaluation process should also include a check of whether the tool is able to recognize non-standard UI objects employed by the POS system under test. Understanding of the application: A high level understanding of the application not only helps create robust automation scripts but also modularize them. Commonly used actions/ functions can be scripted as Keywords and re-usable Business Process Components. Selection of scenarios for PoC: During the PoC scenarios selected for automation should be chosen such that they are representative of possible technical challenges that the QA team may encounter during the automation of the POS testing. FINANCIAL FEASIBILITY Before we undertake automation we must assess the financial feasibility of the initiative. While assessing the financial feasibility of the automation initiative we need to consider ROI (Number of regression test cycles which need to be executed to break even) and include investments which would need to be made as part of the long term strategy. The ROI we are able to extract depends on the extent to which we can automate the test processes. A traditional approach to POS automation, sees QA teams typically realize ROI in ~20 releases. Infosys View Point 3

4 However, this can be accelerated by extending the automation coverage and adopting a ready-made POS automation framework. In fact the automation framework should be developed in such a manner that it includes reusable business process for POS systems. In this manner we can not only eliminate the time and effort required to build a framework from scratch but also reduce significantly the time required to develop scripts for functional scenarios. The financial feasibility analysis should include long term investments which need to be made to ensure a robust, scalable and re-usable automation suite. Investment considerations should include: Procurement of the right automation tool with the required add-ins. The add-ins help overcome technological obstacles and ensure compatibility between the automation tool and the POS applications. Upgrading POS Registers: the memory of POS Registers, which need to host the automation tool, needs to be upgraded regularly Simulators: Most POS scenarios involve humans interacting with peripheral devices. Wherever possible such actions/calls should be simulated to ensure scenarios are tested end-to-end. Process & Methodology KEYWORD DRIVEN FRAMEWORK Advanced techniques such as a Keyword driven approach, rather than the traditional Record & Replay approach, enable high degree of modularization and promotes reuse. Common POS functions such as item lookup, price lookup, credit-authorization, void etc. can be scripted as re-usable functions / keywords and invoked by multiple scripts addressing different test scenarios. Importantly modularization also enhances the maintainability of test scripts. DATA PARAMETERIZATION Hard-coding of data into scripts makes them difficult to maintain a robust automation script should be data driven and designed such that test data can be supplied and manipulated at run time. This not only ensures flexibility at the time of executing test scripts but also ensures better quality of testing. EARLY AUTOMATION Choosing an automation framework which supports early automation helps improve the ROI. Traditional automation frameworks require the application-under-test to be fully developed and stable for scripting to start, and therefore only regression testing can be targeted for automation. On the other hand, a framework built on the principles of Early-Automation can enhance the scope of automation and in turn help realize ROI faster. For example, a framework can be used to define Business Processes and validation rules in terms of models that are abstracted from the underlying scripting technology. Such models can be created before the application is functionally stable. Once the application stabilizes and is available, automation scripts can be generated from these models. Effectively this allows test automation to begin as early as the application build stage itself. SCOPE OF AUTOMATION It is important to identify the right scenarios for automation. It would be a waste of time, effort and money to automate scenarios which will not be executed often during successive test cycles. A Scorecard based approach helps identify the right scenarios for automation. The approach, assigns scores) to scenarios (on a scale of 1-n) against pre-determined evaluation criteria. The criteria themselves carry differing weightages. Scenarios/transactions that score higher than a threshold value are selected for automation. Typical transactions that can be automated are: Sales / Return / Exchange for different tender types like Cash, Debit, Credit, Gift and Retailer Cards Price change, Discounts, Promotions, Capture of Customer Information Merchandise Locator, Price Lookup etc TEST MANAGEMENT Integrating the automation suite with test management tools such as HP Quality Center, IBM Rational Test Manager etc. allows us to leverage the UI features of the test management tool. This makes it easy to understand and report the test execution status. 4 Infosys View Point

5 Environment & Infrastructure The test environment, data and infrastructure used for POS Test automation play an important role in ensuring the quality of testing and should ideally be a scaled down version of a real-time store. Unlike a manual tester, an automation script does not have the intelligence to verify the validity of the test processes. The test data setup for automation too should be absolutely error free. The following points should be given due importance to ensure what is being automated is a true reflection of real-life business scenarios. TEST ENVIRONMENT Ensure same hardware, technology, platform used in the production environment is made available for testing activities too. Reserving a few registers exclusively for developing the automation suite helps expedite the process and improve the quality of scripts. It is also difficult to utilize the same set of registers for both manual and automation testing due to the limited memory of the registers which is used up during the installation of the automation tool. DATABASE BACKUP Over the course of multiple test cycles data integrity is affected adversely and it is necessary to restore the database to a previous baseline. Hence maintaining a backup of the database at regular checkpoints is essential. TEST DATA Manual testing, which includes exploratory testing, usually affects test data which gets altered during the course of the test execution. Hence it is important to maintain separate test databases for manual and automated testing. CODE VERSIONING AND RELEASE MANAGEMENT To ensure right scripts are executed on right environment & right version of the application code Tools & Accelerators TOOLS EVALUATION AND SELECTION Automation tools constitute a significant portion of the investments and one needs to ensure that the tool being procured is compatible with existing technology of POS systems. Hence it is important to evaluate the automation tool vis-à-vis application Architecture and, Technology spectrum. The automation tool needs to be compatible with the technology platform on which the POS system is running`. We need to pay special attention to the following: 32-bit or 64-bit: Automation tools for Windows are generally built to work with either 32-bit or 64-bit applications. Some are even designed to work with both. It is important to understand whether the POS Application-Under-Test (AUT) is a 32-bit or a 64-bit application and accordingly determine the suitability of a tool. For example, earlier versions of QTP are known to support only 32-bit applications, whereas newer versions support 64-bit applications too (with some limitations). Operating Systems (OS): OS manufacturers are known to supply versions of OS that are tailor-made to support a specific POS system (e.g. Windows Embedded OS for POS). Such customized OS incorporate drivers and libraries specific to the POS hardware. While selecting an automation tool we need to ensure it is compatible with the specific OS. Touchscreen UI: Automation tools sometimes fail to recognize objects used in modern POS systems with touchscreen UI features. For example, touchscreen UI (developed using Java Apps Support Framework), when spied using QTP, is displayed as default WinObject with Standard Properties set to Null (including x- and y-coordinates). Such touch screen UI can be automated using tools which simulate object methodology (object region is mapped to standard objects like Button, Textbox, Checkbox, Radio etc.). Infosys View Point 5

6 ACCELERATORS The process of automation itself can be expedited using accelerators. Technology Accelerators: These can either be technical workarounds to help the automation tool recognize nonstandard UI objects, or approaches/methodologies to simulate peripheral devices attached to the POS terminal. Alternatively we would use ready-made platform-independent POS automation frameworks too. Besides increasing automation coverage in general, these workarounds can save time, cost and effort. Domain Accelerators: Pre-built knowledge of POS domain in the form of libraries of critical POS business processes. Conclusion A one size fits all approach prevents successful automation of POS systems. Not only is testing POS systems different from other types of testing, it also varies from one POS system to the other. Hence to successfully automate the testing of POS systems one requires a deeper understanding of the POS-specific challenges which QA teams may encounter and the factors which need to be considered while automating the test process. These challenges can be overcome by adopting a comprehensive strategy which focuses on selecting the appropriate tools, accelerators and a structured process. 6 Infosys View Point

7 About the Author Sujith George, Senior Project Manager, Infosys Sujith George is a Senior Project Manager at Infosys Independent Validation and Testing Services practice. With over 12 years of experience, he has managed large test engagements for clients from the retail industry and has helped them define and implement productivity improvement initiatives. Sujith has a keen interest in Point-of-Sale (POS) Testing and has developed the test strategy for several POS Testing engagements. Infosys View Point 7

8 About Infosys Many of the world's most successful organizations rely on Infosys to deliver measurable business value. Infosys provides business consulting, technology, engineering and outsourcing services to help clients in over 30 countries build tomorrow's enterprise. For more information, contact Infosys Limited, Bangalore, India. Infosys believes the information in this publication is accurate as of its publication date; suchinformation is subject to change without notice. Infosys acknowledges the proprietary rights of the trademarks and product names of other companies mentioned in this document.