At the Heart of Testing Effectiveness

Transcription

1 At the Heart of Testing Effectiveness Testing on the Fly Effective Management for Successful PSS Migrations Abstract In the current business environment, organizations face multiple challenges including outdated systems, newer technologies, and changing business processes. The airline industry is no exception. To manage passenger expectations and demand for seamless personalized services at each stage of the journey, a number of airlines are migrating current systems to different Passenger Service Systems (PSS). PSS migration programs are complex and critical as they impact most of the other applications and can potentially affect airline operations. One of the key factors to successful PSS migration is management of the test data and test environment at all stages of testing.

2 Ushering Fundamental Change PSS governs critical airline systems such as airline reservation system, airline inventory system, and a Departure Control System (DCS). PSS holds a wide range of product information, as well as important passenger data (such as journey history and customer preferences) that flows across applications. Changes in business strategy and obsolescence of existing systems necessitate the need for PSS migration. PSS migration involves business change, IT system change, and data migration, all of which trigger the need to train the business team. Changes in the airline industry are driving both large airlines and low cost carriers to consider migration to PSS. Cathay Pacific Airways and Singapore Airlines migrated from their in-house Mainframe systems to Amadeus; the LAN Group and Etihad left Amadeus for Sabre; Virgin Australia moved from Amadeus and Navitaire to Sabre. PSS migration entails a fundamental change in the way airlines operate. Its successful implementation requires Management (TDM). TDM ensures that parallel testing activities can take place without impacting each other. This is particularly important for migration projects where it is critical that migrated data is not being modified by other processes. Parallel modifications may potentially result in incorrect test results and require additional effort in defect analysis. Testing determines quality and reliability in PSS programs to ensure smooth transition Migrations are complex, data must be protected, correctly migrated, and unmodified to avoid any impact on operations and test results Data-related tasks occupy about 40-50% of application development and testing time Only 10% of production data can be made available in test environment for testing 69% of IT professionals use production data for testing of their applications 50% of business data is considered confidential and cost about a million dollar data breach to a company Figure : Insights

3 This paper explores an effective approach to test data management for smooth PSS migration that reduces the effort spent in test data creation, maintenance, and idle wait time. It proposes that test data managers can ensure that test data is available to all without overlap and invalidation. Containing Chaos: The Challenge of Migrating Data The complexity of PSS migration has made TDM even more challenging. It is, therefore, crucial to maintain secure integrated data over repeated test cycles and other non-production environments. Ineffective governance policies results in the following challenges: Limited availability of environment while multiple teams are accessing the same data There are only two to three environments available to perform development, testing, and training. Among these, one environment is designated for production. PSS data is accessed by a number of teams simultaneously viz. testing team, business team, PSS owner, and interfacing application owners. This means that there will be a number of user groups performing different activities in each environment. An integrated test environment poses the risk of test data overstepping, making the data invalid for other users. Aggressive delivery plan does not allow time delay or wastage Data mining, data preparation, and other data-related tasks consume around 40-50% of development lifecycle time. This means that the user groups must follow data management guidelines to ensure that there is no chaos or misconception when using data, especially migrated data in each environment. Specific data may be required for validating application functionality. User groups must also have a clear idea of their data requirements ensuring data variety, test coverage, and scheduled delivery. Non-production environments have space and performance constraints With full dump of production data not advisable for use due to storage constraints, a subset of production data is migrated.

4 Given the large volume of data flowing across applications and interacting with the PSS and numerous other transactions generated from different use groups, performance is of vital importance. Judicial use of space necessitates liaising with different user groups and selection of correct subset of production data ensuring full coverage of test cases and fulfillment of user groups requirements. All environments reside in a multi-hosted client environment These environments have maintenance activities occurring at scheduled times and user groups must work within this maintenance schedule. Any transactional or new data created by user will not be stored for use in later test cycles; hence test schedules and refresh cycles need to be properly aligned. TDM protects the usage of migrated data, ensuring that all user groups working in each environment can access both migrated and new data that suits their work scenarios and perform activities without impacting data of other user groups. Absence of TDM would result in chaos between the teams using test data indiscriminately, invalidating the test data created for other purposes. It would also result in delays arising from insufficient test data and conflicting test data requirements. Management to the Rescue Effective Management, a coordinated and easy-to-manage process, improves testing effectiveness and reduces time and cost of providing high quality, safe test data for smooth PSS migration. Our TDM framework depicts different components of test data management in sequential manner. The objective is to cater to the challenges user groups face and define best practices of test data management.

5 Management Framework Capture Data Request Define Filter Criteria Transform and Load Data Maintain Data Workflow management system for request provisioning Understanding data requirements Criteria for production data Criteria for new data generation Communicate filter criteria Extract data based on filter Subsetting and Masking Load data in different environments Guidelines or search criteria to access data Configuration management backup and versioning Data quality audits Manage and define request for interfacing systems Run utilities to generate new data Periodic data refresh Notify users Core Objectives of Management Approach Develop a TDM process to support filter-based data extraction, data creation, workflow, and data management Publish data management guidelines to avoid any chaos for data usage and reservation Timely data availability in various environments considering execution schedule and environment maintenance and data refreshes Leverage TDM tools and techniques to simplify, automate, optimize the test data use and new data creation Apply data subsetting techniques based on filter criteria to extract realistic data Apply data privatization techniques to safeguard customer privacy/security Quickly and periodically refresh data in Test environments Maintain reference copy for reuse across testing phases Figure: Management Framework The PSS owner performs the following tasks to counter the challenges faced in test data preparation: Data extraction Data subsetting and masking Data loading in different environments Configuration management Quality audits and data refresh To ensure coordination and communication, organizations need a who is responsible for: Availability of test data for all test phases Coordinating with the Environment Test and ensuring that the test data is available for testing Actively managing test data requests for applications and setting up data sets as per the master test strategy to ensure there are no delays in testing Ensuring that all user groups in each environment can access the required data and perform activities without impacting other user groups data. We also leverage on Management Governance approach that describes actions for effective test data management in PSS projects. User Groups Interfacing Third Parties PSS Owner Dev QA Training New Data requests PSS Data requests Training Data requests Data Request based on filter criteria Masked prod Subset Test Environment 1 Data Requests Workflow Management System New Data Generation Test Environment 2

6 Different users perform different tasks in the test data management framework. Task Objective Tasks Owner Tools 1. Capture Request 1.1 Different user groups provide test data requests to in predefined data templates. There is a common interface for placing test data requests and data management activities. 1.2 We use centralized test management tools such as JIRA and TFS for workflow management. The tools provide visibility to the test data management activities including status of data requests, interface for calendar publication within the test environment refresh periods and other broadcasts. 1.3 The requests need to be made two weeks in advance for the to comply with the requirement. Ideally, test data requirements should be made during the test design phase. 1.4 Specific requests for data reservation requirements need to be made. For example, a user wants to test a scenario of overbooking a flight. The user needs to book the whole flight and bar further booking on the flight by some other user. Similarly cancel flight or close flight scenarios may need such reservations to be made. This is mandatory to avoid data-overstepping. 1.5 discusses with different user groups to understand data requests, if needed. 1.6 Liaise with business SMEs as they have a good understanding of data. User Groups (QA/Dev/Training) JIRA/TFS (workflow management) Predefined templates (data requests) Define Filter Criteria 2. Define Filter Criteria for PSS 2.1 Depending on the data requirements of each user group in different environments, the may need to define separate filters for each environment. 2.2 Define criteria for selecting the PNR records to be migrated to different environments for each data refresh. Filter should be based on flight numbers, and markets covering domestic and international, short-haul and long-haul flights for a specified departure period. 2.3 Coordinate with data migration team to identify data in the production system that satisfies the requirements. PSS owner

7 Task Objective Tasks Owner Tools Ensure that the data is sanitized to cater the privacy regulations Take latest production snapshot to avoid issues with obsolete data Script for any defect fixes is executed before using the data 2.4 Create dummy flight and PNR using tools/apis/script, if required for specific testing needs. This may include bulk PNR data requirement in different environments. The data specification details should be placed in the spreadsheet/data template. 2.5 Carefully define filter(s) to ensure that the selection of data contains enough scenarios to allow user groups to work with each of the environments. 2.6 Manage requests for creating new data for user groups and coordinate to load data for these requests. These requests may be for new data such as PNR (possibly) or data scenarios that cannot be found in the environment. There are some new functionalities that are not productive. For example, data related to check-in of international passengers may not be found in production environment if the international route has been rolled out recently. 2.7 If there is insufficient variety of data scenarios that can be picked out of the selected data, then there is an option of creating specific data in the current production systems for data migration and loads during data refresh. The needs to manage this data in order to avoid a situation where this data affects live data. 2.8 holds calls, meetings and formal reviews of documents to ensure alignment of all teams on test data requirements. 2.9 will communicate filter criteria with the PSS owner The will specify the filter for selecting and moving PNR records from production to different test environments. 3. Request/Response for Third Party Interfaces 3.1 Data requests involving the interfacing third parties should be specifically managed to ensure that the data in these interfacing systems are synchronized with the data in the PSS environment. 3.2 The will liaise with respective user groups and owners of the interfacing systems as well as the Test Environment to determine these requirements. This will hopefully enable the interfacing systems to be tested or training on these modules to be performed without impacting the actions that are taking place in the test environment. Transform and Load data 4. Load Migrated Data as per filter criteria 4.1 PSS owner will move PNRs from production to different environments based on the filter criteria provided. PSS owner CA

8 Task Objective Tasks Owner Tools 4.2 PSS owner will consider data subsetting and privacy regulations for production data. 5. New Data Generation 5.1 validates and approves test data creation and management tools. 5.2 PNR generation test data utilities will be used to create new test data in different environments. PNR generation utilities/sql procedures/shell scripts 5.3 Command-based emulators may be used for bulk PNR generation. Shell scripts and SQL procedures may also be used for this purpose. 6. Notify 6.1 will notify the user group that the data is set up for use. This will be reflected through workflow status as part of the workflow management system. JIRA/TFS Maintain Data 7. Post Data Setup 7.1 Establish simple guidelines for accessing migrated PNR / flight data in different environments so that user groups can use migrated data for their activities without interfering with one another. Any data reserved for a particular user group needs to be clearly specified. TFS 7.2 Test environment manager or PSS owner to store the copy/backup of data, assign a version number to the data and then make a note of the version number along with data details. This allows restoration of the environment to original condition with minimum effort, as and when required throughout the testing phase. Due to factors such as execution of negative scenarios during execution, some inconsistency may creep into the test environment and may require restoration. PSS owner 8. Transition of TDM Post Cutover 8.1 TDM process documentation 8.2 Best practices to be followed 8.3 Lead time required for any type of data request / PSS owner See the Bigger Picture Several benefits are envisaged by following a streamlined process where all requests are funneled through the TDM team: Reduced effort involved in the overall TDM process due to workflow automation, data request templates, and process improvement No rework and data overstepping with reservation and distribution of test data Reduced efforts with reusable tools Reduced time-to-market with no delays in test data delivery Increased defect detection efficiency due to availability of correct, timely data Improved productivity and overall efficiency Higher customer satisfaction and growth

9 The NIIT Technologies Thought Board: Testing on the Fly Which Processes are Governed by Passenger Service Systems? Reservation processes Inventory processes Departure control processes Which Large Airlines and Low Cost Carriers are Considering PSS Migration? Cathay Pacific Airways Singapore Airlines Etihad Virgin Australia What Tasks does a PSS Owner Need to Perform? Data extraction Data subsetting Data loading in Configuration Quality audits and and masking different management data refresh environments What Tasks is a Responsible for? Round-the-clock Coordination with Management of Setting up of Accessibility of Permitting user availability of environment test test data data sets as per required data by groups to test data manager requests for the master test different user perform activities applications strategy groups without any impact on other groups data What Actions should be Performed for Effective Management in PSS Projects? Capture Define Request/respond Load Generate new Notify user Post data Transition requests filter to third-party migrated data groups when setup TDM post criteria interfaces data as per the data is cut-over the filter available for criteria use

10 Streamlining Data Management Effective test data management is crucial to ensure success of PSS migration projects. Dedicated test data management teams managing the test data setup and administration through appropriate TDM frameworks would help ensure accurate test data is provided in the test environment as per user requirements. The challenges in test data management can be addressed through implementation of a workflow-based request provisioning system for timely test data request, status and provisioning, tools for test data generation, effective coordination with user groups, improved defect detection, and increased customer satisfaction. References Tools providing a holistic solution to test data management like CA can be leveraged to automate the test data management process. Data acquisition, subsetting, masking, test data generation, test data matching and coverage are some of the vital features of this tool toward improving the testing efficiency.

11 For more information, contact 2016 NIIT Technologies. All rights reserved. NIIT Technologies is a leading global IT solutions organization, differentiated on the strength of domain expertise; it services clients in travel and transportation, banking and financial services, insurance, manufacturing, and media verticals. Leading with its service vision New Ideas, More Value, NIIT Technologies is committed to delivering new ideas combined with operational excellence to provide exceptional value to its clients. The Company is focused on helping businesses design sustainable, optimizable and winning digital operating models, enabling them to become agile, scalable, and flexible. Visit us at D_133_ Stay connected: