contents AUT H O R : S W A T I J I N D A L Effective Testing & Quality Assurance in Data Migration Projects Agile & Accountable Methodology Executive Summary... 2 Risks Involved in Data Migration Process... 4 How to Ensure Quality Assurance in Migration Projects... 6 Efficient Testing Strategy in Migration Project... 9 Reference Material... 12 L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 1 of 12
Executive Summary What is data migration? Data migration is the process of moving data from one environment to another. It is different from ETL where the data is moved between the existing environments. However, these ETL processes are sometimes used to support the migration program. Data Migration is usually supported by a variety of tools and techniques in order to accomplish the efficient migration. Migration of data is a process of moving data from one system to another i.e. from the legacy source systems into a new system, known as the target system or the application. It is considered as an opportunity for evaluating and improving data quality that can have a critical impact on business after the new application goes into production. Data Migration is frequent in IT business and covers considerable budget and resources. The combination of the occurrence of data migration processes and the resources consumed in a data migration project results in significant amount of the IT budget for any company. As data storage architecture becomes larger and more complex, data migrations are also becoming more complex and risky. Organizations should manage their migration projects more effectively as it involves large share of their IT budgets. Data Migration Process When is data migration done? Change in one database to another, like from SQL server to Oracle database A change in version of the database in use Upgrades or changes in servers or storage equipments Data storage centers are relocated L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 2 of 12
Data migration a big challenge Data migration has become one of the most demanding proposals for IT companies. Due to bulk data, a high level of risk is involved in these projects. The major challenge of the migration project is to ensure that the data is retained in its original i.e. the content and metadata remains unchanged.in order to reduce risk and guarantee that the data has been migrated and transformed successfully, it is essential to employ a thorough Quality Assurance strategy in migration projects. Testing is a key phase of migration project that delivers a successful migrated data and addresses issues which occur prior and after the migration. This paper first covers the risks involved in the data migration process, points out how to mitigate these migration risks by effective quality assurance techniques, and how to achieve success in the data migration projects. L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 3 of 12
Risks Involved in Data Migration Process Data loss risk Sometimes, during the migration process, the business objects to be migrated are missed in the target or new system and sometimes few extra records come in the target applications that are not actually present in the source application. Data loss risk involves the risk of whether all the content and metadata is migrated into the new system or not. Semantics risk Semantics risk arises when the migration is done efficiently, but, some units present in few fields are not considered, resulting to inconsistency in data. So, it may happen that the units in the new system are different to that of the source application. Due to these different notations of few fields, sometimes wrong data can go to the target or new system. For example, if we have a field called amount in the source database and it is getting migrated to base amount in target application. In the source application, this particular field deals with the Indian rupee but at target side, the base amount field considers US dollars. If 10 INR (Indian Rupee) is getting migrated to 10 US dollars, it is a mistake because 10 INR can never be considered as 10 US dollars. These types of risks are known as semantics risk. Extended downtime risk This type of risk can be encountered when the migration takes longer time than expected. Because of delay, the application will not remain in use which in turn will afftect the stakeholders. So the organizations should always keep in mind and consider this risk while planning for the migration. Data corruption risk Data corruption risk occurs when some business rules or validations are applied on the target application. Due to these validation points, sometimes, unwanted data gets dumped into the database of the new application. This unwanted data is not prevented by the database and because of this, the application crashes or does not work as expected. Thus, it is a risk for the end user that they may get an unexpected error while using the application. The application stability risk This type of risk arises when the target application is not stable. The target application can be unstable because of many reasons like improper development or coding for the new system or improper coding of business requirement or functionalities in the new system. Hence, it is a risk that the new system might not fulfill the business needs and is of no use for the business end-users. Orchestration risk Orchestration risk involves the order in which the processes are executed in the migration project. It consists of multiple processes to be performed in a planned manner. Since there are various dependencies between different business objects, the order of their migration is very important. Interference risk This type of risk appears when all of the stakeholders are using the source application simultaneously during the transition period. For example, if one stakeholder is accessing a particular table and he locks that table, and if any other person tries to access that table, he would not be able to do. In such situations, interference risk arises. L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 4 of 12
Target application parameterization risks Parameterization risks usually occur due to the target application. If the target application has few restrictions, like it does not support few platforms, etc., it can become incompatible with the data migration programs. For instance, if a new application has a constraint that it does not allow to add new clients/markets, then it will become incompatible with the migration program and will lead to errors. L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 5 of 12
How to Ensure Quality Assurance in Migration Projects The main goal of quality assurance in data migration project is finding faults and ensuring quality in the data migration process. An effective testing approach can be applied to reduce or minimize the risks involved in the data migration process. There are two different testing techniques used for effective testing in migration projects, namely, Data Validation Technique and Migration Run Test. Data validation technique is further divided into two categories i.e. completeness type and semantics. Similarly, migration run test is divided into two categories i.e. partial migration run test and full migration run test. The detail classification of the two different techniques is well depicted in the picture below. Mapping different testing techniques to risks involved in data migration process Different testing techniques are used to mitigate the risks present in the data migration process. A testing technique suitable for mitigating one of the risks in data migration may not be used to resolve the issues generated from other risks. So, it is very important to know different mitigation techniques and their suitable mappings to each of risks involved. The following table clearly depicts the different types of risks and the best testing techniques used to mitigate the same. Risk Testing Technique Data loss risk Semantics risk Data corruption risk Application stability risk Orchestration risk Interference risk Target application parameterization risk Completeness type Apperance, processability, integration Appearance, processability, integration Partial/full migration run test Partial/full migration run test No particular testing technique used as it is an operational risk Appearance, processability, integration and completeness test L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 6 of 12
How to mitigate the data loss risk? Data loss risk can be mitigated by performing completeness tests. These types of tests are used to identify missing business objects that are present in source database but are not found in the target database. This process or test is generally known as Reconciliation process in which the count of source and target is taken and compared to ensure that complete data is migrated from source to target. Reconciliation is the only test that covers almost all the data present in the system. It is very important to test the mapping between the sources and target fields because sometimes the mapping changes between sources to target attributes. Reconciliation test fundamentally checks the objects missing in target database and also the extra objects coming in the target database. After identifying these objects, fresh code is deployed to overcome these mismatches. How to mitigate the semantic risk? Semantic risks can be overcome or minimized using any one or combination of the techniques mentioned below. Appearance tests check the presence of objects on the front-end i.e. User Interface (UI) of the application. Testers manually compare objects present in the source and target application by looking at the main screen of the application. For example, if in the source application one field is for amount and its unit is US dollars. Testers can manually look in the target application and check whether the same field has same unit or different. Also, sometimes the source application fields deal with decimal points upto 2 but target application doesnot consider any such constraints. So, it can be noticed by Quality Assurance (QA) team member by performing testing at Graphical User Interface (GUI) level only, as it can be identified easily. Processability test is used to make sure that there is successful interaction between the target application and the newly migrated data. This test processes the migrated data that helps in identifying the inconsistencies and incompatibilities between the migrated data and the parameterization of the target application. The data migration process marks the data straight into database tables and the schema of the database is not concerned with the units of the fields. If appearance test is used for this particular check, it will not fail, as nothing can be inferred from the UI. But when someone processes the data, the application crashes. These types of errors can only be found when the sampling of data and test cases are written properly to incorporate these types of mismatches. Integration tests are used when an application is not independent but is interlinked with other applications as well. If there is a change in one of the applications, its impact on the other application must be tested. Hence, all the functionalities of the target application with the migrated data in context of its interlinked applications must be tested. L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 7 of 12
How to mitigate the data corruption risks? Data corruption risks can be mitigated by using the testing techniques like appearance tests, process ability tests and integration tests. These tests are already explained above in this paper. How to mitigate the orchestration risks? Orchestration risk can be minimized by performing full migration run tests. Also a partial migration is sufficient if a data set is created that covers all the interactions between the different migration processes. Migration run tests validate the data migration programs from the starting i.e. profiling, extracting, transforming and loading data in the target application. Migration run tests are further divided into two categories i.e. full and partial migration tests. Full migration run tests uses all the migration programs with the entire data set available in the application. In this test, the execution time of the overall data migration process is which directly influences the downtime of the application, i.e. the interruption of the business operations. Accordingly, the stakeholders can plan which business objects takes shorter duration and which are taking long time to migrate, and hence, they can plan their migration process accordingly, so that there will be minimum impact on the application. Partial migration run tests migrates few business objects in less span of time and also speed up the overall migration process. Hence, the development is done in lesser time and it can be tested simultaneously. But as a consequence, there is a higher risk of discrepancies in the migrated data. Since, in these types of tests, small set of data is used in the migration, many a times the critical data is ignored and because of that the application crashes more frequently. How to mitigate the interference risks? The interference risk is an operational risk, that cannot be addressed by testing. It must be managed at the organizational level only. How to mitigate the target application parameterization risk? The parameterization risk requires determining whether all data could be migrated, or some data is not accepted by the new application, or the data was migrated completely but the target application crashes somewhere and does not work properly. Thus, mitigating the parameterization risk can be covered by the combination of testing techniques used to address the completeness, the semantics, and the corruption risks. L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 8 of 12
Efficient Testing Strategy in Migration Project The success of the migration project depends on proper planning, strategy, approach and testing techniques used in the particular project. The following steps should be taken care by the QA team for successful implementation of the migration program. Analyze the business requirements i.e. the staging area requirements and data integrity check requirements Prepare the migration test plan Identify success and failure conditions and also the application interface requirements End-to-end migration plan testing and validation Performance and tuning testing of migration process by recording the execution time and checking it against the acceptance criteria Application acceptance test and sign-off Post-production support to eliminate any issues that may occur when the system go live Pre-requisite in migration program to be monitored by all the stakeholders involved Before migration The organizations must look for the expertise in the migration area so that he can provide better guidance for the migration process. The data migration project involves specialized skills, tools and resources. But sometimes the resources identified for the migration project may not have the essential knowledge to carry out the migration program. All stakeholders must be informed in advance of the migration project so that they are well known about the time period of the migration process, the duration the old system will not be in use, and benefits accrued through migration of the legacy system into the new application. Ratify the working condition of old systems and address the issues found during migration. Ensure that the backup of the old environment or system is taken so that if the migration fails, the data can be reloaded or migrated again. During migration Always be interactive to all the end users and other stakeholders when the migration process is in progress so that they will not raise the incident/service tickets to resolve the issues occurring in the existing application. The environments or platforms used for migration should not be changed during migration. In the planning phase only, the estimations for proper backup of environments should be considered so that the backup is taken properly and is not impacted in any chance by the system failure. After migration All the failed items should be reviewed, migrated and inspected well to ensure why they failed to migrate. All the stakeholders should be informed about the expected time when the new system will come into existence. The original data and its backup should be deleted after the successful migration. L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 9 of 12
Case Study- Data Migration for Leading US-based Reinsurance Major The client The client is a leading global risk and reinsurance specialist. It has over 90 years of experience in providing integrated solutions to the (re)insurance industry. With more than 50 offices around the world, it delivers a powerful combination of analytics, reinsurance broking expertise and strategic advisory services to help our clients achieve profitable growth. As our clients' most trusted strategic advisor and reinsurance broker, we leverage our intellectual capital to anticipate and solve a range of emerging strategic business challenges and opportunities. Overview of the project In the project, the old system used was a suite of custom web-based software, developed internally to harness the power of FileNET, the premier image vendor. It provided a comprehensive and enterprise-wide approach for image documents, original format documents, and workflow management as it relates to the client business. The documents from different source systems are present in the old application through which the documents can be send to the intended recipient. The client decided to move from old to new system and send documents through new application. The migration project required moving all technical claims related data and documents from one system to other. There were two levels of testing involved in the project Integration Testing & Migration testing. In integration testing, we have business requirements for different functionalities of the application. It also involves how different functionalities are integrated with each other. So, as per business requirements, different tests were performed to test whether the application is as per the business requirement or not. In migration testing, all the documents related to one source system were migrated to the new application so that the old system will not be used by the users. Challenges faced during the project Since it was a migration project, there were several risks associated with it. Different testing techniques were used to mitigate those risks. For efficient testing, the strategy was to focus on the major risks generally encountered in the migration processes. The testing was done for all the risks discussed in earlier topic of this paper to ensure the success of the migration project. Some of the major challenges faced during the project are: The major objective was to test whether all the documents related to one particular category are migrated properly to the new system or not. Since, there were huge number of documents to be migrated to the new system, it was not feasible to check each and every document (Completeness Risk). The big challenge was to test the metadata of the documents migrated in the new system. Metadata is an attribute associated with documents, such as document ID, owner, created date, etc. without which, the document serves no purpose. So, to ensure correct data migration, proper metadata mapping was required to integrate with new system (Semantics Risk). Ensuring that no other data or document had any effect or breakdown as a part of regression in old legacy system, since, the old system is huge and a part of it was migrated in this project. (Data Corruption Risk). The overall volume of data migrated was very large, so, for effective testing, the whole migration process itself presented a greater challenge. L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 10 of 12
Solution to the problems faced during the project Since there were huge number of documents and metadata migrated to the new system, the effective testing strategy and approach is used to overcome the problems faced. The major objective was to analyze all the risks involved in the migration process and perform the testing accordingly. Few steps taken to overcome the risks highlighted above are as follows: The first major step was to check whether all the documents in the old system are migrated or not in the new system. To mitigate the completeness risk and ensure better coverage, the test was applied on different types and names of the document, to ensure all different combinations of documents were taken. Second important check was to test the metadata associated with documents migrated in the new system. The mapping between the source and the target attributes/columns/fields were provided by the business. We use Data Testing Framework (DTF), a proprietary tool of L&T Infotech, through which the Structured Query Language (SQL) queries were built in both the old and new database and are compared for any discrepancies. By using this tool, the large number of document s metadata were compared in one test case. The greater coverage and the reduced testing efforts were accomplished by the tool usage. Integration tests were performed to ensure that new data was not corrupting the other document, functionalities and other data sources. For the mitigation of all the possible risks, testing was done efficiently to cover all the risks. Defect analysis based on different risks encountered In total there were 30 defects raised Out of 24, 15 defects were related to data loss risks 7 defects were due to application stability risks 5 defects were target parametrization risks 3 defects fall into the category of others i.e. related to performance and security issues Conclusion L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 11 of 12
Creating an effective data migration testing strategy is critical to reduce risk and deliver a successful migration. So, to ensure the same, appropriate steps were taken in the project. This project itself presented major challenges by large volumes of data to be migrated. Selective migration of these large volumes of data came up with its own level of challenges such as integration tests and impact on other functionalities. Proper planning, ensuring usage of correct mitigation techniques and extensive use of automated tools (DTF), helped us to cope up with all the challenges and make the deliverable an out & out quality product. Reference Material http://en.wikipedia.org/wiki/migration_testing http://www.datamigrationpro.com/data-migration-articles/2009/11/30/how-toimplement-an-effective data-migration-testing-strateg.html http://www.syntelinc.com/solutions.aspx?id=260 About the Author About L&T Infotech Swati Jindal is an ISTQB Certified Sr. Test Engineer at L&T Infotech, working in DWH/BI Testing in Insurance domain. She has a development background in VB 6.0 technology and has sound knowledge of SQL Management Studio 2005. She also has experience in data warehouse testing including datamart and report testing. Swati has worked on different platforms in database testing i.e. Teradata, Oracle, and SQL Server Management Studio and is well versed in creating SQL scripts and validating huge data through it. L&T Infotech is a wholly-owned subsidiary of the USD 14 billion plus Larsen & Toubro (ranked 4th in the global list of Green Companies in the industrial sector by Newsweek international) which has a presence in construction, engineering, manufacturing and financial services. L&T Infotech is ranked by NASSCOM among the top 10 software & services exporters from India and among the top 20 IT BPO employers. L&T Infotech provides end-to-end solutions and services in BFSI; Travel & Logistics; Media & Entertainment; Healthcare; Energy & Process; Utilities and E&C; Hi-tech & Consumer Electronics; CPG, Retail & Pharmaceuticals; Auto & Aerospace; Industrial Products; as well as Testing; Mobility; Infrastructure Management System; BI/DW; SAP; Oracle and Microsoft; Enterprise Integration; and Manufacturing Execution Systems in addition to an innovative CIOthought partnership program that provides a value-driven edge to clients. L&T Infotech s horizon is filled with the promise of new and cutting edge offerings including an end-to-end cloud computing adoption toolkit and cloud advisory consulting services; mobility solutions; big data advisory services; and in-memory computing. L&T Infotech has developed intellectual properties (IPs) in all the vertical and horizontal service lines and leverages them to provide IP-led solutions. Headquartered in Mumbai, India, the Company is differentiated by its unique three-pronged value proposition encompassing Business-to-IT Connect, Engage the Future and Execution Excellence. For more information, visit us at www.lntinfotech.com or email us at info@lntinfotech.com Follow L&T Infotech Blog: L&T Infotech Proprietary Effective Testing & Quality Assurance in Data Migration Projects Page 12 of 12