Realizing the Benefits of Data Modernization

Transcription

1 February 2015 Perspective Realizing the Benefits of How to overcome legacy data challenges with innovative technologies and a seamless data modernization roadmap. Companies born into the digital world are on an accelerated path to mobility, cloud computing, and big data analytics. These companies are realizing the benefits of delivering a rich customer experience through innovation and agility. But the path to the digital world is not an easy one for the vast majority of enterprises that face the herculean task of legacy modernization. Core legacy applications must be replaced with new cutting-edge technologies, and data must be unlocked from legacy databases and moved to modern databases where it can be leveraged efficiently and effectively. Despite the challenges they face with legacy modernization, enterprises in industries like banking and financial services, life sciences, retail, and transport and logistics recognize the necessity of legacy modernization and data modernization in particular as a means of remaining viable in today s competitive markets. Thus, data modernization will be a key focus area for these enterprises going forward. This Perspective is intended to help those enterprises transition from the legacy world to the digital world by addressing the challenges associated with data modernization. Understanding For years, enterprises have faced the rising costs associated with running and maintaining mainframe databases. Many enterprises, particularly those in banking and financial services, are finding that legacy databases like VSAM, ISAM, IDMS and IMS have outlived their usefulness. These databases simply cannot meet the demands of mobile, cloud computing, big data, gamification, and social media. In order to take advantage of these technologies, enterprises must undergo data modernization projects. Put simply, data modernization is the movement of data from legacy mainframe databases to modern databases. An enterprise s ability to compete in the digital age rests on business agility. Cloud, mobile, and big data analytics not only help enable that agility, but also require agility. Big data analytics, for example, enables companies to make critical business decisions in real time. Enterprises need faster access to data, seamless integration services, and highly scalable, flexible, and low-maintenance databases. Legacy databases just can t keep up. It is difficult and risky to make changes to monolithic legacy databases. Making a simple modification to support a business goal requires a lot of time and effort, which ultimately impacts agility and delays time to market. Likewise, the cost of maintaining legacy databases is also driving data modernization efforts. Enterprises may run hundreds of applications and databases. The legacy database software licenses and maintenance consume an increasing amount of time and money, with the resulting costs running into the millions of dollars. And it will only become more costly as the databases themselves continue to age. In the meantime, it is becoming increasingly difficult to find IT professionals with the necessary skill sets to run and maintain legacy databases. The programmers and DBAs who are savvy with mainframe databases will soon be retiring, THINK SMART. ACT FAST. FLEX YOUR BUSINESS.

2 and younger professionals do not have the experience or expertise. As a result, enterprises are carrying technical debt that makes it difficult to run the business. Eventually, database vendors stop supporting older versions of their databases, and enterprises running those systems have no choice but to upgrade to the latest releases if they wish to have continuous technical support. Such is the case for enterprises running SQL Server 2008, which will stop supporting later this year. Those enterprises face a data modernization project that entails moving their SQL Server databases to a more current version for which support is available. The main objective is to get the most value out of existing investments by upgrading aged and out-of-support systems to a new RDBMS. Another technical driver for data modernization and the core of data architecture is the data model. Data models established several years ago include many constraints in terms of business logic. These data models do not easily support web, mobile, or big data. Similarly, critical business logic and data are locked in legacy and proprietary database platforms, making them difficult to access and use in new ways, such as application deployment in mobile platforms. Finally, changes in regulatory compliance requirements are driving the need for data modernization. Federal and state requirements for data retention vary from five to 30 years, depending on the nature of the data and the particular regulation. Quick and complete access to records going back to as many as 10 to15 years is often required. Inaccessible data in formats that do not allow robust querying can lead to loss of revenue and fines for non-compliance. Challenges While there is plenty driving the need for data modernization, there are also challenges that may cause enterprises to delay a data modernization project. These challenges can be categorized as either in-flight (during the data modernization process) or post-data modernization. In-flight data modernization challenges begin with a lack of documentation for the business rules and incremental changes made in the legacy code. Without documentation or automation tools to extract the business rules from legacy code, there is the risk that some of the business rules will be missing. Most enterprises also lack the tools, strategies, and methodologies necessary for a successful data modernization project. In fact, most data modernization projects fail due to a lack of standard methodologies, frameworks, templates, and guidelines. Enterprises need an appropriate questionnaire and templates to arrive at the best decisions for data modernization. They must select the appropriate data modernization tools and replacement database(s), both of which can be challenging. And, of course, enterprises must have the skills for data conversion and data migration. As a result, organizations face a steep learning curve for a mission-critical project. Post-data modernization challenges pertain to the business implications of moving from one system to another. Most organizations have been running legacy databases for two to three decades and cannot afford to modernize all of their data at once. These organizations must undergo a phased approach to data modernization. However, this too introduces technical challenges. Data redundancy, data duplication, data merge, and data integration issues must all be addressed. Other challenges include overlapping functionality and the need for temporary systems to bridge the gap between legacy and the new systems for the intermediate phases. It is also important that care is taken to prevent business disruptions during the data modernization project. Proper strategy and planning are required to ensure that business continues as usual during every phase of the project. Staging and production releases, along with pilot releases, must be meticulously planned. The costs incurred with data modernization can also pose a challenge to some enterprises. For example, business users must be trained on the new systems and applications impacted by data modernization. Training should not be avoided as a way to reduce expenses. It is necessary to ensure the adoption of the applications and systems, and to ensure that users can properly use and manage the new database tools and applications. In addition, enterprises will incur costs associated with information changes related to the change management program. The Benefits of Enterprises that successfully address the challenges of a data modernization project are poised to take advantage of a number of benefits, the primary one being the ability to bring in more stable, scalable, and flexible databases. Moving to more robust RDMBS, NoSQL, and Hadoop-based platforms enables the agility. For example, changes can be quickly made to support business goals. The organization can also define a data strategy that aligns business and technological goals for social media, gamification, big data, and mobile efforts. 2

3 Enterprises also benefit from increased productivity and reduced maintenance costs. Most of the newer databases have GUI tools that help facilitate database administration, performance monitoring, etc., and with legacy database maintenance no longer an issue, enterprises can assign IT staff to more strategic projects. Newer databases are also easier to run, with ad hoc reporting and intuitive functionality. Finally, enterprises no longer have to rely on a dwindling workforce to run and maintain an aging technology. It will be easier to find and perhaps even less expensive to hire programmers and DBAs with the skills required to maintain modern databases. Methods There is no one-size-fits-all approach to data modernization. However, there are three general approaches enterprises take, depending on their business goals: Data migration Involves moving data to a different vendor Source and target schemas remain the same Involves migrating code (procedures, etc.) Usually no major changes to the application Automation tools can be used to complete migration Example: Moving out of Sybase to save on licensing costs Data conversion Source and target schemas are different Involves transformations during migration Typical during application re-engineering and legacy application modernization ETL tools are available, but process is manual Example: Moving from legacy databases like ISAM, VSAM, IMS or IDMS to RDBMS Database upgrade Involves upgrading to a newer version No transformation required Deprecated code is replaced Automation tools can be used to complete upgrade Example: Upgrading from SQL Server 2005 to SQL Server 2012 For a more technical understanding of the three data modernization options, it helps to consider specific scenarios: Scenario 1: Legacy database to RDBMS In this scenario, data is sourced from the mainframe system and converted/re-platformed into a modern RDBMS like, SQL Server, or IBM DB2. Source Platform Mainframe Scenario 2: RDBMS to RDBMS In this scenario, source data is available in an RDBMS. It is moved to a newer target RDBMS, which may belong to a different vendor. If the source and target schema/structures differ, then data cleansing and transformation are required. The re-engineered system is operational only after the data is transformed. Source Format VSAM ADABAS IMS ISAM IDMS Target PostgreSQL Server Target Platform RDBMS / SQL Server/ DB2 Categories Data Conversion/ Re-hosting Categories 3

4 Services from NTT DATA While every data modernization project is different, the potential challenges are minimized with NTT DATA s proven scenario-driven data modernization framework. The framework consists of three steps: Assessment Stakeholders are consulted and a detailed data modernization strategy and methodology that aligns with the enterprise s data modernization and business goals are defined Roadmap A roadmap is determined for executing data modernization, beginning with the least risky data and moving towards more critical data Implementation Multiple steps are taken to implement data modernization Data analysis Entity mapping Map source entity attributes to target entity attributes Identify data loss Determine if unmapped attributes would result in data loss Standardization rules Classify data for standards Maintainability analysis Analyze ease of maintainability based on analysis report Migration strategy and design Detail downtime required due to migration, required staging storage/ hardware, migration impact on target database size, tracking updates to source during migration Error-handling during migration Proof of concepts As defined in migration strategy document Pre-migration Data inconsistency check Highlights the possible inconsistency of an entity against pre-defined standards Data redundancy check Highlights the amount of redundant data present in legacy databases Data stability check, data type check, data length check and valid value check Data cleansing Based on the findings of the inconsistency check and stability check, data is cleansed in the source system or during transformation of the migration phase Maintainability analysis Determine whether maintainability is affected as a result of data cleansing and consolidation ETL-based migration Data transformation Transform data-based transformation rules Migrate data Move data from source to target Report and resolve errors Log and report errors during migration; Log and report status of migration Resolve errors Continuous migration Change capture Capture the changes in the source system Propagate changes Propagate the changes to the target system Apply changes Apply the changes onto the target system. Transformations may be required. Post migration Data assurance Translate target system to source format and compare against source This proven method is supported by our proprietary acceleration process which fast-tracks data conversion by generating template-based code out of the defined source and target data mapping in the system. The accelerator expedites data modernization and helps migrate data faster to target RDBMS and new NoSQL databases by reducing a significant amount of the development, data conversion rework, and unit testing associated with data modernization. As a result, our customers benefit from a 30% to 40% reduction in development costs. Conclusion To leverage innovative technologies, enterprises need faster access to data, seamless data integration services, and highly scalable, flexible, and low-maintenance databases. Unfortunately, many enterprises continue to run legacy databases, which fail to meet these requirements. These organizations need to act fast and modernize their data and applications sooner rather than later. The key to a successful data modernization project is to partner with a company like NTT DATA that has experience working with a variety of companies and has a proven data modernization strategy, methodology, framework and roadmap. 4

5 Author Biography Prakash Mishra Senior Director Enterprise Data Architecture and Management practice. Prakash Mishra is a senior director at NTT DATA, Inc. He holds a master s degree in computer science and has 19 years of experience in enterprise data architecture and management. Prakash has vast experience in various domains, including financial services and insurance, life sciences, manufacturing, retail, information content and technology, and security technologies and networking. About NTT DATA NTT DATA is a leading IT services provider and global innovation partner with 75,000 professionals based in more than 40 countries. NTT DATA emphasizes long-term commitment and combines global reach and local intimacy to provide premier professional services, including consulting, application services, business process and IT outsourcing, and cloud-based solutions. We re part of NTT Group, one of the world s largest technology services companies, generating more than $112 billion in annual revenues and partner to 80% of the Fortune Global 100. Visit to learn how our consultants, projects, managed services, and outsourcing engagements deliver value for a wide range of businesses and government agencies. THINK SMART. ACT FAST. FLEX YOUR BUSINESS _02-P-Data_modernization