Migrating a Large Scale Legacy IT System to a System with Current Technology Bill Wood, Mike Gagliardi, Phil Bianco 4/18/2014 Migration Planning 1
Copyright 2014 Carnegie Mellon University and IEEE This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu. DM-0001020 4/18/2014 Migration Planning 2
Background Agency wanted to Migrate two paired and tightly coupled systems Both are 24/7/365 and have disaster recovery at multiple sites Both have many classes of users 1. Service Based System - 15% functionality (going to 40%) Java, Relational DB, COTS tools, Service-based, J2EE, SAP, Informatica 2. Legacy System - 85% of functionality (going to 60%) COBOL, Hierarchical DB, mainframe Migrate to a well defined target reference architecture (TRA) as a basis for a common platform infrastructure (CPI) developmental, operational, test Briefing is focused on the Legacy Migration 4/18/2014 Migration Planning 3
Its Complicated Understanding the Legacy System Architecture Infrastructure Tools Application Component Relationships (Data and Code) Business Process Threads Understanding the Target System Target Reference Architecture (TRA) SOA; Layered Infrastructure Designing Services on top of TRA Business Process threads Mapping between Legacy and Target in Phases Architecture mismatches (development, operational, certification, sustainment, COTS) Operating with dual authoritative data systems, cutover, synchronization Relationships between Application Components (Legacy versus TRA, COTS) 4/18/2014 Migration Planning 4
Is there too much code/ data coupling and spaghetti code to partition for migration? Are the current business processes and screens appropriate? Is the CPI (and TRA ) stable? Are COBOL to Java transformation tools up to the job? Can we operate with 2 systems overlapping authoritative data? Do we have sufficient technology expertize in: legacy system, target reference architecture, discovery and analysis tools, transformation tools? Is the business logic only understandable in the legacy code? How can we overcome the lack of architectural documentation? Will the entire testing and certification process change? Will I be creating a maintenance nightmare? It s a 24/7/365 operation- no downtime! It s Worrisome 4/18/2014 Migration Planning 5
Keep the Right Vision The TRA is a good start, but an application architecture with data modeling is needed Synchronize with phased TRA and CPI infrastructure deployment Operating with multiple sources of authoritative data Using new technology (e.g. customized security to infrastructure security) Capability of transformative toolsets 4/18/2014 Migration Planning 6
Don t Live in a Dreamworld Big-Bang changes usually fail Conducting transactions across networks and keeping response times satisfied! Transforming spaghetti code automatically Separating presentation from business processing from data access! Moving from green screens to windows Making multiple types of changes simultaneously! Edict no changes to the legacy system! Paul Delvaux 4/18/2014 Migration Planning 7
Target Reference Architecture This is a good start But it is not enough Application Presentation Layer Business Layer Data Access Layer Data Layer Need a system and application software architecture End-state Each Release Data Infrastructure Metadata Data Warehousing Transactional Data Failover Load Balancing Integration Layer Linux Software Commodity Hardware Security 4/18/2014 Migration Planning 8
Migration Approaches An RFI was conducted and multiple approaches suggested Lift and Shift (L&S) Reduce Operations and Maintenance (O&M) Costs ASAP Fast retirement of legacy infrastructure Move to the CPI Don t come close to satisfying the TRA Make minimal changes- keep COBOL, Hierarchy etc. etc. Modernize Phased transformation to TRA Keep legacy architecture Change technology- Java, SOA Use Conversion tools? Re-engineer Re-Develop the system in Java on CPI Developed system and application architecture Satisfy TRA Options Considered 1. Do nothing (baseline) 2. L&S (baseline) 3. L&S and modernize 4. L&S and re-engineer 5. Re-engineer 6. Hybrid- L&S, modernize, re-engineer 4/18/2014 Migration Planning 9
Migration Process Determine and Score Options Explore implementation alternatives for options Build an End State Architecture Build a Roadmap List the Options Develop evaluation criteria Score the options Its not reasonable to explore Make every a selection possible option (combination Goals for of sequencing approaches), and then to explore every Constraints possible implementation Phasing List the Alternatives alternative Leads Approach to Conduct analysis to Migration Discovery paralysisand Analysis of Legacy Have Build to Data, a make high code, level some Understand user, Target recommendations business Architecture TRA and processes CPI ordering incrementally Map legacy Quality elements Review and attribute choose Migration to considerations TA alternative elements Toolsets approaches Code, Migration Develop data, tooling test, Issues configuration for Resolution Define Keep phases vs Throwaway Develop Design Factors Perform Groups an Architectural Evaluation (and fix) Legacy: functionality, code, data BP, users Tiers/layers Transient code in Legacy and TA Throwaway Align with Infrastructure Roadmap 4/18/2014 Migration Planning 10
Determine and Score Options Understand Migration Determine and Score Goals Options Explore implementation alternatives for options Build an End State Architecture Build a Roadmap Create and Evaluate Options Recommend an Option Reduce operation and maintenance (O&M) costs Platform Legacy Licensing, Relationships obsolescence, electric, space, people Vendor locklanguage Architecture Documentation Database Create technical uniformity across organizational boundaries Subject Matter User Interface Experts (UI) Options Take advantage of new technology System usage Transactions 6 mentioned previously measurements by element Performance, Business Process Interfaces, users Initial crude Scoping evaluation Architecture parameters against TRA Doc Detailed Static evaluation relationships Standards against using tools Cost, Code, risk, performance data, Quality Attributes configurations, factors (23) interfaces Choose an Option Important Operational Developmental Quality Attributes (QA) Best Overall Measure Operational - cost, risk, performance Recovery- failures, Sustainment disasters Determine Security ordering: Infrastructuresequential, concurrently Important Developmental and Sustainment QAs 30.00 25.00 20.00 15.00 10.00 5.00 0.00 Do Nothing Lift &Shift (L&S) Element L&S and Modernize Goal- Examples Re-Engineer L&S and Re-Engineer Business Process Threads Users, code, data, interfaces 4/18/2014 Migration Planning 11 L&S and Hybrid Performance Cost Risk
30.00 20.00 10.00 0.00 Evaluation Factors Cost Performance Risk W Factor 123456 3 How much will be the migration labor cost? How much recovery of investment cost after cancellation How early will cost savings happen- license and support during migration? How much will be the on-going legacy license and/or support costs post migration? How is data organized wrt TA : relational, normalized, distributed, partitioned How readily can new functionality be put into production during the migration process- e.g. adaptive maintenance, changing user workflows How well does the end-state system satisfy the TRA architectural structures (application layers, understanding, tiers, SOA) for Partitioning of software and data for L&S causes cumbersome migration Are there tool factors that make the schedule unpredictable: learning curve, tool underperforms requiring extended manual intervention What is the risk of technical obsolescence of the end state? Extent of external interfaces (OGA, There was 30.00 developmental a separate and sustainment ROM cost attributes team (extensibility, maintainability, reuse across We started Developed with the team s a ROM architectural for each option 25.00 applications, standards) knowledge Commercial) 40 or so We factors provided How well does them system with meet data the as a basis for estimation How solid is the architectural operational attributes: satisfy end-user foundation, supported by We discovered Their 20.00 the ROM response OMB sizing under factors maximum was -19 about workload, factors the same as our cost factor sizing scalability, manageability, load balancing, Merged them 15.00 reliability, into the availability) initial factors- 50 or so How is data organized wrt TRA : relational, We started to weight the factors 10.00 normalized, distributed, partitioned 115888 Too many factors How many had internal an equivalent interfaces changes distribution will be of weights needed 5.00 procurements We combined factors with like distributions- 23 factors How much will be the level of effort for testing to 0.00 make it production ready How well are the TRA security conditions satisfied: easily changed, no unauthorized access, no data corruption What will be the impact for data synchronization, parallel run and cutover? How modernized are the user screens wrt the TRA documentation, at the end state? How much will be the Impact on Users Interface functionality? How well can the two migrations be coordinated? Risk of creating a monopoly for future How complex will the program management effort be? How much business functionality will be negatively affected? Budget/ Cost/Schedule profile goes out of alignment 4/18/2014 Migration Planning 12
Selected Hybrid Option- Example Legacy Phase 1 Phase 2 Phase 3 Legacy L&S Re-engineered Modernize Two Systems with Shared Authoritative Data Modernize Lift and Shift Re-engineer How to partition: L&S and modernize; and Re-engineer Significant Business Process Changes re-engineer Layered stable legacy code- L&S and modernize Experiment with modernization tools 4/18/2014 Migration Planning 13
Explore Implementation Alternatives- Alternatives Determine and # Description Score Options 1Finds Move the relationships the Data then between move the Code The subsystems and their programs Assumptions alternatives for options 2 Move The the Data Code Stores then and move their the files Data The hybrid 3 Analysis Move by showed groupings the business of Code processes, and Data and screens, Build Discovery an End State and The subsystems and the data stores Users and software and End-to-End data elements Business to be Lifted Threads and Shifted. Architecture Analysis End-to-end business threads/user classes/subsystems/data stores It is preferred that Identify the L&S Dead be done code in and stages dead data Build a Roadmap 4 Clean up legacy in situ first (or not) Legacy API s usage # Legacy transactions Issues usage for Resolution Issues for # 1 Functionality/Data Batch usage may be too entangled to group effectively Factors Resolution What commercial Such that authorized toolsets data are split best can to take do this? place 1 2 What Cutover toolset is best for converting data structure, 2 copying Rollback data, modifying APIs & JCL, writing adaptors? Design 3 Replacing Synchronize transactional support Factors 4 Should Data Copy we do some cleanup before L&S Explore implementation Lift and Shift 5 Performance 4/18/2014 Migration Planning 14
Build an End State Architecture Determine and Score Options Develop a Basis Explore alternatives for options Build an End State Architecture Build a Target Build a Roadmap Architecture Evaluate Target Architecture TRA and CPI constraints on End State Architecture Legacy System Architecture Findings Identify Relationships the Target Application between Legacy Elements and Target COTS Services, products data models References Identify totechnical Legacy elements challenges and risks Interfaces to external actors TRA elements used Build an Application Target Architecture Connections between Application elements Business process Conduct flows an through Architecture the elements Evaluation User screen contents Identify (appropriately risks and risk mirror themes legacy) Map to TRA elements used Update the architecture to mitigate the risks Keep a risk register 4/18/2014 Migration Planning 15
Build an Architectural Roadmap Develop a Basis Determine and Score Options Explore alternatives for options Define a Pilot Build an End State Architecture Build a Roadmap Define Next Phase Evaluate Consistency Phase selection can be done in a number of ways Select Goals a Business Pilot Migration processes, for users, First services, Phase authorized data Lightweight, Define Low hanging the tools low fruit, risk, being budgeting, few used challenges for scheduling, migration early success Data, code, screens Gain Reduce target legacy and transformation sustainment costs tool experience Define the TA contents of the phase Externally Include data induced extraction, schedules analysis, and reporting Business processes, screens Perhaps aligning Tables, no phased data cutover capabilities with external milestones sources, services and end-points, screens Organizational Happy User path impacts functionality boundaries Architectural Infrastructure Workarounds But views include for some and every Technical quality major Debt release, attribute that drivers correlate to the End State Aligned Define data Architecture model, with the TRA applications, Issues and infrastructure the not Legacy UI Addressed development Architecture. schedule Shows Identify certification Infrastructure, COTS tools, platforms, networks Tasks involving how target the legacy architectural testing- per release? Constraints transformation elements elements of are legacy replaced needed artifacts at each to TA stage Shows Define the on Legacy Phasing system (Releases) artifacts being migrated Tools used what Business in end-state Process the transformation elements Threads, are Screens introduced each stage Define User Migration Subsystems, impact, security, Scenarios tables, site datarollout, authoritative data Describes Services, the correlation data models, between COTS the tools legacy and the end-state Developmental, Define Platform the and quality networks operational, attribute and COTS sustainment considerations products Describes Identify how legacy the architectural users interact items being migrated Evaluate Target Screens, Cross System the roadmap coupling Infrastructure Reports against between the systems scenarios in-place, Data transfer, build epics cutover, during synchronization, migration Capture Subsystems risks, gaps, (partial), sensitivities, tables (partial), tradeoffsdata sources Approach Recovery to migration from failure- routine and disastrous Update the roadmap Confirm Data first, that code all first, events code/data synchronize grouping, BP first Identify Establish technical Risk repository challenges and risks Tools to assist migration 4/18/2014 Migration Planning 16
Summary Technical Criteria for selecting between options OMB guidelines were good Plan, but don t overdo it Architect Understand Legacy Include quality attributes High Level End State Roadmap Detailed architecture at each phase Management Changing political environment Drift and Re-direction can become a way of life After The Party 4/18/2014 Migration Planning 17