Methodology for sustainable MDM and CDI success. Kalyan Viswanathan Practice Director, MDM Practice - Tata Consultancy Services

Transcription

1 Methodology for sustainable MDM and CDI success Kalyan Viswanathan Practice Director, MDM Practice - Tata Consultancy Services

2 Agenda Some Definitions - SOA and MDM Transitioning from Legacy to SOA Some Definitions - Data Governance Methodology for sustainable success with MDM and CDI The TCS MDM / CDI Framework

3 Some Definitions What is a Services oriented Architecture? What is Master Data? What is? What is Data Governance? What is Data Stewardship? What is the relationship between Master Data Management and Data Governance?

4 What is a Service Oriented Architecture (SOA)? A Service Oriented Architecture involves : 1. The Building blocks of - Services and Business Processes Monolithic Applications decompose into Services and Business Processes 2. A Robust Interoperability Architecture 3. Maximization of Re-use of services that are loosely coupled 4. Elegant Enablement of Composite Business Process across heterogeneous application and Service domains 5. Standard agreements and protocols 6. The exploitation of the Convergence of XML, Web Services, and EAI resulting in the potential for unprecedented levels of integration (Meta Group)

5 What are the promised benefits of SOA? 1. Agility of IT to respond to Business change 2. Re-use of legacy code without a total rip and replace 3. Flexibility of IT systems easy adoption of new sales channels and B2B interfaces 4. Enhanced Productivity of the IT organization 5. Information integration and standardization (eg. Single customer view) 6. Potential to build new kinds of applications and business processes faster and cheaper

6 What are the Challenges of implementing SOA? 1. Introduces new ways of thinking about IT systems almost a cultural paradigm shift 2. Unclear as to what the first steps and the sequence of activities towards an SOA architecture are 3. Re-use has to be engineered carefully It does not occur by accident 4. Difficult to create business cases for stand alone SOA projects Ownership and Funding 5. SOA standards still developing 6. Service Levels, Performance and Scalability

7 What is Master Data? Master Data includes the core business entities which constitutes the enterprise s data assets such as Customers, Products, Accounts, Suppliers, Parts, Employees, Locations, Factories, Reference Data, Hierarchies, and so on that meet the following three criteria: Definitions Criteria for Master Data 1. Master Data is relatively slowly changing data that is updated infrequently (each subject area may differ in frequency) 2. Master Data is referenced by many different transactional scenarios (reference data is a subset / category of master data) 3. Master Data generally tends to be widely replicated in many applications and used by many business processes (some master data may be totally private, and is still master data)

8 What is Master Data as distinct from other data? Eg. Customer Master Data Master Data Describes relatively static dimensions of a Customer Includes Customer Identity, Profile, Demographics, Relationships with other customers (both individual and corporate), Credit History, Account Relationships, Privacy, Channel Preferences etc. Data Transactional Data Describes a Customer s Operational State with respect to his/her relationships Eg. Balances, Principal, Premiums, Interest accrued, Claims, Invoices, withdrawal, payment, Deposit, Transfers, Sales status, Service tickets Represents business activity at a point in time Analytical Data Describes a Customer s performance with a historical or futuristic view Eg. Trends, Forecasts, Sales history, Buying patterns Also includes derived information such as profitability, segmentation, propensity to buy, Lifetime Value, Risk exposure

9 What is Master Data Contd.? Characteristics Master Data Semantics i.e. Entity and Attribute definitions have to be reconciled, between different departments and business functions Master Data ownership is often distributed and sometimes the same attribute has multiple owners, while others have no owners Master Data has a definitive scope / list for any organization Master Data occurs as dimensions in analytical environments From 15% to 40% of any Legacy Application deals with Master Data

10 What is (MDM)? is the combination of 1. Governance 2. Processes and 3. Technologies needed to define, create, store, maintain, secure, and make available a Consolidated, Consistent, Contextual, Complete and Accurate view of master data across multiple Business Processes, external business partners, and application systems.

11 Data Access Services and Business Services Operational and Analytical Applications : Transactional Systems Portals, Reporting Apps, Workflow, Imaging et al. Business Services Layer Meta Data Data Access Services Layer DATA FARM Legacy Data Sources : DB & file Master Data Hubs ODS & Warehouses Data Marts & ADS Data Access Services provide : Virtualization of Data - Enterprise-wide data abstraction layer Integrated views of data from multiple sources Relational databases, Applications, Files, Data Warehouses, Data Marts, ODS & MDM environments Ubiquitous Access - Re-useable Data Services for data consistency Data Access Services are consumed by other Services and Applications Data Access Services are closely related to Enterprise Data Management, which organizes and manages data over its life cycle

12 The Information Value Chain Looking at the Data Farm Products Customers Dealers Contracts Channels Transactions Master Data System Transactional Systems Operational Data Stores (ODS) Data Warehouse (DWH) Data Marts (DM) Data Class Master Master Transaction Master Transaction Master Transaction Analytic Master Transaction Analytic History Yes No None Limited Yes Yes Integration Yes No Yes Limited Yes Yes Data Currency Data Scope Data Creation Real Time Fully Integrated Application Neutral Real Time Local Application Specific Close to Real time (+1 Day) Integrated (Limited to a few applications) Sometimes Daily Weekly Monthly Fully Integrated Application Neutral Yes Yes No Limited to Derived Data Sometimes Daily Weekly Monthly Analytical, Derived and Summarized, Application specific Limited to Derived Data Master Data moves left to right through the Information Value chain over its life cycle Global Consulting Practice (GCP)

13 Data Access Services versus Business Services Attributes Business Services Data Access Services Core concern Business Logic composition Data Access virtualization Encapsulates Re-usable Business Logic, Business Rules Data Definitions & Sources, Data Integrity constraints Characteristics Large Grained, Functionally oriented Fine Grained, Data Oriented Invocation Patterns From the Web, through EAI layer, direct API, from Applications Other DAS and Business Services, from Applications, from the Web, through EAI Layer Design Method Top down, Process Driven Bottom up, Data Driven Modeling Method Business Process Models Canonical Data Models User Interfaces Relationship Degree of Re-use Will frequently invoke Business Services for executing Business Logic Business Services will invoke Data Access Services Re-used within similar Business Processes eg. Open Account May occasionally directly invoke a Data Access Service for direct data access Data Access Services will rarely invoke business services but may invoke other Data Access Services Re-used across widely varying Business Processes Maximal Re-use

14 Architecting the Service Mosaic Large grained Business Service Increasing Value of Reuse Increasing Degree of Reuse Fine grained Data Service How should a service be defined in order to maximize its Re-use potential and its compos-ability characteristics?

16 The Key Challenge on the SOA Journey Legacy SoA Monolithic, Large sized Legacy Applications somehow have to unbundle or dis-aggregate into an orderly set of Business and Data Services, in the new Architecture. How do we accomplish this while also ensuring Business Continuity and Service levels?

17 Service Identification Ensuring Re-usability and Compos-ability 1. How should a service be identified and defined in order to maximize its Re-use potential and its compos-ability characteristics? 2. How do combine fragments of shared business logic currently scattered in many Legacy Applications to form a useful Service? 3. How do we ensure that Business Rules that are scattered in different Applications are consolidated, rationalized and encapsulated into Reusable services? 4. Should we start with an Enterprise wide Services Architecture first? 5. How do we consolidate Data that is widely replicated in many Applications, so as to encapsulate them into Re-usable Data Services? 6. How do we ensure that Services are tested for future use, in service composition scenarios not yet identified?

18 Master Data Services A natural place to start Legacy SoA Customer Product Location Account The Re-organization of Master Data in an enterprise that consolidates widely replicated instances of Master Data, reconciles and creates a single version of the Master Data Record, along with a sufficient set of Master Data Access Services, is an important milestone in the SOA Road Map. It represents a critical unbundling step of the Legacy Applications Global Consulting Practice (GCP)

19 Legacy Transition A Method Legacy Applications undergo the following sequence of transitions in the wake of their Master Data being Re-architected : 1. The Master Data from Legacy Applications are extracted and consolidated into a separate Master Data environment. 2. Interfaces are constructed to keep the Legacy Master Data in synch. with the new Master Data environment 3. New Master Data Access Services are created that deliver Master Data to Requesting Applications 4. Gradually, Master Data Access in the Legacy Applications are transitioned to the new Master Data Environment 5. Eventually Master Data Creation and Maintenance is also transitioned into the new Master Data Environment 6. Finally, the physical instance of Master Data within the Legacy Applications are altogether removed, representing a near surgical Cut of the Legacy environment (15 to 40%)

21 What is Data Governance? Data Governance creates a culture where creating and maintaining high quality data is a core discipline of the organization. Data Governance is the formalized discipline of ensuring accountability for the management of an Enterprise s Core Information Assets. Data Governance includes 1. A defined Process 2. An Organization Structure 3. Well defined Roles and Responsibilities 4. Clear Rules of Engagement and Escalation

22 What is the Relationship between and Data Governance? The scope of Data Governance goes well beyond Master Data. It includes Structured Data 1. Master Data 2. Transactional Data 3. Analytical Data 4. Meta Data Unstructured Data 1. Documents 2. Content 3. Knowledge

23 Data Warehouse and BI versus MDM oriented Data Governance Products Customers Dealers Contracts Channels Transactions Data Class Master Data System Master Transactio nal Systems Master Transaction History Yes No Operational Data Stores (ODS) Master Transaction None Limited Data Warehous e (DWH) Master Transaction Analytic Yes Data Marts (DM) Master Transaction Analytic Yes Data Warehouse and Business Intelligence focused Data Governance Initiatives Integration Yes No Yes Limited Yes Yes Data Currency Real Time Real Time Close to Real time (+1 Day) Sometimes Daily Weekly Monthly Sometimes Daily Weekly Monthly Data Scope Fully Integrated Application Neutral Local Application Specific Integrated (Limited to a few applications) Fully Integrated Application Neutral Analytical, Derived and Summarized, Application specific focused Data Governance Initiatives Data Creation Yes Yes No Limited to Derived Data Limited to Derived Data Data Warehouse and Business Intelligence focused Data Governance Initiatives lack a Data Life-cycle view, and get exclusively focused on preparing Data for Data warehouses, Data Marts and Reporting environments. focused Data Governance Initiatives, are concerned with the entire life cycle of the Master Data, starting from how they originate and are used in the various Operational Systems as well as the analytical environments, however they tend to ignore the transactional data.

25 The Case for a Strategic MDM Vision

26 represents an Architectural paradigm shift Rationale Master Data is highly shared across many business processes and applications At the heart of MDM is a radical idea that the governance of Master Data must be de-coupled from Transactional, Operational and Analytical systems The way Master data is delivered to various applications, must be through well defined and highly re-used Master Data Access Services Implication The introduction of an MDM Solution is disruptive to the existing Architecture The way existing applications own, manage and share Master data will fundamentally change in the wake of MDM initiatives The Architectural style of Off the shelf Applications (i.e. Products) must also adapt to his emerging paradigm shift

27 De-coupling Master Data from Transactional Systems is the end state Rationale The concept of goes beyond merely consolidating master data and providing appropriate data governance It even goes beyond synchronizing master data across many different applications, from a central Master Data Hub In a Services oriented Architecture, it is possible to deliver Master Data to various consuming applications, through Data Access services Implication Application design must undergo a radical transformation. Instead of making the Master Data as part of the Application design, it must allow for the possibility that Master data may be accessed through services Off the shelf products must also begin to allow for externally provided Master Data Services (as an alternative to their internal Master Data tables) Managing referential integrity will be more complex in this environment

28 is a key milestone in SOA adoption Rationale At the heart of Service Oriented Architecture is the concept of separation of data from process. As SOA adoption scales beyond the initial Pilot exercises, it will encounter a critical cross road i.e. the fact that Master Data is scattered in many different applications Top down process oriented approaches to SOA adoption do not adequately address the problem of highly replicated, and scattered data Implication Bottom up Data oriented approaches to SOA adoption will be recognized as key milestones in the SOA Road map MDM initiatives and SOA initiatives must be closely coordinated and converged Master Data services will be building blocks for creating more coarse grained Business oriented services

29 initiatives are enterprise programs Rationale initiatives are not yet another project. They deliver critical Enterprise infrastructure They impact many different applications and business processes including most concurrent projects within the organization It these implications are not adequately accounted for, the real potential value of Master data initiatives will not be realized Implication An enterprise level vision and road map must be established for Master Data Management initiatives The governance of Master Data Initiatives must be co-ordinated at the enterprise level The Executive level governance structures must provide the right steering signals to align concurrent initiatives with the MDM initiatives

30 must address all Master Data subject areas Rationale Some subject areas are more obvious and critical to the business than others. They may not get recognized in the initial stages of an MDM program. An enterprise level vision and road map for must encompass all Master Data subject areas These subject areas are typically the key Master Data concepts within an enterprise such as Customer, Product, Location, Contract, Geography, Employee, Asset and so on. Implication An enterprise level MDM program, will have many tracks, with each track pertaining to a subject area There may be different business sponsors for these tracks The MDM Solution pattern for each subject area, may also vary by subject area

31 Customer Data Integration (CDI) is an important track within MDM Rationale The emerging CDI space is a critical subject area within the Master Data Management space CDI has gained momentum, simply due to the fact that the Customer Data is critical to most businesses Customer Master Data is also scattered widely amongst many operational and analytical systems Implication Enterprise wide MDM initiatives can benefit from the early success of CDI initiatives When CDI initiatives do not have an Enterprise wide MDM context, the potential value of MDM may not be realized When CDI initiatives are created as stand alone initiatives, they will not yield re-usable patterns across other subject areas

32 Enterprise Master Data Models are critical for Data consolidation Rationale Data Modeling allows an enterprise to establish unique definitions and address issues of semantic dissonance, across its various organizational components Data concepts such as Customer and Party acquire different definitions over time, especially in different contexts. MDM initiatives are occasions to address the fundamental definitions that a business is based upon a well defined business concept and context Implication The notion that we are buying a Product so why focus on Data modeling which is common in the contexts of ERP applications, must be counteracted While an MDM product vendor can provide a Starter Data model, these must be carefully assessed and customized MDM by its very nature is Application neutral. So Application specific data models, must be avoided in MDM initiatives

33 Data ownership and governance are critical for Data Synchronization Rationale Data ownership implies that there is a single business owner, for each business attribute, in each Master Data subject area Historically data ownership has been confined to Application boundaries. The resolution of data semantics across applications has been relegated to the domain of Integration Enterprise wide Data ownership and Governance is not well understood or developed, today. But it is a critical milestone in MDM Implication It is obvious that the SVP of Sales and Marketing is a good candidate to own and govern Customer Master Data. However, he has to own and govern Customer Master Data on behalf of the entire enterprise, not just Sales and Marketing. This is a different paradigm. New Enterprise wide roles and responsibilities are required in the wake of initiatives

34 The Case for tactical MDM Delivery

35 MDM initiatives impact many business areas Rationale results in improved data quality, semantic consistency and can impact many business areas. For example, a typical CDI initiative can impact a variety of areas related to the Customer subject area such as Sales, Marketing, Servicing, Business Intelligence, Risk Management, Privacy, Compliance and so on The real business value of MDM initiatives is the aggregate of the business benefits in each impacted area Implication Building a comprehensive business case for MDM requires understanding its potential business value across these varied business areas No one Business Executive is going to have MDM on the top of his or her agenda Selling the concept of MDM to the business requires constant evangelization, and advocacy across a broad range of business executives

36 MDM Initiatives must deliver business value for sustainability Rationale The creation of a Centralized Master data hub cannot be an end in itself. It has to be leveraged appropriately to deliver measurable business value The business value of an MDM initiative must be forecasted before the initiative begins, and assessed after it ends When business value is clearly demonstrated, subsequent iterations of the MDM Road map can be funded more easily Implication The business value must be articulated clearly through a well developed business case The business case must be built on the basis of critical business measures that will be impacted by the MDM initiative Business executives must be educated on the fact that MDM initiatives deliver critical infrastructure, which must be accompanied by business strategies and solutions

37 The business benefits of MDM are intuitive yet difficult to quantify Rationale MDM initiatives only deliver the foundational infrastructure for potential business value. For example, the fact that we now have a single customer view in the customer data hub, does not guarantee that the Cross Sell ratio will go up. The single view of the customer has to be leveraged through effective cross sell business strategies Implication As an enabling infrastructure, it is difficult to directly attribute business value to an MDM initiative. Selling the concept and acquiring funding for an enterprise wide MDM initiative is going to be challenging It will always be easier to sell an enterprise on the concept of a tactical MDM initiative, (as part of a larger business project) with limited scope and budget.

38 MDM Solution delivery must be aligned with tactical business projects Rationale Often there are in-flight, funded projects that are currently being addressed through custom development, or off the shelf products These efforts may not have considered an MDM based approach at all If an MDM Architectural vision is not aligned tactically with ongoing funded initiatives, it may never get off the ground By aligning an MDM Road map with tactical Business initiatives, the enterprise can begin the MDM journey Implication Some central agency must own the MDM architectural vision and arbitrate it across numerous in flight and upcoming initiatives Tactical business projects must be assessed for convergence with the MDM architectural vision There must be a well articulated MDM Architectural vision, that has been established at the enterprise level

39 MDM Initiatives must have many iterations within each subject area Rationale MDM initiatives impact multiple business areas. The delivery of business value to a specific business area or function could be constituted as a specific MDM iteration For example, delivery of Customer Master data to the Sales function, and the attendant benefits could be a single iteration in the MDM Road Map The delivery of business value to a specific business area is easier to manage in a shorter time frame. Shorter time to value builds confidence in the initiative Implication The MDM vision for an enterprise must encompass the notion of an MDM adoption iteration The business case and the solution footprint of each iteration must be carefully constructed in order to build growing consensus for the initiative There are no tactical MDM projects only MDM adoption iterations that build upon a Strategic Architectural vision

40 The Case for creative balance between Strategic Vision and Tactical Alignment

41 Lack of Strategic Focus will yield disconnected MDM infrastructures Rationale Without a strategic MDM Vision, it is possible for most large enterprises to end up with multiple MDM infrastructures, which are not appropriately integrated The design of MDM solutions will not be established to evolve through multiple MDM delivery iterations Tactical MDM design patterns that do not scale up to a coherent MDM architecture are predictable Implication A strategic MDM Vision that has the buy in with the key enterprise stakeholders, on both the Business and IT organizations is critical to success along the MDM journey A mechanism to co-ordinate the project portfolio with the MDM initiatives is also critical to MDM success

42 Lack of tactical alignment will result in challenges in getting started Rationale Without tactical alignment, the MDM journey will remain a vision without a sponsor A big bang approach without tactical short term value delivery will be difficult to get funded Implication The MDM Vision must be reconciled with the current funded project work A beach head project must be identified that can become the pilot to get the MDM program started The business value of tactical MDM iterations must be clearly demonstrated

43 MDM initiatives must balance strategic focus with tactical alignment Rationale Without an appropriate balance between strategic vision and tactical alignment, MDM initiatives can suffer for want of adequate sponsorship and visibility Every MDM initiative during its life cycle will lean either towards the strategic vision or the tactical alignment Maintaining this balance is a key challenge with steering MDM initiatives Implication Enterprises must have the appropriate ownership and governance structures to achieve this balance with MDM initiatives Executive steering signals must drive organizational behavior Senior executives must be sensitized to the nature of this balance

44 Enterprise Architecture must provide the Strategic guidance for MDM Rationale Enterprise Architecture has the broadest view of the IT portfolio in the organization Without such a broad view of the IT portfolio, MDM initiatives can become narrowly focused Enterprise Architecture must provide the leadership and guidance, to establish an MDM road map Implication There must be a strong Enterprise Architecture team EA must educate and evangelize the concept of MDM and the nature of the MDM journey with the senior IT and Business executives Enterprise Architecture must guide individual projects, product selection exercises, and technology choices to align them with the MDM vision

45 Enterprise Program Management must align Projects with MDM Rationale Without a program management function governing and overseeing the current in flight projects, there is a risk of individual projects making decisions that are not in alignment with the over all MDM vision Individual project design and architecture decisions made without the overall MDM vision in mind can be counter productive and cause inefficiencies It is not uncommon to have multiple MDM like initiatives sprout up, each with a narrow focus and little convergence Implication The enterprise program management function, must work closely with Enterprise Architecture to provide Architectural convergence across the portfolio of projects currently in flight Clear mechanisms for Architectural reviews, compliance assurance and exception management must be established

47 TCS Approach to MDM

48 Enabling the MDM / CDI Adoption Cycle Envision MDM Deliver MDM Geo Spatial Data Management Location Data Managem ent Account Data Managem ent Manage Your Master Data Architect MDM Customer Data Integration Reference Data Manageme nt Product Information Managemen t Strategize MDM is a journey, not a project. MDM Solutions redefine an Enterprise s approach to data management, across both their operational and analytical environments Envision Building an MDM consensus Enabling the complete MDM Adoption cycle Envision Business Value Assessment Envision To-Be Business Concept model Envision Program Road Map Strategize Master/Meta Data Architecture Strategize Data Quality Strategize Data Governance Framework Strategize Technology Selection Architect Detailed Analysis & Planning Architect Solution Architecture &Design Deliver Build & Deployment Deliver Support & Evolution

49 Thank You & Q & A