Master Data Management

Transcription

1 Master Data Management David Loshin AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO Ик^И V^ SAN FRANCISCO SINGAPORE SYDNEY TOKYO W*m k^ MORGAN KAUFMANN PUBLISHERS IS AN IMPRINT OF ELSEVIER MORGAN KAUFMANN PUBLISHERS

2 Contents Preface Acknowledgments About the Author xvii xxiii xxv CHAPTER 1 Master Data and Master Data Management l 1.1 Driving the Need for Master Data Origins of Master Data Example: Customer Data What Is Master Data? What Is Master Data Management? Benefits of Master Data Management Alphabet Soup: What about CRM/SCM/ERP/BI (and Others)? Organizational Challenges and Master Data Management MDM and Data Quality Technology and Master Data Management Overview of the Book Summary 20 CHAPTER 2 Coordination: Stakeholders, Requirements, and Planning Introduction Communicating Business Value Improving Data Quality Reducing the Need for Cross-System Reconciliation Reducing Operational Complexity Simplifying Design and Implementation Easing Integration Stakeholders Senior Management Business Clients Application Owners Information Architects Data Governance and Data Quality Metadata Analysts System Developers Operations Staff Developing a Project Charter Participant Coordination and Knowing Where to Begin 32 vii

3 viii Contents Processes and Procedures for Collaboration RACI Matrix Modeling the Business Consensus Driven through Metadata Data Governance Establishing Feasibility through Data Requirements Identifying the Business Context Conduct Stakeholder Interviews Synthesize Requirements Establishing Feasibility and Next Steps Summary 41 CHAPTER3 MDM Components and the Maturity Model Introduction MDM Basics Architecture Master Data Model MDM System Architecture MDM Service Layer Architecture Manifesting Information Oversight with Governance Standardized Definitions Consolidated Metadata Management Data Quality Data Stewardship Operations Management Identity Management Hierarchy Management and Data Lineage Migration Management Administration/Configuration Identification and Consolidation Identity Search and Resolution Record Linkage Merging and Consolidation Integration Application Integration with Master Data MDM Component Service Layer Business Process Management Business Process Integration Business Rules MDM Business Component Layer MDM Maturity Model Initial 56

4 Contents ix Reactive Managed Proactive Strategic Performance Developing an Implementation Road Map Summary 65 CHAPTER4 Data Governance for Master Data Management Introduction What Is Data Governance? Setting the Stage: Aligning Information Objectives with the Business Strategy Clarifying the Information Architecture Mapping Information Functions to Business Objectives Instituting a Process Framework for Information Policy Data Quality and Data Governance Areas of Risk Business and Financial Reporting Entity Knowledge Protection Limitation of Use Risks of Master Data Management Establishing Consensus for Coordination and Collaboration Data Ownership Semantics: Form, Function, and Meaning Managing Risk through Measured Conformance to Information Policies Key Data Entities Critical Data Elements Defining Information Policies Metrics and Measurement Monitoring and Evaluation Framework for Responsibility and Accountability Data Governance Director Data Governance Oversight Board Data Coordination Council Data Stewardship Summary 86

5 x Contents % CHAPTER5 Data Quality and MDM Introduction Distribution, Diffusion, and Metadata Dimensions of Data Quality Uniqueness Accuracy Consistency Completeness Timeliness Currency Format Compliance Referential Integrity Employing Data Quality and Data Integration Tools Assessment: Data Profiling Profiling for Metadata Resolution Profiling for Data Quality Assessment Profiling as Part of Migration Data Cleansing Data Controls Data and Process Controls Data Quality Control versus Data Validation MDM and Data Quality Service Level Agreements Data Controls, Downstream Trust, and the Control Framework Influence of Data Profiling and Quality on MDM (and Vice Versa) Summary 103 CHAPTER 6 Metadata Management for MDM Introduction Business Definitions Concepts Business Terms Definitions Semantics Reference Metadata Ill Conceptual Domains Ill Value Domains Reference Tables Mappings Data Elements Critical Data Elements Data Element Definition 116

6 Contents xi Data Formats Aliases/Synonyms Information Architecture Master Data Object Class Types Master Entity Models Master Object Directory Relational Tables Metadata to Support Data Governance Information Usage Information Quality Data Quality SLAs Access Control Services Metadata Service Directory Service Users Interfaces Business Metadata Business Policies Information Policies Business Rules Summary 126 CHAPTER 7 Identifying Master Metadata and Master Data Introduction Characteristics of Master Data Categorization and Hierarchies Тор-Down Approach: Business Process Models Bottom-Up Approach: Data Asset Evaluation Identifying and Centralizing Semantic Metadata Example Analysis for Integration Collecting and Analyzing Master Metadata Resolving Similarity in Structure Unifying Data Object Semantics Identifying and Qualifying Master Data Qualifying Master Data Types The Fractal Nature of Metadata Profiling Standardizing the Representation Summary 142 CHAPTER8 Data Modeling for MDM Introduction Aspects of the Master Repository 144

7 xii Contents Characteristics of Identifying Attributes Minimal Master Registry Determining the Attributes Called "Identifying Attributes" Information Sharing and Exchange Master Data Sharing Network Driving Assumptions Two Models: Persistence and Exchange Standardized Exchange and Consolidation Models Exchange Model Using Metadata to Manage Type Conversion Caveat: Type Downcasting Consolidation Model Persistent Master Entity Models Supporting the Data Life Cycle Universal Modeling Approach Data Life Cycle Master Relational Model Process Drives Relationships Documenting and Verifying Relationships Expanding the Model Summary 157 CHAPTER9 MDM Paradigms and Architectures Introduction MDM Usage Scenarios Reference Information Management Operational Usage Analytical Usage MDM Architectural Paradigms Virtual/Registry Transaction Hub Hybrid/Centralized Master Implementation Spectrum Applications Impacts and Architecture Selection Number of Master Attributes Consolidation Synchronization Access Service Complexity Performance Summary 176

8 Contents xiii CHAPTER10 Data Consolidation and Integration Introduction Information Sharing Extraction and Consolidation Standardization and Publication Services Data Federation Data Propagation Identifying Information Indexing Identifying Values The Challenge of Variation Consolidation Techniques for Identity Resolution Identity Resolution Parsing and Standardization Data Transformation Normalization Matching/Linkage Approaches to Approximate Matching The Birthday Paradox versus the Curse of Dimensionality Classification Need for Classification Value of Content and Emerging Techniques Consolidation Similarity Thresholds Survivorship Integration Errors Batch versus Inline History and Lineage Additional Considerations Data Ownership and Rights of Consolidation Access Rights and Usage Limitations Segregation Instead of Consolidation Summary 199 CHAPTER11 Master Data Synchronization Introduction Aspects of Availability and Their Implications Transactions, Data Dependencies, and the Need for Synchrony Data Dependency Business Process Considerations Serializing Transactions 206

9 xiv Contents 11.4 Synchronization Application Infrastructure Synchronization Requirements Conceptual Data Sharing Models Registry Data Sharing Repository Data Sharing Hybrids and Federated Repositories MDM, the Cache Model, and Coherence Incremental Adoption Incorporating and Synchronizing New Data Sources Application Adoption Summary 216 CHAPTER12 MDM and the Functional Services Layer Collecting and Using Master Data Insufficiency of ETL Replication of Functionality Adjusting Application Dependencies Need for Architectural Maturation Similarity of Functionality Concepts of the Services-Based Approach Identifying Master Data Services Master Data Object Life Cycle MDM Service Components More on the Banking Example Identifying Capabilities Transitioning to MDM Transition via Wrappers Maturation via Services Supporting Application Services Master Data Services Life Cycle Services Access Control Integration Consolidation Workflow/Rules Summary 234 CHAPTER13 Management Guidance for MDM Establishing a Business Justification for Master Data Integration and Management Developing an MDM Road Map and Rollout Plan 240

10 Contents xv MDM Road Map Rollout Plan Roles and Responsibilities Project Planning Business Process Models and Usage Scenarios Identifying Initial Data Sets for Master Integration Data Governance Metadata Master Object Analysis Master Object Modeling Data Quality Management Data Extraction, Sharing, Consolidation, and Population MDM Architecture Master Data Services Transition Plan Ongoing Maintenance Summary: Excelsior! 257 Bibliography and Suggested Reading 259 Index 261