Knowledge Integrity Incorporated Busting 7 Myths about Master Data Management Prepared by: David Loshin Knowledge Integrity, Inc. August, 2011 Sponsored by: 2011 Knowledge Integrity, Inc. 1 (301) 754-6350
Introduction The promise of creating a single source of truth containing a golden copy of your customers or products has driven many organizations to prematurely invest in technologies that may not necessarily meet their needs. There still is a lot of hype about master data and master data management (MDM); analyst briefs, popular press articles, and vendor product sheets provide high- level descriptions that lead the information management consumer to believe that MDM delivers significant values straight out of the box. And while MDM can add value, the reality is that a recent survey noted that only 24 per cent rated their own [MDM] projects as successful or better. 1 This low success rate must be attributable to a difference between what we believe to be true about master data management and what the real implications are for success or failure. This paper will investigate those differences, and our goal is to highlight some common misunderstandings or myths about MDM, explain why these are myths, and to provide some guidance about making better choices when deciding to pursue a master data strategy. Introducing some critical thought into different aspects of MDM and putting some common beliefs into perspective will help you put together a more thoughtful plan for a successful MDM deployment. There are no substitutes for good data management disciplines, and this paper will advise on transitioning from a magical, solely product- based master data strategy to one that embraces best practices for data modeling, managing metadata, and implementing the right data quality and data governance practices that can help differentiate what you heard, why it is a myth, and some considerations to planning your MDM blueprints and roadmaps. At that point, well- defined processes within a culture of data governance will simplify technology acquisition and reduce time to value for MDM implementations. 1 See http://www.cio.co.uk/article/3280854/the- missing- link- in- data- quality 2011 Knowledge Integrity, Inc. 2
Myth #1: MDM Provides a Single Source of Truth Out of the Box Master data management is positioned by vendors, contractor companies, and analysts alike to provide a simple and straightforward way to create a single source of truth for your customer or product data as soon as product suites are installed. The concept of a single source of truth proposes significant promise to any organization in which similar data entities are represented in different data subsystems supporting different applications. But what is really meant by the concept of a single source of truth? At an abstract level, it is a presumption that a product can materialize a single repository record completely containing all the information about each customer or product, and that single record can equally satisfy the needs of every business process. The devil, though, is in the details, and when the concept is examined more closely, we start to see some potential gaps. The concept of a single source of truth itself is somewhat shrouded in mystery, especially when considering that The intended uses of the master data may be different than the original uses when the source systems are developed. There is little control of the interpretation of the meanings associated with the master domains (such as customer or product ) once the data is made available for sharing. The original systems were developed in a vacuum, with little coordination in defining business terms, data element concepts, etc. The products are not intended to resolve the issues associated with the original sources, but to provide a framework for integration and consolidation. Suffice it to say, no product is likely to immediately resolve the issues caused by variation in the original sources or the different perceptions of meanings by the data consumers. Considerations The reality is that a reasonable amount of effort must be invested prior to the installation of the product and the extraction, integration, and consolidation of data into a single master data set: Differences in data element structures, types, and semantics must be examined for critical differences and those discrepancies must be resolved; Master data consumers must be engaged so that their needs can be assessed and their requirements documented; 2011 Knowledge Integrity, Inc. 3
Master domain models must be crafted in ways that support data consolidation and provide a structurally and semantically coherent view of core master data elements; Inheritance characteristics for modeling must reflect the embedded relationships associated with the master domains (such as when a customer is also an employee ) to ensure that unique entity attributes are managed in one and only one location; Reference data sets must be analyzed and conformed to provide a unified set of conceptual domain values used within the master data models; and Hierarchies for classification and organization of master data entities must be defined. These are some of the steps that need to be considered as part of a master data management program to ensure that the data within the master repository or registry appropriately satisfies the needs of the consumer community. Savvy vendors understand these demands, and seek to engage their prospects to focus on the governance aspects of MDM early in the program. By incorporating support for data discovery, semantic and structural use of shared metadata, and careful alignment of data quality methods and techniques within the data integration and consolidation processes, software vendors can help their clients combine good data management practices with best- of- breed tools for a successful master data implementation. 2011 Knowledge Integrity, Inc. 4
Myth #2: MDM is a Product All you need to do is buy and install an MDM product and immediately all your data will be consolidated into a single repository. There have always been silver bullet technologies that are touted as solving even the most difficult problems, yet these products often deliver far less than what is promised. Disabuse yourself of the notion that a product will solve your customer or product data integration and sharing challenges. Master Data Management is about managing master data, requiring the proper processes, governance, skills, and business engagement. The value of master data management is derived from enhancing the utility of disparate data through governed integration and centralized oversight and management. Yet even if MDM itself is not a product, once all the aspects of the master data management discipline are in place, there is a broad range of tools, technologies and products that are necessary to support the deployment. Considerations The high failure rate for MDM activities may be partially attributable to high expectations that the technology alone will solve the hard problems. In reality, the deployment of MDM and data quality tools is most effective when preceded by an assessment of organizational expectations for master data use and a synthesis phase in which business needs are clearly identified. The tools work best when the policies and procedures for data governance and process renovation direct the integration of technology to meet those needs. MDM products can simplify the deployment of a master data strategy, but before starting the procurement process, understand your organization s intent for a unified view of master data domains. After documenting clear business requirements, the team will not only have a simplified process for evaluating product suitability and selecting vendor tools, but can also be confident in making more informed choices. By providing the framework for managing master data, the right MDM products will enhance the maturity of the master data management discipline for managing and sharing corporate information resources. 2011 Knowledge Integrity, Inc. 5
Myth #3: MDM is a Project Implementing MDM is a self- contained project with a set of tasks; when those tasks are completed you have a fully functional master data environment. The term project implies that once a set of tasks are completed, the activity is done. However, the objective of a master data management initiative is transforming the way the organization looks at and employs information in a collaborative and synergistic way. Some key aspects of this transformation include: Developing information as a management resource that provides a unified access to each master domain of uniquely defined entities; Organizing the relationships between master data domains to reflect the ways business processes expect to use data models; Classifications and organizations of concepts from a semantic perspective to optimize business processes. But this cannot be a one- time deal because organizations and their corresponding needs, business processes, applications, and data sets are constantly changing. There is a need for continued investment in management, governance, and processes along with technology, and a discrete list of tasks cannot fully support the infrastructure s needs to support this business process evolution. Considerations MDM cannot be a project, since it involves constant evolution. And as we have seen, MDM is not a product either, since the products and tools can support innovation and change, but change won t happen as a byproduct of acquiring tools. But if MDM is neither a product nor a project, what is it? A mature approach would define master data management as a set of best practices in data management and corresponding disciplines for managing quality and usability of commonly shared data concepts. When viewed in that perspective one can differentiate between project steps for specific aspects and the continued management and procedural aspects for long- term oversight and maintenance. 2011 Knowledge Integrity, Inc. 6
Myth #4: MDM is about Data Consolidation MDM is a set of methods for consolidating all of your data sources into a single repository. Most vendors of MDM products focus on the mechanical aspects of extraction, transformation, reduction, and integration of data pulled from multiple originating data sources into a single master data repository. Tools vendors promote their record linkage capabilities and their ability to apply survivorship rules to create a golden record. But even with a perfect set of methods for consolidation, the benefits of master data management come from the use of master data, not its creation. In other words, the benefits of MDM are really based on the formulation, accessibility, and publication of a unified view of the master domains for the individual and procedural master data consumers. From a different perspective, MDM goes beyond the data MDM also involves reduction in replicated functionality and improved enterprise services. MDM is not (only) about consolidation; one might say that MDM is really about meaningful approaches for data sharing and repurposing. Considerations Organizations whose approach to MDM is to dump their data into a repository without considering the downstream usage scenarios will be at a disadvantage. The focus on the front end of the process will essentially bias their methods for modeling, impose constraints on their extraction of data from the original sources, and introduce context- sensitive rules based on immediate demands. This leads to a continuous need for reviewing the current data models, determining existing structure and service gaps, and reengineering the models and the business services layers. But for the consolidation process to support meaningful data sharing and repurposing, one must consider two aspects of data use: the current needs, and anticipating expectations for master data. An alternative approach identifies how master data is to be used prior to initiating the consolidation process. The benefits of this performing this analysis first helps one to Understand the usage scenarios for the selected master data domains, Enables the careful development of models for each master domain, and Design master data services to support its use. Understanding how master data is to be used will reduce the complexity and scope of the master data environment and define requirements to drive the consolidation effort. In turn, this helps refine the scope and complexity of front- end data extraction and transformation while simplifying the data integration and publication for the master data consumer community. 2011 Knowledge Integrity, Inc. 7
Myth #5: MDM is the same as Data Quality The reason to do MDM is to improve the quality of your data. In addition to the common themes of single source of truth and golden copy of the customer (or product, etc.), the proponents of MDM promote its value as a way of increasing the quality of your data. Certainly, the parsing, standardization, and linkage applied to identify duplicate records, followed by applying survivorship rules can consolidate duplicates into a single record. Applications can employ these activities as a filter when creating new records to ensure that duplicates are not being added to the mix. And operational data governance can also help simplify areas of structural and semantic variation. But these methods are just one aspect of master data management, which implies controlled and structured migration of business processes to transition to the use of a master repository. And if one desired to improve the quality of organizational data, parsing, standardization, and entity/identity resolution methods can be incorporated with data governance without fully committing to a using a master data environment. More simply put, if the issue to be resolved is poor data quality, instead of initiating the significant effort involved in MDM, focus on a data quality effort instead. Considerations If the objective is improving the quality of organizational data, consider instituting a data quality management and control program aligned with your data governance program. Good data management practices for assessing data quality, issue prioritization, root cause analysis, incident management, and remediation can be combined with methods and tools such as data profiling, data quality scorecards, issue notification, parsing, standardization, entity/identity resolution, and cleansing to enable services and procedures to measurably improve the quality of enterprise information. In their own right, these are not trivial tasks, and expecting those results as a side effect of implementing master data management can lead to a false sense of quality. Data quality management can be a challenge, but the efforts involved can be mapped out into an operational program, dependent on best- of- breed data discovery and data quality improvement methods and tools, with key stages over a reasonable time frame with predictable results. 2011 Knowledge Integrity, Inc. 8
Myth #6: MDM is Unrelated to Data Quality Master data management is completely independent of data quality management. This myth is the opposite of the previous one that equated MDM with data quality, and suggests that the value proposition for MDM can be achieved independent from deploying a corresponding data quality strategy. The presumption is that the master data system manages the creation of, persistence of, and access to the single representation of any entity, but the quality of the representation is outside of its scope. This approach is evident with MDM systems that are completely model- based but do not provide a native connectivity to external data quality tools or methods. But deploying the master models without considering the necessary data quality implications will lead to poorly- implemented data management practices, resulting in yet more duplication, structural variation, and differences in data meanings. Considerations and Alternatives Effectively providing a unified view of any master data domain requires some approaches to standardization of both metadata and of the data values themselves to reduce the opportunities for bypassing good practices for entity representation creation and use. A reasonable approach to MDM incorporates some fundamental metadata management, data quality, and data governance policies and processes. Incorporate processes for proposing and approving data structures and definitions by having data producers and consumer collaborate regarding the ways that data entities are created, managed, used, and repurposed. Data quality management, control, and cleansing will all contribute to the ability to create a master registry, while a general approach to improving data quality simplifies the mastering process. And lastly, the data governance practices that operationalize validation and control for quality data values will reduce the overhead necessary for creating, maintaining, and providing shared access to high quality master data. 2011 Knowledge Integrity, Inc. 9
Myth #7: MDM is Only About the Data Warehouse Your master data domains are the same thing as the dimensions in your data warehouse, and therefore MDM a part of the data warehouse. Data management professionals working with data warehouses are quick to note the similarity between the concept of master data domains and the dimensions of a data warehouse. Yet despite the similarity, a master data management framework provides more than just a conformed set of unique entities to supplement the fact tables. Because the data warehouse is frequently segregated from the operational systems from which its data is sourced, there is the risk of inconsistency of data as well as a potential lack of coherence between up- to- date data in the sources vs. the versions in the warehouse. By creating a unified view of each master domain and providing a single set of services for accessing the master data, MDM can be used to synchronize and maintain master data in operational systems intraday, and that solidifies consistency and coherence among the operational systems and the warehouses and marts used for reporting and analysis. Considerations and Alternatives By focusing on the methods for simplifying access to a unified view of master data, we can go beyond reducing replicated copies within each master domain. Creating a single service layer for creation, updates, and access and exposing those services to both operational and analytical applications, the reduction in replicated functionality provides the techniques to maintain data consistency and temporal coherence and synchrony. Other considerations such as effective data integration or even federation methods can reduce duplication even more, as well as provide a seamless set of methods for ensuring consistency. Taking these steps will reduce inconsistencies between different operational systems as well as discrepancies between BI reports and analyses and existing operational data. That will eliminate the perceived differences between application data silos 2011 Knowledge Integrity, Inc. 10
Considerations: Taking the Steps in Formulating an Effective MDM Roadmap We have an opportunity to transition the popular perception of master data management into a set of best practices and disciplines for better supporting the business information consumers, and ultimately our technology acquisition strategy must complement the renovated business processes in their reliance on high quality shared information. When reviewing our collected considerations stemming from busting these common MDM myths, a number of comment themes continue to emerge: Information requirements analysis: Recognize that the value of an MDM framework is based on shared use of a unified view of the critical master domains. This implies shoring up and establishing good data management practices focused on soliciting, understanding, and synthesizing a set of requirements for shared data. These practices, when scoped within the framework of the organizational perspective, will simplify the evaluation, acquisition, and deployment of good data management tools. Focus on the shared use of data: Once the requirements have been gathered, ensure that all potential consumers can access and use the master data by providing the appropriate methods and technologies for creating data models that satisfy the collected needs and provide data integration techniques and tools for consolidation as well as for publication, federation, and sharing. Foster common structure and semantics: As more consumers contribute their expectations about the usability of master data, it is imperative that the variations in formats, structures, and meanings be identified and reduced when possible. Providing a collaborative set of data governance procedures for business term definitions, data element structures, and shared semantics will reduce inconsistency and filter out potential errors. Manage shared reference data: A good approach to piloting the MDM methodology is to pilot the MDM tools and technology by applying good metadata practices to commonly- used reference data sets. Program manage your master data: Take an approach that treats master data management as a long term program that combines people, process, and technology, with the requisite roadmap and funding, and maintenance plans to ensure success. Don t make MDM subsidiary to the data warehouse: Rely on the dimensional concepts taken from the data warehouse to help solidify a set of shared data models that can support all data consumers. Recall the mutual dependence of master data and data quality management: Consider implementing a data quality program in lock step with the planning and deployment of your MDM program, along with the corresponding data quality practices, processes, and tools. High quality data is necessary for a successful MDM deployment, but realize that improved data quality can provide both long- term and immediate business benefits. 2011 Knowledge Integrity, Inc. 11
With an appreciation for the differentiation between the hype and the reality, your team can not only develop a reasonable blue print and road map for MDM, you can also identify the key business needs for technology acquisition at the appropriate times in the cycle. Working with progressive tools vendors and service providers who will help you understand the planning, design, and implementation cycle for master data management will help your team evaluate the best tools and technologies to fit your business needs. 2011 Knowledge Integrity, Inc. 12
About the Author David Loshin, president of Knowledge Integrity, Inc, (), is a recognized thought leader and expert consultant in the areas of data quality, master data management, and business intelligence. David is a prolific author regarding BI best practices, via the expert channel at www.b- eye- network.com and numerous books and papers on BI and data quality. His book, Business Intelligence: The Savvy Manager s Guide (June 2003) has been hailed as a resource allowing readers to gain an understanding of business intelligence, business management disciplines, data warehousing, and how all of the pieces work together. His most recent book is The Practitioner s Guide to Data Quality Improvement, and his insights on data quality can be found at www.dataqualitybook.com. His book, Master Data Management, has been endorsed by data management industry leaders, and his valuable MDM insights can be reviewed at www.mdmbook.com. David can be reached at loshin@knowledge- integrity.com About the Sponsor About Information Builders Information Builders software and services transform data into business value for game- changing results. Our solutions for business intelligence and analytics, integration, and data integrity empower people to make smarter decisions, strengthen customer relationships, improve employee performance, and drive growth. Our dedication to customer success is unmatched in the industry. That s why tens of thousands of leading organizations rely on Information Builders solutions to run their business for developing and executing on a world- class information strategy. In addition, we are a major provider to leading technology vendors including HP, IBM, Oracle, SAP, Teradata, Amdocs, and many others. Founded in 1975, Information Builders is headquartered in New York, NY, with offices around the world and remains one of the largest independent, privately- held companies in the software industry. Visit us at informationbuilders.com, follow us on Twitter at @infobldrs, like us on Facebook, and visit our LinkedIn page. 2011 Knowledge Integrity, Inc. 13