AVOIDING SILOED DATA AND SILOED DATA MANAGEMENT Dalton Cervo Author, Consultant, Management Expert September 2015 This presenta?on contains extracts from books that are: Copyright 2011 John Wiley & Sons, Inc. Copyright 2015 Elsevier Inc.
Synopsis Siloed, disparate, fragmented, and conflic?ng data are a fairly known issue faced by many companies today Companies have come to the realiza?on Management disciplines are a must to address their data problems However, Management disciplines themselves cannot remain siloed Highest poten?al can be achieved by properly blending data management disciplines
About Dalton Cervo President and founder of Gap Consul?ng, providing data management consul?ng services in Master Management (MDM), Architecture, Integra?on, Quality, Governance, Stewardship, Reference Management, Metadata Management, Lifecycle Management, Warehouse, and Analy?cs & Business Intelligence. Over 24 years of experience in data management, project management, and so[ware development, including architecture design and implementa?on of mul?ple MDM solu?ons, and management of data quality, data integra?on, metadata, data governance, and data stewardship programs. Experience in a wide variety of industries, such as automo?ve, telecom, energy, retail, and financial services.
About Dalton Cervo (cont.) Prior to Gap Consul?ng, served as a consultant for SAS/Flux, providing expert knowledge in MDM, data governance, data quality, data integra?on, and data stewardship. Prior to that, held the posi?on of senior program manager at Sun Microsystems and Oracle Corpora?on, serving as the data- quality lead throughout the planning and implementa?on of Sun s enterprise customer data hub. Coauthor of the following two books: Mul?- Domain Master Management Advanced MDM and Governance in Prac?ce (Morgan Kaufmann, Elsevier, April 2015). Master Management in Prac?ce: Achieving True Customer MDM (John Wiley & Sons, June 2011).
Publisher: Morgan Kaufmann Publica?on date: April 8, 2015 Mul?- Domain Master Management delivers prac?cal guidance and specific instruc?on to help guide planners and prac??oners through the challenges of a mul?- domain master data management (MDM) implementa?on. Authors Mark Allen and Dalton Cervo bring their exper?se to you in the only reference you need to help your organiza?on take master data management to the next level by incorpora?ng it across mul?ple domains. Wriden in a business friendly style with sufficient program planning guidance, this book covers a comprehensive set of topics and advanced strategies centered on the key MDM disciplines of Governance, Stewardship, Quality Management, Metadata Management, and Integra?on.
In this book, authors Dalton Cervo and Mark Allen show you how to implement Master Management (MDM) within your business model to create a more quality controlled approach. Focusing on techniques that can improve data quality management, lower data maintenance costs, reduce corporate and compliance risks, and drive increased efficiency in customer data management prac?ces, the book will guide you in successfully managing and maintaining your customer master data. You'll find the expert guidance you need, complete with tables, graphs, and charts, in planning, implemen?ng, and managing MDM. Publisher: John Wiley & Sons, Inc. Publica?on date: May 25, 2011
Avoiding Siloed and Siloed Management
Agenda, Informa?on, and Knowledge State of Affairs Avoiding Siloed with MDM Avoiding Siloed Management DGq
DATA, INFORMATION, AND KNOWLEDGE
The Basics 4, 2 (without context, these values are meaningless) Informa?on Temperature 4 C, Dew Point 2 C (context adds meaning) Knowledge A temperature of 4 C and a dew point of 2 C, together with a rain, means that there is a chance of icing. This icing can adversely affect the performance of an aircra[. This is the same condi?ons that led to an accident last year. Deicing is recommended.
Types of (sample) Alphanumeric Strings Characters Types Integers Floa?ng Numbers Booleans
Structuring of Structured Semi-structured Unstructured
Categories of (main ones) Master represen?ng key data en??es cri?cal to a company opera?ons and analy?cs because of how it interacts and provides context to transac?onal data Transac?onal associated to or resul?ng from specific business transac?ons Reference typically represented by code set values used to classify or categorize other types of data such as master data and transac?onal data Metadata Descrip?ve informa?on about data en??es and elements such as regarding the defini?on, type, structure, lineage, usage, changes, and so on Others: Historical data, temporary data, etc.
Big (3 V s) 5 V s: + Veracity, Value
STATE OF AFFAIRS: TYPICAL COMPANY
State of Affairs 57% of all companies need more than two days to generate a complete list of customers. 75% of informa?on workers have made business decisions that later turned out to be wrong due to flawed data. Up to 70% of IT resources are spent on building and maintaining connec?ons between systems. A total of 56% of CIOs and IT managers could integrate less than 40 percent of their IT applica?ons with other applica?ons in their organiza?on.
State of Affairs (cont.) Over the next two years, more than 25 percent of cri?cal data in Fortune 1000 companies will con?nue to be flawed, that is, the informa?on will be inaccurate, incomplete or [unnecessarily] duplicated The size of the digital record will grow by a compound annual growth rate of 60%.
Typical Company Internal Systems Vendors Supply- Chain Management Reference Management OperaRonal Systems ODS Business Reports Other 3 V s Order Mgmt MDM CRM ERP EDW and Marts AnalyRcs, Business Intelligence Big Management Social Media
How to solve the problem? TradiRonal, applicaron- driven organizarons Transform - driven organizarons
Management Quality Management Governance Stewardship Metadata Management Integration and Synchronization MDM Management Reference Management Security Architecture Big Predictive Analytics
AVOIDING SILOED DATA WITH MASTER DATA MANAGEMENT (MDM)
Why MDM? Vendor Customer Product Employee Master Pa?ents Service Providers Sites
The Narrow and Shallow View of Domain Col 1 Col 2 Col 3 Col 4 Col 5 Col 6 Col 7 Col m Row 1 Row 2 X X X X Row 3 Row 4 X X X Row 5 X X X Row n *Table represents the full set of data for a par?cular domain at a given source Business Process 1** Business Process 2** **Business processes use a small set of data
Business Case MDM gives companies the opportunity to beder manage its key data assets and thereby improve the overall value and u?lity the data provides within the company It exposes internal process issues and business prac?ces (or lack thereof) that are the underlying constraints to having and maintaining good data
Business Case (cont.) Lack of data management prac?ces leads to: Increased costs due to opera?onal and data redundancies or differences across lines of business. Higher risk of audits and regulatory viola?ons. Poorer BI and analy?cs, adding to customer frustra?on and missed opportuni?es. Customer/partner/vendor/employee dissa?sfac?on and consequently un- realized revenues. Possible overpayment of vendors and customers stemming from duplicate records. Over or under delivery of customer services due to inconsistent customer iden?ty and tracking.
MDM Business Case
Priori?zing Domains
A LOOK INTO SYSTEM- OF- RECORD IN THE CONTEXT OF MDM
Typical Integra?on Silos of Information Sales Order Mgmt Finance Other LOB s Direct Interface Direct Interface Middle Tier Middle Tier Middle Tier Enterprise Service Bus
Which one is the SOR? Sales Order Mgmt Finance Other LOB s <master data> Migra?on Master Hub
SOR a[er MDM Sales Order Mgmt Finance Other LOB s Transac+onal Transac+onal Transac+onal Transac+onal Middle Tier Middle Tier Middle Tier Middle Tier Enterprise Service Bus Middle Tier Master Hub
Remember Start a Governance program early Start a Quality Management program early Clearly establish SOR for each set of adributes per domain Avoid bi- direc?onal interfaces to simplify synchroniza?on: change data at SOR and propagate IT and business collabora?on: SOR management has business and technical implica?ons
DATA GOVERNANCE
Governance Governance is the exercise of authority and control (planning, monitoring, and enforcement) over the management of data assets. DAMA Management Body of Knowledge (DMBOK 2010)
Governance Focus Areas (from The Governance Ins?tute) Policy, Standards, Strategy Quality Privacy/Compliance/Security Architecture/Integra?on Warehouse and BI Management Support
The DGI Governance Framework
DATA LIFECYCLE MANAGEMENT
CRUD Management ERP CRM Vendor SOA Vendor
CRUD Management (cont.) ERP CRM Vendor Duplicated Vendor X No Synchroniza?on?
MDM Styles AnalyRcal Registry Style TransacRon or Persistent Style Hybrid Style
Analy?cal MDM
Registry Style
Transac?on or Persistent Style
Hybrid Style
Why is MDM Complex? Mul?- Domain MDM Governance Stewardship Entity Resolution Quality Integration Metadata Management Mul?ple Domains System of Record Real-?me vs Batch Execu?ve Sponsorship Ownership Business Defini?ons & System Architecture Build vs. Buy Top- down, Middle- out, Bodom- up Rules & Regula?ons Security CRUD
AVOIDING SILOED DATA MANAGEMENT
Management
Quality Management (DQM) DQM is about employing processes, methods, and technologies to ensure the quality of the data meets specific business requirements Trusted data delivered in a?mely manner is the ul?mate goal DQM can be reac?ve or preven?ve. More mature companies are capable of an?cipa?ng data issues and prepare for them
Stewardship stewardship encompasses the tac?cal management and oversight of the company s data assets stewardship is generally a business func?on facilita?ng the collabora?on between business and IT, and driving the correc?on of data issues
What s Metadata It s more than just data about data Metadata is structured informaeon that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an informaeon resource. NISO Na?onal Informa?on Standards Organiza?on
Improving Quality Management Governance Maturity Requirements Maturity++ - - - - Metadata Business Glossary - Lineage Business Rules - Interface Informa?on Models (Conceptual, - Transforma?ons Logical, Physical) - Security Rules Dic?onary - Standards & Frameworks
What s Important to Stewards Policies & Procedures Business Requirements and DefiniRons Fitness for Use Stewards
Improving Stewardship Stewardship Governance Stewardship Quality Maturity++ - - - - Metadata Business Glossary - Lineage Business Rules - Interface Informa?on Models (Conceptual, - Transforma?ons Logical, Physical) - Security Rules Dic?onary - Standards & Frameworks
Metadata Management and Governance - - - - - - - Context Business Defini?ons Business Rules Quality Rules Expected Values Rules and Regula?ons Usage by Business Processes Metadata Metadata Management Business Units Governance Metadata management arrfacts are sure to increase knowledge, which is crircal to befer governance decisions. But the not so obvious is the following: - Ownership Management - Impact Analysis - Audit Trail - Lifecycle Management
Reference Management Reference Management Reference Lookup Lists Credit Reports Business Profile Vehicle Catalogs Individual Demographics Auto- Online Auc?ons Household Informa?on Auto- Physical Auc?ons Address Reference Auto Used Car Pricing Tax Informa?on OEM <more> Governance Integra?on Architecture & CRUD Sync & Design DQM
CMMI Ins?tute Management Maturity Model
Management Quality Management Governance Stewardship Metadata Management Integration and Synchronization MDM Management Reference Management Security Architecture Big Predictive Analytics
DGq A Gap product to measure and monitor DQ ReporEng and Monitor Layer Web- driven Configurator Robust - model Rule Engine Quality Repository Layer Quality Measurement Layer Oracle MySQL MS SQL Server NoSQL Layer
DGq Model Highlights On-premise or Cloud-based It supports the following primary concepts: - Domains/EnRRes - Business Areas - Stewards - Quality Dimensions - Rules and Metrics - Scorecards and Dashboards - Thresholds and Alerts I18n Scalable and extensible Multiple DB Platforms
DGq at Work Current- state summary and data quality score Essen?al to priori?ze and drive efforts Iden?fy areas in need of improvements Monitor trends and gauge data fitness for use Recognize detailed profiling needs Opportunity to iden?fy new technological needs Quickly deploy new rules and metrics Standardiza?on of reports allow for consistent interpreta?on Measure investment results Gauge level- of- effort related to data quality improvements Standardiza?on of the repor?ng repository allows for quick deployments Decoupling facilitates maintenance
FINAL THOUGHTS
Con?nuous Improvement Multi-domain MDM Governance Stewardship Integration Quality Management Metadata Management Program Maturity Continuous Improvement
Where to Find Me www.datagapconsul?ng.com www.mdm- in- prac?ce.com www.dcervo.com dalton.cervo@datagapconsul?ng.com hdp://www.linkedin.com/in/dcervo @dcervo
THANK YOU