AVOIDING SILOED DATA AND SILOED DATA MANAGEMENT

Transcription

1 AVOIDING SILOED DATA AND SILOED DATA MANAGEMENT Dalton Cervo Author, Consultant, Management Expert September 2015 This presenta?on contains extracts from books that are: Copyright 2011 John Wiley & Sons, Inc. Copyright 2015 Elsevier Inc.

2 Synopsis Siloed, disparate, fragmented, and conflic?ng data are a fairly known issue faced by many companies today Companies have come to the realiza?on Management disciplines are a must to address their data problems However, Management disciplines themselves cannot remain siloed Highest poten?al can be achieved by properly blending data management disciplines

3 About Dalton Cervo President and founder of Gap Consul?ng, providing data management consul?ng services in Master Management (MDM), Architecture, Integra?on, Quality, Governance, Stewardship, Reference Management, Metadata Management, Lifecycle Management, Warehouse, and Analy?cs & Business Intelligence. Over 24 years of experience in data management, project management, and so[ware development, including architecture design and implementa?on of mul?ple MDM solu?ons, and management of data quality, data integra?on, metadata, data governance, and data stewardship programs. Experience in a wide variety of industries, such as automo?ve, telecom, energy, retail, and financial services.

4 About Dalton Cervo (cont.) Prior to Gap Consul?ng, served as a consultant for SAS/Flux, providing expert knowledge in MDM, data governance, data quality, data integra?on, and data stewardship. Prior to that, held the posi?on of senior program manager at Sun Microsystems and Oracle Corpora?on, serving as the data- quality lead throughout the planning and implementa?on of Sun s enterprise customer data hub. Coauthor of the following two books: Mul?- Domain Master Management Advanced MDM and Governance in Prac?ce (Morgan Kaufmann, Elsevier, April 2015). Master Management in Prac?ce: Achieving True Customer MDM (John Wiley & Sons, June 2011).

5 Publisher: Morgan Kaufmann Publica?on date: April 8, 2015 Mul?- Domain Master Management delivers prac?cal guidance and specific instruc?on to help guide planners and prac??oners through the challenges of a mul?- domain master data management (MDM) implementa?on. Authors Mark Allen and Dalton Cervo bring their exper?se to you in the only reference you need to help your organiza?on take master data management to the next level by incorpora?ng it across mul?ple domains. Wriden in a business friendly style with sufficient program planning guidance, this book covers a comprehensive set of topics and advanced strategies centered on the key MDM disciplines of Governance, Stewardship, Quality Management, Metadata Management, and Integra?on.

6 In this book, authors Dalton Cervo and Mark Allen show you how to implement Master Management (MDM) within your business model to create a more quality controlled approach. Focusing on techniques that can improve data quality management, lower data maintenance costs, reduce corporate and compliance risks, and drive increased efficiency in customer data management prac?ces, the book will guide you in successfully managing and maintaining your customer master data. You'll find the expert guidance you need, complete with tables, graphs, and charts, in planning, implemen?ng, and managing MDM. Publisher: John Wiley & Sons, Inc. Publica?on date: May 25, 2011

7 Avoiding Siloed and Siloed Management

8 Agenda, Informa?on, and Knowledge State of Affairs Avoiding Siloed with MDM Avoiding Siloed Management DGq

9 DATA, INFORMATION, AND KNOWLEDGE

10 The Basics 4, 2 (without context, these values are meaningless) Informa?on Temperature 4 C, Dew Point 2 C (context adds meaning) Knowledge A temperature of 4 C and a dew point of 2 C, together with a rain, means that there is a chance of icing. This icing can adversely affect the performance of an aircra[. This is the same condi?ons that led to an accident last year. Deicing is recommended.

11 Types of (sample) Alphanumeric Strings Characters Types Integers Floa?ng Numbers Booleans

12 Structuring of Structured Semi-structured Unstructured

13 Categories of (main ones) Master represen?ng key data en??es cri?cal to a company opera?ons and analy?cs because of how it interacts and provides context to transac?onal data Transac?onal associated to or resul?ng from specific business transac?ons Reference typically represented by code set values used to classify or categorize other types of data such as master data and transac?onal data Metadata Descrip?ve informa?on about data en??es and elements such as regarding the defini?on, type, structure, lineage, usage, changes, and so on Others: Historical data, temporary data, etc.

14 Big (3 V s) 5 V s: + Veracity, Value

15 STATE OF AFFAIRS: TYPICAL COMPANY

16 State of Affairs 57% of all companies need more than two days to generate a complete list of customers. 75% of informa?on workers have made business decisions that later turned out to be wrong due to flawed data. Up to 70% of IT resources are spent on building and maintaining connec?ons between systems. A total of 56% of CIOs and IT managers could integrate less than 40 percent of their IT applica?ons with other applica?ons in their organiza?on.

17 State of Affairs (cont.) Over the next two years, more than 25 percent of cri?cal data in Fortune 1000 companies will con?nue to be flawed, that is, the informa?on will be inaccurate, incomplete or [unnecessarily] duplicated The size of the digital record will grow by a compound annual growth rate of 60%.

18 Typical Company Internal Systems Vendors Supply- Chain Management Reference Management OperaRonal Systems ODS Business Reports Other 3 V s Order Mgmt MDM CRM ERP EDW and Marts AnalyRcs, Business Intelligence Big Management Social Media

19 How to solve the problem? TradiRonal, applicaron- driven organizarons Transform - driven organizarons

20 Management Quality Management Governance Stewardship Metadata Management Integration and Synchronization MDM Management Reference Management Security Architecture Big Predictive Analytics

21 AVOIDING SILOED DATA WITH MASTER DATA MANAGEMENT (MDM)

22 Why MDM? Vendor Customer Product Employee Master Pa?ents Service Providers Sites

23 The Narrow and Shallow View of Domain Col 1 Col 2 Col 3 Col 4 Col 5 Col 6 Col 7 Col m Row 1 Row 2 X X X X Row 3 Row 4 X X X Row 5 X X X Row n *Table represents the full set of data for a par?cular domain at a given source Business Process 1** Business Process 2** **Business processes use a small set of data

24 Business Case MDM gives companies the opportunity to beder manage its key data assets and thereby improve the overall value and u?lity the data provides within the company It exposes internal process issues and business prac?ces (or lack thereof) that are the underlying constraints to having and maintaining good data

25 Business Case (cont.) Lack of data management prac?ces leads to: Increased costs due to opera?onal and data redundancies or differences across lines of business. Higher risk of audits and regulatory viola?ons. Poorer BI and analy?cs, adding to customer frustra?on and missed opportuni?es. Customer/partner/vendor/employee dissa?sfac?on and consequently un- realized revenues. Possible overpayment of vendors and customers stemming from duplicate records. Over or under delivery of customer services due to inconsistent customer iden?ty and tracking.

26 MDM Business Case

27 Priori?zing Domains

28 A LOOK INTO SYSTEM- OF- RECORD IN THE CONTEXT OF MDM

29 Typical Integra?on Silos of Information Sales Order Mgmt Finance Other LOB s Direct Interface Direct Interface Middle Tier Middle Tier Middle Tier Enterprise Service Bus

30 Which one is the SOR? Sales Order Mgmt Finance Other LOB s <master data> Migra?on Master Hub

31 SOR a[er MDM Sales Order Mgmt Finance Other LOB s Transac+onal Transac+onal Transac+onal Transac+onal Middle Tier Middle Tier Middle Tier Middle Tier Enterprise Service Bus Middle Tier Master Hub

32 Remember Start a Governance program early Start a Quality Management program early Clearly establish SOR for each set of adributes per domain Avoid bi- direc?onal interfaces to simplify synchroniza?on: change data at SOR and propagate IT and business collabora?on: SOR management has business and technical implica?ons

33 DATA GOVERNANCE

34 Governance Governance is the exercise of authority and control (planning, monitoring, and enforcement) over the management of data assets. DAMA Management Body of Knowledge (DMBOK 2010)

35 Governance Focus Areas (from The Governance Ins?tute) Policy, Standards, Strategy Quality Privacy/Compliance/Security Architecture/Integra?on Warehouse and BI Management Support

36 The DGI Governance Framework

37 DATA LIFECYCLE MANAGEMENT

38 CRUD Management ERP CRM Vendor SOA Vendor

39 CRUD Management (cont.) ERP CRM Vendor Duplicated Vendor X No Synchroniza?on?

40 MDM Styles AnalyRcal Registry Style TransacRon or Persistent Style Hybrid Style

41 Analy?cal MDM

42 Registry Style

43 Transac?on or Persistent Style

44 Hybrid Style

45 Why is MDM Complex? Mul?- Domain MDM Governance Stewardship Entity Resolution Quality Integration Metadata Management Mul?ple Domains System of Record Real-?me vs Batch Execu?ve Sponsorship Ownership Business Defini?ons & System Architecture Build vs. Buy Top- down, Middle- out, Bodom- up Rules & Regula?ons Security CRUD

46 AVOIDING SILOED DATA MANAGEMENT

47 Management

48 Quality Management (DQM) DQM is about employing processes, methods, and technologies to ensure the quality of the data meets specific business requirements Trusted data delivered in a?mely manner is the ul?mate goal DQM can be reac?ve or preven?ve. More mature companies are capable of an?cipa?ng data issues and prepare for them

49 Stewardship stewardship encompasses the tac?cal management and oversight of the company s data assets stewardship is generally a business func?on facilita?ng the collabora?on between business and IT, and driving the correc?on of data issues

50 What s Metadata It s more than just data about data Metadata is structured informaeon that describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an informaeon resource. NISO Na?onal Informa?on Standards Organiza?on

51 Improving Quality Management Governance Maturity Requirements Maturity Metadata Business Glossary - Lineage Business Rules - Interface Informa?on Models (Conceptual, - Transforma?ons Logical, Physical) - Security Rules Dic?onary - Standards & Frameworks

52 What s Important to Stewards Policies & Procedures Business Requirements and DefiniRons Fitness for Use Stewards

53 Improving Stewardship Stewardship Governance Stewardship Quality Maturity Metadata Business Glossary - Lineage Business Rules - Interface Informa?on Models (Conceptual, - Transforma?ons Logical, Physical) - Security Rules Dic?onary - Standards & Frameworks

54 Metadata Management and Governance Context Business Defini?ons Business Rules Quality Rules Expected Values Rules and Regula?ons Usage by Business Processes Metadata Metadata Management Business Units Governance Metadata management arrfacts are sure to increase knowledge, which is crircal to befer governance decisions. But the not so obvious is the following: - Ownership Management - Impact Analysis - Audit Trail - Lifecycle Management

55 Reference Management Reference Management Reference Lookup Lists Credit Reports Business Profile Vehicle Catalogs Individual Demographics Auto- Online Auc?ons Household Informa?on Auto- Physical Auc?ons Address Reference Auto Used Car Pricing Tax Informa?on OEM <more> Governance Integra?on Architecture & CRUD Sync & Design DQM

56 CMMI Ins?tute Management Maturity Model

57 Management Quality Management Governance Stewardship Metadata Management Integration and Synchronization MDM Management Reference Management Security Architecture Big Predictive Analytics

58

59 DGq A Gap product to measure and monitor DQ ReporEng and Monitor Layer Web- driven Configurator Robust - model Rule Engine Quality Repository Layer Quality Measurement Layer Oracle MySQL MS SQL Server NoSQL Layer

60 DGq Model Highlights On-premise or Cloud-based It supports the following primary concepts: - Domains/EnRRes - Business Areas - Stewards - Quality Dimensions - Rules and Metrics - Scorecards and Dashboards - Thresholds and Alerts I18n Scalable and extensible Multiple DB Platforms

61 DGq at Work Current- state summary and data quality score Essen?al to priori?ze and drive efforts Iden?fy areas in need of improvements Monitor trends and gauge data fitness for use Recognize detailed profiling needs Opportunity to iden?fy new technological needs Quickly deploy new rules and metrics Standardiza?on of reports allow for consistent interpreta?on Measure investment results Gauge level- of- effort related to data quality improvements Standardiza?on of the repor?ng repository allows for quick deployments Decoupling facilitates maintenance

62 FINAL THOUGHTS

63 Con?nuous Improvement Multi-domain MDM Governance Stewardship Integration Quality Management Metadata Management Program Maturity Continuous Improvement

64 Where to Find Me in- prac?ce.com

65 THANK YOU