Proactive DATA QUALITY MANAGEMENT. Reactive DISCIPLINE. Quality is not an act, it is a habit. Aristotle PLAN CONTROL IMPROVE

Transcription

1 DATA QUALITY MANAGEMENT DISCIPLINE Quality is not an act, it is a habit. Aristotle PLAN CONTROL IMPROVE 1

2 DATA QUALITY MANAGEMENT Plan Strategy & Approach Needs Assessment Goals and Objectives Program Plan Governance Alignment Priorities Problems & Impact Business Justification Requirements & Effort Timelines & Roadmaps People Data Quality Core Team Data Stewards and Analysts Business and IT roles Data Access Needs Process Data Quality Forum Engagement Process Inputs, Drivers, Constraints Performance Measurement Technology Data Profiling & Analysis Cleansing & Enrichment Metrics & Scorecards Monitoring & Alerts 2

3 Planning: Key Points Data quality focus and investment should always be associated to business case needs and benefits such as: Revenue gain or cost reduction Business process improvement Compliance and regulatory requirements, Business intelligence or reporting accuracy Data Integration A data quality management (DQM) strategy and approach should create a sustainable, closed-loop process that will help drive and support an ongoing quality culture. A successful DQM forum will depend on having committed, capable people -- such as data stewards and data analysts that are able to be highly engaged. Quality management needs to be both proactive and reactive. 3

4 Planning: Key Points (continued) A DQM process needs to be tightly coupled with a data governance so that data policies, standards, rules, and compliance requirements are routinely factored into DQM decisions. As governance matures, the effort to manage quality will decline. Establish an initial baseline measurement of data quality that will serve as the undisputed gauge and starting point for improvement. Quality starts at data entry. Structured data, particularly master data, can largely be a fixed asset with only limited amount of ability to change, enrich, or cleanse it. Therefore data value is highly dependent on how accurately the data is captured and how relevant it will be in context to its usage. The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second rule is that automation applied to an inefficient operation will magnify the inefficiency. Bill Gates 4

5 Planning: DQ Forum Participation Data quality lead Data quality team DQ Forum Line of business liaisons 5

6 Planning: DQ Forum Review Process Rules, policies & procedures Requirements from driver(s) Analyze and review requirements Evaluate compliance of requirements Y Violation? Y N Request data analysis and profiling Drive the design of solution(s) Submit request to DG for amendment Amendment possible? N Requirements cannot be fulfilled Evaluate compliance of solution(s) Violation? N Y Submit request to DG for amendment Amendment possible? Y N Requirements cannot be fulfilled Rules, policies & procedures Review solution(s) and obtain approval Assign work to proper team for execution Obtain final signoff Requirements delivered 6

7 Cost Reactive Planning: Cost & Benefit Analysis example High D&B DUNS Mapping EMEA/APAC Address Cleanup Purchase Data Quality Analyzer Tool US/CAN Address Validation & Cleansing Item Description Cleanup Duplicate Customer Merge 8668 Parts Taxonomy Consolidation Account Code Cleanup Service Code Analysis Parts Code Standardization Duplicate Contact Cleanup Customer Name Standardization User Training Update Low Low High Benefit 7

8 DATA QUALITY MANAGEMENT Improve Analyze Root Cause Analysis Impact Analysis Level of Effort Dependencies Communicate Progress Against Target Final Outcome Positives & Negatives Acknowledgements Measure Measurement Plan Success Factors Define Metrics Current Baseline Solution Level of Effort Assessment Roles & Resources Tools & Technology Improvement Plan Execute Initiate Improvement Plan Project/Task Management Complete Tasks Measure & Validate Results 8

9 Improve: Key Points When defining quality improvement plans, make sure that the business impact issues are well qualified and the improvement efforts can be conducted within a reasonable time. Complex, longer term quality improvement projects will often require a phased approach and series of incremental work. Be sure there is frequent reviews and avoidance of scope creep. Use consistent data analysis techniques and tools in order to built trust and confidence around how issues and solutions are qualified and quantified. Always establish clear success factors and associated quality metrics before beginning the improvement effort. Be able to accurately measure progress and the final outcome. Ensure that any major improvement results are reflected in data quality scorecards and monitors. 9

10 Improve: Key Points Always consider ability to use technology to create automated solutions that can eliminate or minimize manual effort. Expect that improvement plans can involve both front-end and back-end solutions. Business and IT resources need to be actively engaged when necessary. Ensure that efforts from cross-functional and regional teams are appropriately recognized and acknowledged. Ensure that improvement plans also address ongoing maintenance and quality control needs. Provide frequent status updates and maintain a closed-loop process with the Data Qaulity Forum and the Data Governance Council. 10

11 Improve: DQM Process example Drivers 1 Requirements/ Problem description Rules, Policies &Procedures DQ Forum 2 Controls/ Data Governance 6 Proposal 5 Rules, Policies &Procedures Design Team 4 Data Analysis Data Analysts 7 IT Support/Dat a Stewards Approval 8 Execution Metadata & other refs Docs Data Sources 3 Data Profiling Metrics 11

12 Improve: Defining Measurements Example Work through the governance process to agree on the attributes, thresholds, weights, and quality dimensions that will be represented in a data quality scorecard Test, validate, adjust as needed, then promote metrics to production Drive improvement goals and maintenance plans from these metrics 12

13 DATA QUALITY MANAGEMENT Control Focus Compliance Privacy Quality Goals Quality Thresholds Communicate Roadmap & Priorities Achievements Dashboards Decisions & Policies Monitor Key Indicators Situational Monitors Process Performance Quality Scorecards Maintain Data Maintenance Rules Quality Standards Data Steward Roles Steady State Activity Manage Resources, Budget, Priorities New Issues & Initiatives Compliance & Regulations Drive Self-Control Practices 13

14 Control: Key Points Ensure that the DQM control is sufficiently focused around: Compliance and regulatory requirements Data issues that impact business performance and business intelligence User, management and training issues that if left unchecked, could undermine the ability to maintain quality standards Keep data quality metrics and monitors current and relevant. As business priorities and measurement needs change, so should the measurement focus. Obsolete old, stale metrics. Data quality scorecard or index tools should include drill down capability to the underlying detail data so that specific reports can be fed back to data maintenance teams who can conduct the corrective action activity. 14

15 Control: Key Points Quality standards and data validation rules should be easily accessible in profiling and measurement tools or in metadata repositories so that DQM and data governance teams can review these as needed. Clearly communicate what a data quality steady state is and where the current state is in relation to this. Not everything can or should be fixed. Agree on what is sufficient. Well placed data stewards should be at the forefront of quality management. They are the best eyes and ears for managing and monitoring data quality on a day-to-day basis. New emerging data quality issues may need to subordinate other existing issues and priorities. Be prepared to react quickly as needed and justify change in priorities. 15

16 Control: Data Quality Monitor Example 16

17 Control: Data Steward Roles Data Stewards Data Quality Core Team Data Domain Role Manage DQM Initiatives Member of Quality Forum Analysis & Measurement Data Access Gatekeeper Support Governance Needs IT Engagement Sales Service Finance Other Operational Process Areas Role Process Area Data Expert Manage Local DQM Initiatives Enforce Policies & Standards Monitor and Control Raise Issues, Help Resolve 17

18 Appendix Section 18

19 About the Authors Mark Allen and Dalton Cervo are co-authors of the book Master Data Management in Practice: Achieving True Customer MDM (John Wiley & Sons, 2011). For more reference please visit Mark Allen has over 20 years of data management and project management experience including extensive planning and deployment experience with customer master initiatives, customer data integration projects, and leading data quality management practices. Mark is a senior consultant and enterprise data governance lead at WellPoint, Inc. Prior to WellPoint, Mark was a senior program manager in customer operations groups at both Sun Microsystems and Oracle Corporation. At Sun Microsystems, Mark served as the lead data steward for the customer data domain throughout the planning and implementation of Sun s enterprise customer data hub. Mark has led implementation of various customer MDM-orientated programs including customer data governance, data quality management, data stewardship, and change management. Mark has championed many efforts to improve customer data integration practices, improve quality measurement techniques, reduce data duplication and fragmentation problems, and has created hierarchy management practices that have effectively managed customer entity structure and corporate linkage. Mark has served on various customer advisory boards and user groups focused on sharing and enhancing MDM and data governance practices. Dalton Cervo has over 20 years experience in software development, project management, and data management areas, including architecture design and implementation of an analytical MDM, and management of a data quality program for an enterprise MDM implementation. Dalton is a senior solutions consultant at DataFlux, helping organizations in the areas of data governance, data quality, data integration, and MDM. Prior to DataFlux, Dalton served as the data quality lead for the customer data domain throughout the planning and implementation of Sun Microsystems enterprise customer data hub. Dalton has extensive hands-on experience in designing and implementing data integration, data quality, and hierarchy management solutions to migrate disparate information; perform data cleansing, standardization, enrichment, and consolidation; and hierarchically organize customer data. Dalton contributed a chapter on MDM to Phil Simon s book, The Next Wave of Technologies Opportunity in Chaos. Dalton is a member of the Data Quality Pro expert panel, has served on customer advisory boards, and is an active contributor to the MDM community through conferences and social media vehicles. Dalton has BSCS and MBA degrees, and is PM certified. 19

20 Helpful References Publications: Loshin, David. The Practitioner s Guide to Data Quality Improvement. Burlington: Morgan Kaufmann Publishers/Elsevier, Maydanchik, Arkady. Data Quality Assessment. Bradley Beach, NJ: Technics Publications, LLC, McGilvray, Danette. Executing Data Quality Projects: Ten Steps to Quality Data and Trusted Information. Burlington, MA: Morgan Kaufmann Publishers/Elsevier, Web Sites: Data Quality Pro: Obsessive Compulsive Data Quality: MDM in Practice: 20