Enabling Data Quality

Transcription

1 Enabling Data Quality Establishing Master Data Management (MDM) using Business Architecture supported by Information Architecture & Application Architecture (SOA) to enable Data Quality. 1

2 Background & Agenda q Background: Provide an overview of MDM (master data management) within the larger scope of Business, Information, and Application Architecture. q Agenda: Data Quality Challenges & Opportunities Building the business case for MDM Implementing MDM Operationalizing MDM Some things to avoid Some things to consider 2

3 Understanding the Big Picture q The journey for establishing data integrity starts with understanding the business issues and measuring the impact from data integrity issues. q Often times, the business will bring examples (symptoms) of data integrity issues that are impacting customers, products, and operations. Business q A more systematic approach is required to properly identify the root causes for these issues and that begins with examining the Business, Information, & Application Architectures. Application Information 3

4 Systematic Review Business Architecture 4

5 Impact of Data Quality on Business Scenarios q Customer Need for 360º view of customer Consistent Identification of customer (supports all perspectives) Consistent support for customer (regardless of customer classification) Customer loyalty Common understanding of customer needs (at dept. level and overall) q Product Consolidate product SKUs into logical models Reduce the number of products (proliferation drives up cost) Simplify Customer Experience (finding & ordering) Reduce complexity for engineering design Reduce costs with Finished Goods Processing Reduce complexity with Inventory Mgmt & Distribution Consistent view of product across Sales, Marketing, Engineering, Finance, & SCM q Compliance Ensure patent data is aligned and properly secured Ensure financial data is complete and accurate Ensure intellectual property is managed and versioned properly Ensure SLAs are being met for customers, partners, & vendors (contract compliance) 5

6 Impact of Data Quality on Operational Excellence q Stabilizing Data & Information Consistent information about customers Consistent pricing Simplify the number of products (internally & externally) Alignment of information and reporting for decision making q Improving Technical & Operational Services Streamlining integration and transformation processes across all systems Improving the accuracy of information once data integrity is established Measuring data quality and driving continuous improvement q Enabling Federated Security q Timeframe Determine identity (of human and machine resources) Segregation of duties (across the lifecycle of data and information) Alignment of ACLs (across the logical and physical architecture layers) Consistent Access to data, resources, and information Typically takes 2-3 years to achieve (possibly longer depending on the size of the company. Don t attempt to rush through this process since it takes a while to establish governance, accountability and consistent processes for managing data and information. Master Data Management is only effective when implemented as a core competency. 6

7 Aligning Business Scenarios with Strategy q Common Business Capabilities Customer Relationship Mgmt Resource Management Support Management Communications Mgmt 2 Supply Chain Mgmt Order Mgmt Contract Mgmt Operations Mgmt Product Mgmt Knowledge Mgmt Financial Mgmt R&D 3 1 q Common Strategic Goals q Improve decision making by management q Optimize productivity of human resources q Improve innovation and NPD effectiveness q Find and locate expertise and content by employees, business partners, and customers q Improve collaboration across the value chain q Reduce operational costs q Improve quality q Expand market share q Improve customer retention q Minimize risk Scenarios are derived by identifying GAPS between business capabilities & strategic goals Need to understand why customers are leaving & how to optimize loyalty Need to see if we are leaving money on the table or missing SLAs with customers Need to assess our ability to serve the customer with one voice 7

8 Key Challenges for Organizations q Which geographies should we focus our sales & marketing efforts? q Where are the market opportunities for penetration, growth, and transformation (leading the market)? q Which products should we sunset? q Which features should we pursue for existing products? q How should we prioritize value delivery for customers? q Where are our competitive threats coming from in 2 yrs, 5 yrs? q What new products do we need to fund now to remain competitive 2 yrs from now? q What companies / products should we acquire (and what can we afford)? q How well are we serving our customer needs? q How can we accelerate decision making? q How can we improve the quality of decisions? q Where can we cut costs w/o impacting quality? q What is the value of our Technology investments (beyond ROI)? q How can we optimize our logistics w/o impacting quality & SLAs? q How can we optimize our relationships with suppliers, partners, and distributors? 8

9 Key Challenges for IT q Integration is difficult to maintain and support q Multiple versions of truth exist q Little or no sharing of data and information q Redundant systems, applications, and data q Inconsistent data q Data quality issues impact production and client deliverables q Multiple formats are used and don t align (for the same data elements) q Little or no collaboration between systems and users q Inability to manage data effectively Data management processes are inefficient Escalation and workflow are not well coordinated 9

10 Opportunities: Future Trends for Data Quality 1. In some countries, Data Governance will become a regulatory requirement & companies will have to demonstrate the completeness and accuracy of their Data Governance policies and operational processes to regulators as part of regular audits. This will likely affect Financial Services industries first, & will emerge as a growing trend worldwide. 2. The Value of Data will be treated as an asset (tracked on the B/S by CFO) while data quality (DQ) will become a technical reporting metric & key IT performance indicator. New accounting & reporting practices will emerge for measuring & assessing value of data to help organizations demonstrate how DQ fuels business performance. 3. Measuring Risk will become an IT function as companies shift from a manual process to a fully automated calculation. This will allow companies to proactively measure and manage risk in the future Predictions from the IBM Data Governance Council 10

11 Data Quality on Growth & Transformation q Driving Growth Capturing customer needs using a disciplined process and consistent grammar provides a foundation for identifying opportunities and closing gaps in products & services. These needs need to be aligned with demographic, ethnographic, and segmentation analyses by Marketing to drive growth and innovation efforts. Ideas (domain-specific, technology-focused, adjacent space) all need to be rationalized against the list of unmet needs (prior point) so they can be matched up and/or refined as necessary. Product & Service portfolios need to be rationalized against the features that support the customer needs (as well as the regulatory needs) to proactively identify gaps. q Enabling Transformation Once the organization adopts data quality as a core competency and embraces data rationalization for the front end of innovation, the culture is ready to embrace change and transformation of processes, products, and services is a far more agile process. The organization becomes much more in tune with customer needs, more realistic about their product capabilities, and more willing to drive and support innovation as a leader. q Timeframe This can take 2-4 years as the data quality and impacts on the organization are driven by the organization s ability to adopt a customer-centric culture and formalize the processes required to capture business needs and drive innovation. 11

12 Systematic Review Information & Application Architecture 12

13 Data Governance (overview) Data Governance Standards Policies Operations Data Accessibility Data Availability Data Quality Data Security Data Audit-ability q This is a new operational model that is established to ensure Data Quality is maintained throughout the lifecycle for Master Data (Tier 1) and Supporting Data for Master Data (Tier 2). 13

14 Key Challenges for Data Governance q Break down functional & organizational stovepipes q Integrate processes across the enterprise including corporate technology, all LOBs, functional areas & geographic regions q Engage all levels of management & adjudicate between centralized vs. decentralized data stewardship q Evolve key stakeholders from data ownership to data stewardship q Overcome lack of process integration in current DG for MDM offerings 14

15 Operationalizing Data Governance q Policies & Standards Tolerances & Metrics Standards for Data Quality & Integrity Standards for Data Acquisition & Extraction Performance Standards for Aggregation & Analytics Data Alignment & Synchronization Data Matching Data Translation & Transformation q Processes & Operations Data Conversion Data Modification Data Quality Remediation Data Acquisition Data Extraction Data Security Data Availability & Accessibility Data Quality Data Audit-ability Data Reporting & Delivery Data Translation & Transformation 15

16 Data Quality Services: Lifecycle q Data Acquisition q Data Enrichment q Data Transformation q Data Warehousing q Data Extraction q Data Search q Data Aggregation q Data Summarization Reviewing data quality issues and opportunities, it s helpful to examine the flow of data, information, and knowledge across the lifecycle, since data issues can occur at any point along the way. 16

17 Data Quality Services: Mastering Data q Data Profiling The practice of knowing your data, understanding the issues of the data and where the issues arise. q Data Cleansing The practice of detecting and correcting corrupt data records and data sets with a table or application. Data cleansing is the process of identifying incomplete, incorrect, inaccurate, irrelevant etc. parts of the data and then replacing, modifying or deleting the bad data. Typically Data Profiling will help accelerate the identifying the bad data. q Data Transformations Any time data changes from it s original state from the source to a target system. There are typical data transforms that are used in most system integrations. q Data Verification & Validation Verification is the process of determining the correctness of the data, often performed by Application owners or Data Stewards. Testing against specifications Checking of data before processing to ensure that it is acceptable for it or not Validation is the process of determining if the data is correct. Testing against requirements Checking of data that has been copied from one place to another to ensure that is replaces the original one 17

18 Data Quality Services: Managing & Stewardship q Data Synchronization The process of establishing consistency among data from a source to a target data storage and vice versa and the continuous harmonization of the data over time. It is fundamental to a wide variety of applications, MDM become important when there are more then one system involved. An Integration Framework is required to achieve data synchronization q Data Matching & Linking The process of identifying and resolving data elements that are similar. Using varying degrees of complex scientific processes, weighing and scoring to find data elements with close enough like attributes to safely say they are the same record. Many of the MDM tools on the market today will differentiate themselves with the degrees of intellectual properties in this space q Data Stewardship The role in the organization that enforces Data Governance policies and procedures. Often making sure the rules are enforced through Data Verification and Validation processes. Many Data Integration tools and MDM Solutions will offer Data Stewardship tools as part of the solution. Many times a workflow process and technology as well as an Integration Framework will assist in a successful Data Stewardship Program Executive support is a must as well as buy in from LOB Business and IT Owners 18

19 Data Quality Services: Data Acquisition q Data Source Management q Data Input Quality Management q Data Source Identification q Data Aggregation & Consolidation q Data Filtering q Data Loading q Data Analysis & Recognition q Data Validation q Data Formatting & Alignment q Data Classification q Data Enrichment q Data Conversion & Mapping q Data Loading q Data Auditing & Defect Tracking 19

20 Data Quality Services: Translation & Dimensional Mgmt. q Data Translation / Interpretation / Transliteration Data translations are typically basic translations from on value to another used to make data look similar in nature. A simple example would be convert units of measure, making kilograms into bounds. Data Interpretation can often mean using a statistical, synonyms, antonyms, or derivatives of data to assert another meaning of the data and how an organization will us that data. Data Transliterations is conversions of different languages q Dimensional Management As an organization starts to better understand the value of their data and how better managed data can support business initiative the next step is to start managing multiple domains or dimensions. Many organization will start with Customer (once known as Customer Data Integrations or CDI) and as that provided value they moved to additional dimensions such as Product, locations, accounts, and territories for example. 20

21 Getting Support for MDM Building the Business Case 21

22 Identify Key Metrics for Business Success q Improve Customer Intimacy: CRM consolidation Customer ID Customer Contacts & Locations Customer Support Customer Products (portfolios) Customer Loyalty q Identify Quality Gaps: Information quality Decision Support Regulatory Compliance Product Quality Manufacturing Processes Security over data, IP, and core assets 22

23 Identify Key Metrics for Business Success q Find ways to reduce costs: Operations (reactive to predictive) Product Lifecycle Mgmt Support (product, customer, technology) Decision Turnaround Technology costs q Drive Growth and Innovation: Innovation & NPD Deal optimization Contract Pricing Cross-Selling & Up-Selling Identifying new markets for growth Developing technology platforms to drive value delivery 23

24 Enabling Core Business Capabilities Data Model Extensibility Industry knowledge Metadata Driven Data Quality Data Profiling Stewardship, verification, Validation Data Cleansing Metadata Driven Integration and Synchronization Bath and real time Propagation across system Metadata Driven Technology and Architecture Architectural flexibility Satisfy different use cases MDM Business Service and Workflow Granular and packaged Services Base for SOA applications Metadata Driven Performance, Scalability and Availability Integration Architecture HA, unplanned and planned downtimes, etc Measurement and Analysis Effectiveness of Data Architecture ROI, TCO Manageability and Security Integration with systems management Manage access right and privacy (HIPPA, SOX, FDA, etc) 24

25 Establishing Pragmatic MDM A Stepwise Approach for Implementing MDM 25

26 A Pragmatic Approach for Implementing MDM Managing Transformation (People, Processes & Technology) q Level 1 Stabilize Phase Establish the MDM repository for core master data. Data governance rules and processes are defined through workshops with data owners, architects, and business users. Importantly, metrics are defined. The latter half of the Stabilize Phase focuses on data acquisition and consolidation from primary source systems, and the pilot rollout of MDM-enabled processes. MDM tool selection is conducted and data modeling is initiated. Proof of concept scenarios are defined and executed to test product capabilities as well as to define an implementation plan for the program roadmap. 26

27 A Pragmatic Approach for Implementing MDM Managing Transformation (People, Processes & Technology) q Level 2 Transition Phase Once pilot MDM processes have been deployed and feedback incorporated, the Transition Phase of the MDM program can kick in. This is typically executed in waves to expand the MDM footprint to cover additional systems and processes. The end state of the Transition Phase is typically a rationalized system landscape with over 60% of the master data integrated into the MDM hub accompanies by streamlined lifecycle processes. Data Quality processes are formalized along with a dedicated team who provide metrics and assure compliance Adherence to Standards & Policies exceeds 80% across all data sources. An enterprise-wide Data Registry is created to warehouse the business glossary and data definitions. Data Quality rules are managed by business owners and enforced through rules-driven automation. Key business scenarios should reveal improved metrics (justifying the investment for GDAP). 27

28 A Pragmatic Approach for Implementing MDM Managing Transformation (People, Processes & Technology) q Level 3 Growth Phase Operational processes become hardened with steadily improving metrics Data Quality improves across all Tier 1 data (> 90%) and across all Tier 2 data (> 80%). An Integration Framework is established using a federated model of hubs which easily integrate additional systems and data elements without significant rework to each hub. Each hub manages (one or more) sets of Tier 1 data to ensure data conforms to the standards & policies established for GDAP. Redundant systems are retired and duplicate data storage are reduced (may never be eliminated). Information and knowledge management improve significantly to accelerate decision making and improve the quality of decisions. Product development and revenues improve as marketing and R&D / Product Mgmt are able to gain a clear understanding of what customers want and are able to rationalize customer needs against their portfolio of products & services. Key business scenarios (strategic & partnership level) should reveal improved metrics (justifying the investment for GDAP). 28

29 Common Pitfalls to Avoid q Thinking Technology alone will ensure Data Integrity q Not having proper support from Top down or bottom up q Poor Data Governance Policies and Procedures (or none) q Poor Data Quality and lack of Data Quality Rules Enforcement q Full Business and Functional IT support q Address Data Ownership Upfront q Correct people on the project, both strong technical and business people and better if the people are strong in both areas q Lack of clear Vision and Strategy for MDM q Making MDM Software do more than it was designed to do 29

30 Contact us if we can assist you further: Phone: (847) Online: 30