MDM that Works. A Real World Guide to Making Data Quality a Successful Element of Your Cloud Strategy. Presented to Pervasive Metamorphosis Conference

Transcription

1 MDM that Works A Real World Guide to Making Data Quality a Successful Element of Your Cloud Strategy Presented to Pervasive Metamorphosis Conference Malcolm T. Hawker, Pivotal IT Consulting April 28, 2011

2 A Technology Consulting Firm Founded in 2007 Commercially Facing, Mid Atlantic Account Base & HQ History Across Verticals Technology Media Telecom Education Associations 30 Technologists and Growing Pervasive Software Strategic Partner Cloud Computing Practice based in Austin, TX

3 Data & Systems Integration Pivotal IT will lay the plumbing to get your systems and data in synch. Data Cleanups fix your bad data once and for all with low cost solutions from Pivotal IT. You give us your data and we ll fix it. Simple. Master Data Management break down data silos to create true 360 views of your customers. Data Verification let Pivotal check your data quality against the industry leaders in business intelligence solutions. Well do it once or we ll do it on a regular basis except we ll do it cheaper. Data Quality as a Service (DQAS) - outsource your data quality to Pivotal. We ll manage your data as if it were our own. You do nothing other than reap the rewards.

4 What is MDM? MDM is often used as both a noun and a verb: Master Data Management: a set of tools and processes used for the maintenance and application of core business objects (products, customers, vendors, contracts, etc.) Master Data Model: single version of the truth within an organization for one or more business objects Must Have MDM Features / Functions: Systems and Data Integration to break silos and normalize data Configurable Entity Resolution to consolidate data Data Stewardship Capabilities to manage exceptions Data Access / Syndication Functionality to expose mastered data

5 MDM Needs Identification You May Need an MDM if: your mission critical data is not shared across multiple autonomous business units or departments. Autonomous Business Units and Fiefdoms you re growing quickly through acquisitions. one or more cloud based applications have been implemented outside of IT s control in the last 5 years. 1+ Cloud Based Applications Master Data Management Rapid Growth & Acquisitions it takes two weeks to generate a report showing the lifecycle of a given customer or prospect. the most common answer to the question How many customers do we have? is It depends. No 360 Customer Views Highly manual reporting

6 Cloud Adoption and Data Management The adoption of Cloud based applications is creating significant data related challenges for companies: Cloud Based Application Deployment Premium on Adoption De-centralized Controls Lack of Data Governance Rapid Organic Growth Limited App Support Experience Disparate Business Processes No Data Dictionaries Lax Rules for Data Entry App Quickly Becomes Mission Critical Bootstrapped Admin support Unreliable Reports Inaccurate Data Duplicate Data Inefficient Operations Manual Processes Lost Sales Up sell Cross sell Outside IT Control Limited Systems Integration Lack of IT Governance Data Fiefdoms Data Silos Change Control Problematic Increasing Frequency of Outages Inefficient Marketing No accurate visibility on Customers or Prospects.

7 MDM Solution Providers The market for MDM solutions is dominated by a small number of Enterprise software vendors. Enterprise class solutions at enterprise class prices challengers leaders Complex pricing & gang sales Lengthy implementation schedules Oracle (Siebel UCM) IBM (MDM Server) Lengthy due diligence and purchasing processes Vendors have a preference towards their own technology stacks Bias towards on premise licensed software (no cloud based solutions) Oracle (Oracle CHD) SAP D&B DataFlux TIBCO VisionWare IBM (Initiate) Informatica niche providers visionaries Typical enterprise class MDM pricing starts at $1MM for software only Gartner 2010 Magic Quadrant for MDM of Customer Data. The Magic Quadrant is copyrighted 2010 by Gartner, Inc. and is reused with permission. The Magic Quadrant is a graphical representation of a marketplace at and for a specific time period. It depicts Gartner's analysis of how certain vendors measure against criteria for that marketplace, as defined by Gartner. Gartner does not endorse any vendor, product or service depicted in the Magic Quadrant, and does not advise technology users to select only those vendors placed in the "Leaders" quadrant. The Magic Quadrant is intended solely as a research tool, and is not meant to be a specific guide to action. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

8 Alternative Solution Providers Many vendors are marketing their non MDM solutions as MDM s to capture increased SMB demand Data Warehouse and ETL Solutions Vertical Specific Solutions Data Processors / Data Enrichment Providers Data Integration software providers, esp. Open Source options Often missing data stewardship and entity resolution features Limited number of industries Entity resolution is black box, additional investments in data enrichment services often required Entity resolution and data stewardship limited or missing, crowd sourced support and application development Typical MDM pricing for SMB s $250 $500k for software only

9 MDM Project Planning Identify High Level Requirements Understand Business Processes Understand Political Climate Level of Mid & Senior Exec buy in Senior Executive Commitment Data Ownership and fiefdoms Understand Quality of Existing Data Incomplete Records Duplicate Records Number of Records & Sources of Data Identify Preferred Implementation Style Successful MDM projects require strong project leaders who are intune with the political and cultural environments of the organization

10 Common MDM Missteps Data cleanup efforts can get delegated out to the organization Leads to squabbling over data definitions, prolonged schedules, lack of ownership Improving data quality is a technology initiative Business process changes are required in any effective solution Lack of senior executive support and budget Bottoms up, grass roots MDM projects rarely succeed due to required process changes MDM projects can be executed piece meal over time A programmatic approach is required for success. Garbage in can be stopped Bad data will always exist but can be mitigated with effective systems and stewardship MDM or data cleanup solutions can be home grown Entity resolution is a complex process that typically exceeding most organization s capabilities Bad data can be taken care of later Repairing data after the fact costs 10x than it does to ensure it s accurate from day one

11 MDM Implementation Options Project Requirements, organizational culture, and data governance maturity level determine the optimal MDM system design, which can influence the vendor selection. Multiple MDM designs can exist in a single company. Numerous Data Sources? Registry or Virtual MDM Yes Inability to Enforce Data Quality Standards? Frequent Org Changes? No Federated MDM Autonomous Depts. or Divisions? No Current Data Dictionary? Less Data Governance Maturity More

12 Federated MDM Definition a single database of gold master data, typically deployed as a huband spoke architecture. The federated hub for a given business object is the single version of the truth for that object in the enterprise. Best For: Companies with a limited number of data sources, more mature IT and data governance, and centralized operational processes. Advantages: Typical Deployment Dependent System 1 Provides central locus of control Reduced complexity postimplementation 1. Extract data from source systems 2. Clean and de dupe data 3. Create master data hub Dependent System 4 Master Data Hub Dependent System 2 Audit and controls inherent to the system 4. Implement process changes to control creation of new records 5. Source systems defer to master hub Dependent System 3 Disadvantages Larger up front investment in planning and data cleansing Typically deployed as a big bang, higher risk of project failure Requires centralized administration & coordination to mitigate garbage in

13 Registry or Virtual MDM Definition the master record for a given entity is a virtual consolidation of one or more source records. The new virtual master record is linked to all source records and the meta data of those source records. Best For: Companies with immature data and IT governance, autonomous and decentralized operating units in multiple verticals, high change environments. Advantages: No forced process changes on organization Avoids up front galactic data cleanup Avoids internal issues of data ownership Disadvantages Complex technical implementation Requires transactional reliance between the spokes and the data hub Typical Deployment: Source 1 1.Deploy Virtual Hub 2.Link Matched Entities to Virtual Master 3.Integrate Operational Systems to Hub 4.Manage Ongoing Exceptions with Data Steward Source 4 Virtual Hub Source 2 Source 3

14 Your style of MDM will significantly impact your vendor selection and budget. Most companies with entrenched cloud applications, especially CRM s, lean towards Registry MDM s. Established sales operations processes Organizational impacts of significant processes changes Registry or Virtual MDM s require entity resolution and data stewardship, features which have historically only been available with enterprise class solutions Pervasive has solutions for both MDM styles at a fraction of the cost of enterprise solutions.

15 Federated MDM with Pervasive Master Data Hub Data is cleansed and de duped A single gold master hub of data is created by consolidating all records from all sources Creation & modification of records controlled by a centralized Admin/Data steward Master data exposed to all the necessary applications / processes in the organization

16 Registry MDM with Pervasive Virtual Master Data Hub A virtual gold master hub is created by consolidating all data. Rules determine which meta data populates the virtual master. New records matched to master records in hub Data Stewards manage exceptions Master data exposed to all the necessary applications / processes in the organization..

17 MDM Data Validation & Enrichment Through a partnership with MelissaData, Pervasive can implement the following processes to any MDM model Address Standardization Address Validation Validation Highly Recommended to run address standardization pre matching Web Services within Pervasive Data Integration enable external calls for third party data augmentation Contact Verification & augmentation (e.g. Jigsaw) Company verification and augmentation (e.g. D&B, InfoGroup, etc.) Implementing third party data is the most effective way to eliminate any possible in fighting over the definitions of accounts or contacts