Session 802. A Data Governance Journey of Getting Quality Data to Flow into Everyday Business

Transcription

1 Session 802 A Data Governance Journey of Getting Quality Data to Flow into Everyday Business Presented by: Veronica Kinsella, American Water, Enterprise Data Lead Kristin McMahon, SAP, EIM Solution Marketing

2 About the Session American Water s data management vision is to have quality data flow into everyday business. Veronica Kinsella, enterprise data lead at American Water, shares her approach and journey to better data. Veronica will discuss their data stewardship and governance model, what various roles and responsibilities look like, and where technology fits in both fulfilling the model and supporting the people along the way. Join us for a unique data journey and get inspired to start or re-focus your path to better data.

3 Agenda Introduction Who is American Water? Bad Water vs. Bad Data Our Initial Focus Executive Sponsor Policy and Practice What does Success Look Like? Data Organization Design Data Definitions Key Processes Technical Infrastructure Our Approach Approach: Address Immediate Needs/Show Value/Gain Acceptance Data Governance Model: Hybrid of centralized and federated Overview of Roles Day in the Life How it Works Other Uses Cases: Internal Audit, Process excellence, Data Requests

4 About Us

5 About American Water About American Water American Water provides high-quality water and wastewater services to approximately 14 million people in more than 30 states, as well as parts of Canada. Headquartered in Voorhees, NJ, we are the largest publicly traded water and wastewater utility company in the United States, and are the parent company to our state subsidiaries. We employ more than 6,700 people who give back to the community each day by doing their part to provide the highest quality service possible. Our professionals are committed to customer service, operational excellence and the delivery of highquality, reliable drinking water, safe and effective wastewater treatment and release and other water-related management services. Our teams live and work in the communities they serve.

6 We Deliver Quality Water Because: American Water delivers Quality Water Potential business impacts of Bad Water lead to: because: Increased risk for noncompliance of regulatory standards and laws Increased violations Increased costs scientists, lawyers, etc. additional tests and controls Increased time more oversight from regulatory agencies more time spent on problem solving, testing, etc Long term affects weakened employee, customer, and other stakeholder relationships

7 We Deliver Quality Data Because: American Water delivers Quality Data Potential business impacts of Bad Data lead to: because: Increased risk of faulty transactions in an operational system Inaccurate meter readings can cause billing errors and claim issues, which have negative downstream effect on budgeting and forecasting Inaccurate premise data can cause improper billing and inaccurate asset mgt Increased costs related to poor quality data Inaccurate customer data creates the possibility of billing exemptions resulting into loss of revenue for AW as well as for third party providers. Increases cost of poor decision making based on wrong information Increased risk for noncompliance of regulatory standards and laws Decreased master data quality Inability to achieve a single version of the truth or trusted system of record Decreased value of Business Intelligence, CIS, and other solutions Weakened employee, customer, and other stakeholder relationships

8 Bad Water & Bad Data Impact the Business Bad Water & Bad Data have similar business impacts Managing data as a corporate asset much like we treat water quality is a strategic enabler and plays a key role in the success & future of American Water.

9 Business Drivers Business Drivers Achieve clean and correct master and reference data Minimize the possibility of faulty transactions Reduce costs related to poor quality data Increase the value of BI and CRM solutions Improve the morale of employees and strengthen relationships between ITS and Business Be compliant with regulatory standards and laws

10 Data Governance Defined Metrics Executive Support Policies & Standards Processes Corporate View of Data People who Processes how Technology what Data Governance involves refocusing current efforts by proactively setting governance in place to ensure data is correct and consistent across the organization. This typically involves: Defining an enterprise data governance structure to oversee efforts in managing data as a corporate asset Defining roles to govern and own decisions about the data and business intelligence Defining operating processes to carry out policies and procedures Defining technology enablers to help support the Data governance organization Treating Data as a Corporate Asset = Governance

11 What is Data Management? What is Data Management? Data Management is the day-to-day coordination and enforcement of rules, policies, standards, and responsibilities that guide and enforce overall management of our data. American Water s Data Management Program vision is to have the data lifecycle baked into everyday business. This means key data elements are treated as a Corporate Asset.

12 Data As a Corporate Asset Managing data as an asset, and looking at it across people, process, and technology Data as a Corporate Asset is a cultural shift that will enable American Water to drive value. Master Data lifecycle processes (CRUD: create, read, update, delete) Data decision processes Data profiling, cleansing, enrichment and monitoring processes Data policies, and standards KPIs and measurement processes Process People Technology Data Owners Data Directors Data Managers Data Stewards Data Producers Data Consumers Data Management Technology and Tools Data Quality Applications System Integration & Technologies Workflow Capabilities

13 What is the 911 on Data Management Data Management (DM) addresses the source; the host/core systems where Data is created and stored Examples of our desired DM state are: There is a minimum level of poor data e.g., duplicates, violations of standards & naming conventions, etc. Errors & defects are easily and proactively detected and corrected Data is trusted to make management, strategic, and operational decisions How we are going to get there? Real-time upfront validations that prevent bad data from entering the system When upfront rules are not present and/or can not be written, create Monitoring Reports on the back side to ensure the timely correction of errors Scorecards to monitor Master Data Elements to measure the quality of our data Proactive data reviews and potential issue remediation Timely DM issue root cause analysis and solutions outlined by The Business

14 American Water s Policy & Practice American The Policy provides Water s requirements to Data support Policy the Company s and data governance Practice and management efforts. This effort promotes the development and maintenance of information that allows the company to make informed business and operational decisions. The Practice document outlines the activities for managing enterprise data as a corporate asset which aligns to the requirements of the Enterprise Data Policy. Benefits Ensures integrity, consistency and accuracy of the enterprise s data resources Minimizes the amount of rework, reduce data conflicts and increase data integrity Ensures that Data is being managed as a corporate asset

15 Enterprise Data Policy Overview Overview: Data is treated as a company asset. The policy and practice promote enterprise processes to ensure data is collected, used, reported, and protected consistently across the company. Scope: Electronic and Hard Copy Information Enterprise Data Policy Elements Enterprise Data Management Definition Responsibilities Enterprise Data Definition and Standards Data Quality Data Security Third Parties Strategic Objectives Key Points Enterprise data is owned and managed by business users Executive Leadership Team has ultimate responsibility. ITS does not own data. Overview of the individuals responsible for data management. Provides detailed responsibilities and tasks for the following positions: ELT, Senior Vice President of Business Services, Data Lead, Data Owner, Business Data Director, Business Data Manager, Data Steward. Data owners are responsible for categorizing and defining enterprise data through categories, technical characteristics and business rules. Data owners approve standards and business rules related to their respective data domains. Data owners are responsible for quality, data maintenance and monitoring. Data quality is managed through: Key performance indicators, business rules data governance and quality reports, data management tools, periodic data cleansing, escalation of data issues. Covers data usage, data access and sensitive information. Data is only to be used and accessed by the proper individuals. Defines additional third parties: Information technology, data producers, data consumers, communications. Enterprise data is a valuable company asset and data standards should be used company-wide. Impact on Roles and Responsibilities Describes the ELT ownership of data and how ITS is not responsible. Definition of a number of roles within the company. The major resource for identifying the work required. Data owners are responsible for categorizing and defining enterprise data. Data owners are responsible for data quality. N/A Defines the roles of IT, Data producers, data consumers and communications. N/A Business Impact / Change: High Key Changes Defined responsibility of data for ELT. Redesigned roles and responsibilities for the organization. Data standards and conventions are established. Key performance metrics and measurements are being utilized. Identify information sensitivity and usage Defines third party roles. Strategic view of data.

16 Data Organization Roles The data governance process will require a number of individuals to work together throughout the Data organization Organization to ensure that Roles data is being used correctly and managed as corporate asset. Steering Committee Data Lead (TBD) Data Owners Data Owners Data Owners (HTR) (RTR) (PTP) Data Owners Provide oversight of data governance across the company and manage data quality Data owners representing various domains form the Steering Committee under the leadership of the executive sponsor Data lead is responsible for the comprehensive data strategy, processes, tools and delivery Data Directors Data Managers Data Steward Supports the Data Governance and Quality Initiative Defines enterprise data and business rules; enforces Data Policies / Standards Executes the day-today management activities, processes and data quality 19

17 Roles Data Owner Definition: Executive-level individual from the business who supports the Data Governance initiative and has the final decision making authority within the data domains they are assigned. Accountable for the definitions, policies, practices and enforcement of enterprise business rules for the data within the data domains they are assigned. Define the level of quality required to satisfy business needs of data consumers across the enterprise. Define the data producer lifecycle management processes and procedures for creating, reading, updating and deleting (CRUD) data. Make decisions regarding data and process within the data domains they are assigned based on feedback from data directors.

18 Overall Stewardship Activities Review Activity and KPI reports Analyze variances and trending Correct errors where necessary Periodic quality checks of unmonitored data Look for opportunities for improvement Determine business rules Determine weighted levels for reporting Routine Data Quality Monitoring Issue Resolution End user encounters error Research the issue to determine the root cause Perform Root / Cause Analysis Provide business requirements Once root cause is identified, work with the business to determine the resolution and prevention Review project proposals and determine data impact, if any. Define the data requirements and approach; determine the cost/benefit of the project and resource needs Projects and Enhancements Other Routine Tasks Manage AW Glossary of Terms Field questions and serve as SME for Data Managers regarding errors, issues, etc Maintain Logs to validate ROI Conduct / Participate in Governance and other data related meetings

19 What is a Data Definition What is a Data Definition? Data Definition Defined A Data Definition is the collection of fields, metadata and business data standards which when considered in its entirety define a data object within a given system. Why do we need data definitions? Helps to facilitate information exchange both between people and systems through a common language Field Description Required Syntax Description Reduces data maintenance costs as a common set of criteria is maintained and followed Describes data in a consistent way within and across business processes Length Business Rules Active Data Type Data Type Helps to identify external data enrichment needs

20 Data Standards Rules (cont.) Business Rules: Business requirement to populate the field according to pre-defined rules (valid values, content, and/or structure) to fulfill business process, reporting or legal/statutory requirements. Industry Standards or Codes: Any industry standards or codes that may be applied to the object/field. (Ex: UNSPS code or D&B number). Legal/Regulatory: Legal or regulatory rules that apply to the data element. (ex: Vendor Name must be the name of the Legal Entity) Valid Values: Identification of the allowable values for a field. Note: If a drop down field, do not provide a complete listing of all values if all are valid (Ex: UOM field should only allow values CS, LB, EA, etc.) Number Ranges: Number ranges that may have been configured Usage: Identification of how/when the field should be used (e.g. This field should only be used in XYZ scenario ) Tip: Business Rules should be: Readable easy to understand by anyone reading them. Atomic can t be further broken down into different business rules.

21 Data Governance Processes Data Governance 8 Key Processes Root Cause Analysis New Project Enhancement Process Change Training Conflict Resolution Issue Escalation Glossary of Terms Objective: 1 standardized repeatable process for everyone No question as to who is responsible for what Insite into accountability Reduce time it takes to address issues / changes

22 Communications Campaign

23 DM Data Quality Solution Overview Data Quality Checks: Data Ownership Data Definitions Quality Levels Data Validity Consistency Allowable Values KPI s Dashboards Data Quality Categories Description Examples Completeness Accuracy (to Reality) Accuracy (to Point of Capture) Accuracy (Summary Data) Non-duplicate Records Is enough information available to make a decision? Is each "fact" complete? Do I have all the "facts" I need? Does the data match reality, at any given point in time? Have I preserved the information exactly as it came to me (e.g., for audit purposes), regardless of whether that information was "correct" or valid? Is aggregated data accurate? Can calculated values be "trusted", or do we typically seek to look at the raw data? Are there multiple records representing the same entity? Do all addresses have zip codes? Does the move process have access to credit limit information? Is the inventory of material accurate? Is any of the data defaulted to NULL at the point of capture? If average monthly sales for a division is calculated as yearly sales / 12, this result may not be meaningful if a large customer move happened in month 10. Is there a possibility of capturing the same customer or material twice?

24 Data Management Solution Data Management leverages, SAP EIM (Enterprise Information Management) & SAP BOBJ environments collaboratively. SAP EIM (Information Steward, Data Services) SAP BOBJ (Web Intelligence, CMC Administration)

25 Data Quality Solution Overview Leverage SAP Data Services at Point-Of- Entry, cleanse data as close to the source as possible Understand data problems by measuring & tracking data quality improvements Data Quality checks: Online or Real-time/Active Batch

26 Data Quality Real-time service platform Address verification

27 Focused on Immediate Needs Our initial approach at go live was to focus on the immediate needs and problems to help Focused show our value on and to Immediate gain acceptance throughout Needs the business. This was mostly around data quality We continue to work on our EAM organization as we work out issues

28 Data Stewards During Data the Stewards initial phases of our design we said that: We will need to choose Data Stewards carefully & have dedicated resources Business Knowledge and Position Subject matter expert for their area; understands broader implications Visible, respected, and influential with authority Enterprise perspective and political awareness Technical Skills Solid understanding of the data used within their area and the business processes that impact that data and its use Should understand quality improvement concepts including data profiling, root cause analysis, and continuous improvement techniques Interpersonal Skills Team player, excellent people and communication skills; diplomatic but forceful Skilled facilitator with ability to think outside the box

29 Realization of the Data Steward Role Realization of the Data Steward Role During ERP Implementation we learned: The technical skills to profile data combined with the business process knowledge to perform root cause analysis, were not widely available Therefore, in the ERP Phase, a group of Expert Data Stewards were identified to support the Functional Data Stewards. And we would have NO NEW FTE s.

30 Data Steward: Expert vs. Functional Expert Data Steward v. Functional Data Steward High Level Expert Data Steward Creates Projects and Scorecards Liaises between functional data stewards, business process teams, reporting analysts and IT to implement technical and process changes Holistic View of Enterprise Data Works with Data Stewards to identify opportunities for improvement and monitoring Highly skilled using Analysis tools Functional Data Steward Runs routine Date Quality Reports Coordinates Data Cleansing with Local Stewards Reinforce data standards, policies, and compliance within the functional area Provides support to Functional Business Data Owners, Business Directors, and Super Users Works with Expert Data Stewards to identify opportunities for improvement and monitoring

31 Data Organization Roles Data Organization Roles Data Directors Visibly Supports the Data Governance and Quality Initiative Data Managers Defines enterprise data and business rules; enforces Data Policies / Standards Data Lead Expert Data Stewards Responsible for the comprehensive and overall data strategy, processes, tools and delivery Data Solution Business Expert, submits proposals, coordinates activities with the Functional Data Stewards per tower, liaises with business teams, COE, & IT to prevent and resolve issues Functional Data Stewards Local Data Stewards Executes day-to-day DM activities, processes and quality monitoring at the Enterprise Level for their area of expertise. Supports and brings forward issues for local stewards Executes day-to-day management activities, processes and quality monitoring at the State/Local level for their area of expertise.

32 Data Organization Data Owner CIS Vice President Customer Service Data Owners understand data governance policy & see that policy & practices are being followed. They make decisions on data security/access Enterprise Data Lead Data Management Directors, Managers and Stewards monitor the quality of data and conformance to business rules according policy & practices and correct errors in data

33 Day in the Life Day in the Life How tickets come in, and how technology supports the people

34 Data Quality Program We have the tools We have the people We need to improve the health of the system

35 Initial Meeting Schedule/Cadence Groups Initial Sr. Leadership Meeting (Hobbs, Data OwnersSchedule Data Data / Expert Cadence Functional Neafsey, Bigelow) Directors Managers Stewards Data Stewards Local Regional Stewards Local State Stewards All Data Team Meeting: (every 2 months) Initially, Monthly; may Data Governance go to Every 2 Months. Updates, New Projects, Hot Topics within Towers, Lessons Learned, Successes, etc. Primary Communicators Meetings Meetings Meetings Tara Krause w/ contribution from Data Lead Initially, Monthly; may go to Every 2 Months. Tower Meeting: (monthly) Status Report including Metrics, Issues, Proposed Solutions, Training Needs, Support Needs, etc. Expert Stewards w/contribution from Data Lead ALL Data Team Management Meeting - Every 2 Months, 2nd 2 pm EST - Slot 2 hrs RTR Functional Large Group - Report out / Communication - Monthly - 1 hr HTR Functional Large Group - Report out / Communication - Monthly - 1 hr PTP Functional Large Group - Report out / Communication - Monthly - 1 hr CIS Functional Large Group - Report out / Communication - Monthly - 1 hr EAM Functional Large Group - Report out / Communication - Monthly - 1 hr RTR Functional Small Group - Report out / Brainstorming - Every 2 HTR Weeks, Functional 1 hr Small Group - Report out / Brainstorming - Every 2 PTP Weeks, Functional 1 hr Small Group - Report out / Brainstorming - Every 2 CIS Weeks, Functional 1 hr Small Group - Report out / Brainstorming - Every 2 Weeks, EAM Functional 1 hr Small Group - Report out / Brainstorming - Every 2 Weeks, 1 hr Expert / Functional Steward working sessions per tower: (every two weeks) Review of new issues, status updates, collaboration on solutions, knowledge sharing, etc.

36 Issue Reporting Process - Detailed 39

37 Primary Issue Remediation Tools for DQ Functional and Expert Stewards will work together to resolve data issues. Below are six of the most common resolution methods. Often, more than one is deployed to address both the short and long term resolution. Training Process Change Temporary Workaround Routine monitoring through Business Rule Scorecard Data Cleansing Manual or Automated Real-Time Data Validations Typically an Enhancement or Project

38 SAP Information Steward SAP Information Steward SAP INFORMATION STEWARD

39 SAP Information Steward: Data Insight tab SAP Information Steward provides Business Users and the Data Management Team with a single environment to assess, Steward define, monitor, (data and improve insight overall data quality tab) Example: Connection Object Scorecard A connection object in SAP is usually a building or it can also be a piece of property or facility There are two views: Workspace = Data Stewards perform tasks such as profile and analyze data, define rules, set up scorecards. Scorecard = Visual way for Data Stewards, Analysts, and Management to easily understand and analyze data quality from a table/domain perspective.

40 Data Quality Monitoring Report CIS Connection Object The connection object is a piece of property or a structure to which service is delivered Report Purpose: Identifies when any of the below Connection Object information is changed or doesn t meet the business rule. * City * Street * Region * Country * Tax Jurisdiction * PWSID * House Number * Time Zone * Postal Code Business Partner Premise Contract 1: water Contract Account Contract 2: sewer Connection Object Apartment 1 Installation 1: Water Installation 2: Sewer Apartment 2 Apartment 3 Device Basement Details: Failed data results display SAP Username, location and date/timestamp Distribution Network Connection Device Location 1 Frequency: Weekly Legend: Business Master Data Technical Master Data

41 Step 1 - Insight into potential issues Report out when address is not standard, which gives insight into when users are not taking the USPS suggestion Report out when PWSID is modified Report out when County Code is modified or blank Report out when the Tax Jurisdiction Code has the letter 'X' repeated either 4 times or 9 times, as indicated with the examples of 'USNJXXXXXXXXX0' or 'USIN99227XXXX0 (US + state + 9 digit zip) Expert Stewards Create scorecards and does initial analysis

42 Data Quality Report Short Term Functional Stewards reviews reports and manually cleans up data

43 Data Quality Solutions When ordering materials, someone ordered 5 gallons of sand. Gallons is not a valid unit of measure for sand. How can we prevent this from occurring? Plant and profit centers must be in the same state for materials management. How can we ensure proper set up? SAP-ECC is allowing Credit refunds in Euros and Canadian denominations - should only be US$ for regulated company. How can we prevent this from occurring? Short-Term List out acceptable unit of measures per material type Provide training hand-outs to users Build report to monitor exceptions to the standard Build report to monitor exceptions Build report to monitor exceptions Long-Term Real-time cross validation between material types and unit of measure Issue occurs rarely Cost>Benefit for real-time validation Continue to monitor report for exceptions SAP configuration change

44 SAP Data Services Real-time Service Platform Address verification through US National postal directory Upon input, a real time validation will occur to analyze, verify and match with a valid record with the US National postal directory. If input is not accurate, the real time validation will alert the user. The user would be able to use the original address or accept the validated address. Benefits Assures contact information is correct Reduce duplicate records, returned mail and address correction fees Identify vacant addresses Append missing Suite or Apartment numbers

45 SAP Data Services Real-time Validation Validation that certain date information is provided when entering new employee information All employees must have an AW Hire Date and Original Hire Date Active monitoring will prevent potential payroll, 401K, and pension issues Best Practice: Data Quality at Point-Of-Entry Business Rule Every Employee MUST have data types Z1 and Z5

46 SAP Data Services Real-time Validation The connection object is the highest level in the Technical Master Data SAP Data Services Real-Time Validation Hierarchy. It represents a piece of property or a structure to which service is Connection delivered. Object Real-Time Validation prevents duplicate Connection Objects from entering the system

47 Final Thoughts

48 Key Points to Take Home 1. Data will now be treated as a strategic corporate asset Key Points to Take Home 2. The business will now be empowered to manage their data 3. Data quality is everyone s responsibility 4. Use the tools to support the needs 5. Data Quality should be as close to the point of entry as possible 6. This is journey not a destination 7. You can still make big impacts with minimum resources

49 How do we Ensure the Quality of the Data?

50 Shift in Focus: Information Management What data? Is the data internal or external? What type and format is the data in? What is the known value of the data? Is the data structured in EDW or logs in Hadoop? Where is the data stored? Is it in-memory, analytical and/or transactional store, or a Hadoop distributed file? Where type of analysis is required? Is it reporting and dashboards, predictive or visualizations, or text analysis? Where do we do the analysis? Do we analyze at the source? Do we extract and move relevant data to structured store? How do we ensure data quality? Is the data complete and accurate? How can we enrich the data?

51 Understand Business Impacts of Bad Data! Calculate the costs of Data Quality issues on the Business Determine Financial ROI of your data quality and information governance initiatives! Understand how bad data affects business Identify potential savings using What-If analysis of quality level and costs Track metrics of financial impact per failure Costs presented as part of DQ Scorecard

52 SAP EIM e-book Available on itunes and as a PDF file 2014

53 Submit your Information Governance project for the 3rd Annual IGgie Award Presented at the ASUG Data Governance SIG Conference 17 September 2014, Princeton, New Jersey For more details, check the ASUG DG SIG website: