Peter Aiken. Measuring Data Management Practice Maturity

Transcription

1 Measuring Management Practice Maturity Increasing data management practice maturity levels can positively impact the coordination of data flow among organizations, individuals and systems 1 Peter Aiken BS VCU 1981 Information Systems & Management MS VCU 1985 Information Systems PhD GMU 1989 Information Technology Engineering Full time in information technology since 1981 IT engineering research and project background University teaching experience since 1979 Seven books and dozens of articles Research Areas reengineering, data reverse engineering, software requirements engineering, information engineering, human-computer interaction, systems integration/ systems engineering, strategic planning, and DSS/BI Director George Mason University/Hypermedia Laboratory ( ) DoD Computer Scientist Reverse Engineering Program Manager/Office of the Chief Information Officer ( ) Visiting Scientist Software Engineering Institute/Carnegie Mellon University ( ) DAMA International President ( DAMA International Individual Achievement Award (with Dr. E. F. "Ted" Codd) 2005 DAMA Community Award Founding Advisor/International Association for Information and Quality ( Founding Advisor/Meta-data Professionals Organization ( Founding Director Blueprint

2 International Organizations 10% Local Government 4% Public Companies 58% State Government Agencies 17% DM Maturity Organizations Surveyed Federal Government 11% Results from more than 500 organizations 32% government Appropriate public company representation Enough data to demonstrate European organization DM practices are generally more mature 3 IT Project Failure Rates Recent IT project failure rates statistics can be summarized as follows: Carr % of IT Projects completed on time, within budget, with full functionality OASIG Study (1995) 7 out of 10 IT projects "fail" in some respect The Chaos Report (1995) 75% blew their schedules by 30% or more 31% of projects will be canceled before they ever get completed 53% of projects will cost over 189% of their original estimates 16% for projects are completed on-time and on-budget KPMG Canada Survey (1997) 61% of IT projects were deemed to have failed Conference Board Survey (2001) Only 1 in 3 large IT project customers were very satisfied" Robbins-Gioia Survey (2001) 51% of respondents viewed their large IT implementation project as unsuccessful MacDonalds Innovate (2002) Automate fast food network from fry temperature to # of burgers sold-$180m USD writeoff Ford Everest (2004) 4 Replacing internal purchasing systems-$200 million over budget FBI (2005) Blew $170M USD on suspected terrorist database-"start over from scratch" (accessed 9/14/02) New York Times 1/22/05 pa31

3 DM Origins Which arrives first DM or DBMS? % 68% 75% DM 1st 9% DBMS 1st Simultaneously 6% A Key Indicator 70% reacting instead of anticipating Best practices are obvious 6% 5 DM Involvement Initiative Leader Initiative Involvement Not Involved Warehousing XML Quality Customer Relationship Management Master Management Customer Integration Enterprise Resource Planning Enterprise Application Integration Particpation Percentage 6

4 Why Projects Fail by Joseph R. Hudicka Assessed 1200 migration projects! Surveyed only experienced migration specialists who have done at least four migration projects Median Project Expense Median Project Cost The median project costs over 10 times the amount planned! Biggest Challenges: Bad ; Missing ; Duplicate The survey did not consider projects that were cancelled largely due to data migration difficulties " problems are encountered rather than discovered" $0 $125,000 $250,000 $375,000 $500,000 7 Joseph R. Hudicka "Why ETL and Migration Projects Fail" Oracle Developers Technical Users Group Journal June 2005 pp Monitization: Legacy System Migration to ERP Challenge Millions of NSN/SKUs Key and other data stored in clear text/comment fields Original suggestion was manual approach to text extraction Left structuring problem unsolved Solution Proprietary, improvable text extraction process Converted non-tabular data into tabular data Saved a minimum of $5 million Literally person centuries of work 8

5 An Iterative Approach to Quality Engineering Unmatched Items Unmatched Items Ignorable Ignorable Items Avg Extracted Items Matched Rev # (% Total) NSNs (% Total) Items Matched Per Item (% Total) Items Extracted % % N/A N/A N/A % % N/A N/A N/A % % N/A N/A N/A % % % % % % % % % % % % % % % % % % Quantitative Benefits Time needed to review all NSNs once over the life of the project: NSNs 2,000,000 Average time to review & cleanse (in minutes) 5 Total Time (in minutes) 10,000,000 Time available per resource over a one year period of time: Work weeks in a year 48 Work days in a week 5 Work hours in a day 7.5 Work minutes in a day 450 Total Work minutes/year 108,000 Person years required to cleanse each NSN once prior to migration: Minutes needed 10,000,000 Minutes available person/year 108,000 Total Person-Years 92.6 Resource Cost to cleanse NSN's prior to migration: Avg Salary for SME year (not including overhead) $60, Projected Years Required to Cleanse/Total DLA Person Year 93 Saved Total Cost to Cleanse/Total DLA Savings to Cleanse NSN's: $5.5 million 10

6 Misunderstanding Management 11 Expanding DM Scope Base Administration (DBA)! base design base operation requirements analysis modeling Administration (DA)! Organization-wide DM coordination Organization-wide data integration stewardship, use Enterprise Administration (EDA)! Management (DM)! Governance, Quality, Security, Analytics, Compliance, Mashups, Business Rules (more...) 12

7 Enterprise Information Management is concerned with Architecture 13 He who doesn t lay his foundations before hand, may by great abilities do so afterward, although with great trouble to the architect and danger to the building. 14 Nicolo Machiavelli ( ) Machiavelli, Niccolo. The Prince. 19 Mar

8 Building from the Top 15 Look Familiar? 16

9 Motivation "One day Alice came to a fork in the road and saw a Cheshire cat in a tree. Which road do I take? she asked. Where do you want to go? was his response. I don't know, Alice answered. Then, said the cat, it doesn't matter." "We want to move our data management program to the next level" Question: What level are you at now? You are currently managing your data, But, if you can't measure it, How can you manage it effectively? Lewis Carroll from Alice in Wonderland How do you know where to put time, money, and energy so that data management best supports the mission? 17 Management Program Coordination Organizational Strategies Goals Implementation Guidance Organizational Integration Integrated Models Stewardship Standard Development Application Models & Designs Feedback Direction Support Operations Business Asset Use Business Value 18

10 Management Manage data coherently. Program Coordination Organizational Integration Share data across boundaries. Stewardship Development Assign responsibilities for data. Engineer data delivery systems. Maintain data availability. Support Operations!! Management Capability Maturity Model Levels We have a process for improving our DM capabilities Optimizing (5) We manage our DM processes so that the whole organization can follow our standard DM guidance We have experience that we have standardized so that all in the organization can follow it Repeatable (2) Defined (3) Managed (4) One concept for process improvement, others include: Norton Stage Theory TQM TQdM TDQM ISO 9000 and focus on understanding current processes and determining where to make improvements. We have DM experience and have the ability to implement disciplined processes Initial (1) Our DM practices are ad hoc and dependent upon "heroes" and heroic efforts 20

11 Key Finding: Process Frameworks are not Created Equal With the exception of CMM and ITIL, use of process-efficiency frameworks does not predict higher on-budget project delivery Percentage of Projects on Budget By Process Framework Adoption while the same pattern generally holds true for on-time performance Percentage of Projects on Time By Process Framework Adoption Source: Applications Executive Council, Applications Budget, Spend, and Performance Benchmarks: 2005 Member Survey Results, Washington D.C.: Corporate Executive Board 2006, p Assessment Components Management Practice Areas program coordination Organizational data integration stewardship development support DM is practiced as a coherent and coordinated set of activities Delivery of data is support of organizational objectives the currency of DM Designating specific individuals caretakers for certain data Efficient delivery of data via appropriate channels Ensuring reliable access to data Capability Maturity Model Levels Examples of practice maturity 1 Initial Our DM practices are ad hoc and dependent upon "heroes" and heroic efforts 2 - Repeatable We have DM experience and have the ability to implement disciplined processes 3 - Documented We have standardized DM practices so that all in the organization can perform it with uniform quality 4 - Managed We manage our DM processes so that the whole organization can follow our standard DM guidance 5 - Optimizing We have a process for improving our DM capabilities 4

12 Weakest Link Results Reporting Results Understand five organizational data management practice areas Rate each area per capability maturity model Understand the "weakest link" nature of the results reporting Engineered components can only be as strong as their weakest component Low scores seem harsh but are realistic (and on the upside) easily improvable A single "1" degrades the entire practice area as shown with "stewardship" DMPA results are granularized for each practice area providing improvement process guidance 23 Assessment Components Management Practice Areas program coordination Organizational data integration stewardship development support DM is practiced as a coherent and coordinated set of activities Delivery of data is support of organizational objectives the currency of DM Designating specific individuals caretakers for certain data Efficient delivery of data via appropriate channels Ensuring reliable access to data Capability Maturity Model Levels Examples of practice maturity 1 Initial Our DM practices are ad hoc and dependent upon "heroes" and heroic efforts 2 - Repeatable We have DM experience and have the ability to implement disciplined processes 3 - Documented We have standardized DM practices so that all in the organization can perform it with uniform quality 4 - Managed We manage our DM processes so that the whole organization can follow our standard DM guidance 5 - Optimizing We have a process for improving our DM capabilities 4

13 The challenge ahead 5.00 The chart represents the average scores presented on the previous slide - interesting that none have apparently reached level CMU's Software Engineering Institute (SEI) Collaboration Results from hundreds organizations in various industries including: Public Companies State Government Agencies Federal Government International Organizations Defined industry standard Steps toward defining data management "state of the practice" Management Practices Measurement (DMPA) Initial (I) Repeatable (II) Documented (III) Managed (IV) Optimizing (V) Program Coordination Organizational Integration Focus: Guidance and Facilitation Stewardship Development Support Operations Focus: Implementation and Access 26

14 Interpreting Assessment Results for a Sample Organization Perceptions are higher than actual practice Perceptions are below actual practice 0 Development Guidance Adminstration Support Systems Asset Recovery Capability Development Training Verified Average 27 Comparative Assessment Results Program Coordination Challenge Organizational Integration Challenge Stewardship Development Support Operations Challenge Nokia Client Industry Competition All Respondents 28

15 High Marks for IFC s Program Mgmt Audit Leadership & Guidance Asset Creation Metadata Management Quality Assurance "These IFC scores represent the highest aggregate scores in the area of data stewardship recorded in our database of hundreds of assessments that has been recognized as as a representative scientific sample." Change Management Quality Page TRE ISG IFC Industry Benchmarks Overall Benchmarks Dashboard Implementation 30

16 Why is our organizational Stewardship score so low? 31 What expertise do we have in Program Coordination? 32

17 Contact Information: Peter Aiken, Ph.D. Department of Information Systems School of Business Virginia Commonwealth University 1015 Floyd Avenue - Room 4170 Richmond, Virginia Blueprint Maggie L. Walker Business & Technology Center 501 East Franklin Street Richmond, VA office : cell: [email protected] datablueprint.com 3/3/2010 Copyright 12/18/07 Copyright by this and Blueprint previous - all years rights by reserved! Blueprint - all rights reserved!