Peter Aiken. The Evolution of Data Management (a 25+ year study)

Transcription

1 The Evolution of Data Management (a 25+ year study) TRA- Copyright 11/11/08 and prior years by Data Blueprint - all rights reserved! Peter Aiken Full time in information technology since 1981 IT engineering research and project background University teaching experience since 1979 Seven books and dozens of articles Research Areas reengineering, data reverse engineering, software requirements engineering, information engineering, humancomputer interaction, systems integration/systems engineering, strategic planning, and DSS/BI Director George Mason University/Hypermedia Laboratory ( ) Published Papers Communications of the ACM, IBM Systems Journal, InformationWEEK, Information & Management, Information Resources Management Journal, Hypermedia, Information Systems Management, Journal of Computer Information Systems and IEEE Computer & Software DoD Computer Scientist Reverse Engineering Program Manager/Office of the Chief Information Officer ( ) Visiting Scientist Software Engineering Institute/Carnegie Mellon University ( ) DAMA International Advisor/Board Member ( DAMA International Individual Achievement Award (with Dr. E. F. "Ted" Codd) 2005 DAMA Community Award Founding Advisor/International Association for Information and Data Quality ( Founding Advisor/Meta-data Professionals Organization ( Founding Director Data Blueprint

2 Dogs New Clothes 3 Contact Information: Peter Aiken, Ph.D. Department of Information Systems School of Business Virginia Commonwealth University 1015 Floyd Avenue - Room 4170 Richmond, Virginia Data Blueprint Maggie L. Walker Business & Technology Center 501 East Franklin Street Richmond, VA office : cell: [email protected] 4

3 5 September 21,

4 Hmm Confusion Correct Name: Yusuf Islam TSA No Fly Listing: Youssouf Islam 7 15,000 people 15,000 want off the US terror watch appealed list to be removed from list 2,000 month requesting removal TSA promised 30 day review process Actual time is 44 days American Civil Liberties Union estimates 1 million people on US government watch lists 8

5 US Terror Watch List Facts Fall 2008 comments: Fewer than 2,500 people on US "no-fly" list 10% those are US citizens 16,000 people on "selectee" list (additional screening) Transfer responsibility of comparing names on lists from dozens of airlines to TSA 9 IT Project Failure Rates Recent IT project failure rates statistics can be summarized as follows: Carr % of IT Projects completed on time, within budget, with full functionality OASIG Study (1995) 7 out of 10 IT projects "fail" in some respect The Chaos Report (1995) 75% blew their schedules by 30% or more 31% of projects will be canceled before they ever get completed 53% of projects will cost over 189% of their original estimates 16% for projects are completed on-time and on-budget KPMG Canada Survey (1997) 61% of IT projects were deemed to have failed Conference Board Survey (2001) Only 1 in 3 large IT project customers were very satisfied" Robbins-Gioia Survey (2001) 51% of respondents viewed their large IT implementation project as unsuccessful MacDonalds Innovate (2002) Automate fast food network from fry temperature to # of burgers sold-$180m USD writeoff Ford Everest (2004) 10 Replacing internal purchasing systems-$200 million over budget FBI (2005) Blew $170M USD on suspected terrorist database-"start over from scratch" (accessed 9/14/02) New York Times 1/22/05 pa31

6 Data Integration/Exchange Challenges Customer typically has had different meanings to different parts of the organization: Accounting -> organization that buys products or services Service -> client Sales -> prospect Assigning the same mission to the DoD lines of business to: Secure the building elicits very different results from each line of business : Army: Posts guards at all entrances and ensures no unauthorized access Navy: Turns out all the lights, locks up, and leaves Marines: Sends in a company to clear the building room-by-room; forms perimeter defense around the building Air Force: Signs three year lease with option to buy [Second example courtesy of Burt Parker] 11 FBI & Canadian Social Security Gender Codes 1. Male 2. Female 3. Formerly male now female 4. Formerly female now male 5. Uncertain 6. Won't tell 7. Doesn't know 8. Male soon to be female 9. Female soon to be male Hypothesized extensions contributed by a Chicago DAMA Member 10. Both soon to be female 11. Both soon to be male 12. Psychologically female, biologically male 13. Psychologically male, biologically female If column 1 in source = "m" then set value of target data to "male" else set value of target data to "female" 12

7 Predicting Engineering Problem Characteristics Legacy System #1: Payroll Platform: Amdahl OS: MVS 1998 Age: 15 Data Structure: VSAM/virtual database tables Physical Records: 780,000 Logical Records: 60,000 Relationships: 64 Entities: 4/350 Attributes: 683 Platform: UniSys OS: OS 1998 Age: 21 Data Structure: DMS (Network) Physical Records: 4,950,000 Logical Records: 250,000 Relationships: 62 Entities: 57 Attributes: 1478 Legacy System #2: Personnel Characteristics Logical Physical Platform: WinTel Records: 250, ,000 OS: Win'95 Relationships: 1,034 1, Age: new Entities: 1,600 2,706 Data Structure: Client/Sever RDBMS Attributes: 15,000 7,073 New System 13 "Extreme" Data Engineering 2 person months = 40 person days 2,000 attributes mapped onto 15,000 2,000/40 person days = 50 attributes per person day or 50 attributes/8 hour = 6.25 attributes/hour and 15,000/40 person days = 375 attributes per person day or 375 attributes/8 hours = attributes/hour Locate, identify, understand, map, transform, document, QA at a rate of - 52 attributes every 60 minutes or.86 attributes/minute! 14

8 Why Data Projects Fail by Joseph R. Hudicka Median Project Expense Assessed 1200 migration projects! Surveyed only experienced migration specialists who have Median Project Cost done at least four migration projects The median project costs over 10 times the amount planned! Biggest Challenges: Bad Data; Missing Data; Duplicate Data The survey did not consider projects that were cancelled largely due to data migration difficulties " problems are encountered rather than discovered" $0 $125,000 $250,000 $375,000 $500,000 Joseph R. Hudicka "Why ETL and Data Migration Projects Fail" Oracle Developers Technical Users Group Journal June 2005 pp Link business objectives to technical capabilities 16

9 Data Management "Understanding the current and future data needs of an enterprise and making that data effective and efficient in supporting business activities" Aiken, P, Allen, M. D., Parker, B., Mattia, A., "Measuring Data Management's Maturity: A Community's Self-Assessment" IEEE Computer (research feature April 2007) 17 A Model Specifying Relationships Among Important Terms Data Intelligence Wisdom & knowledge are often used synonymously Data Information Use Data Data Data Request Fact Data Meaning Data 1. Each FACT combines with one or more MEANINGS. 2. Each specific FACT and MEANING combination is referred to as a DATUM. 3. An INFORMATION is one or more DATA that are returned in response to a specific REQUEST. 4. INFORMATION REUSE is enabled when one FACT is combined with more than one MEANING. 5. INTELLIGENCE is INFORMATION associated with its USES. 18 [Built on definition by Dan Appleton 1983]

10 Do you know the game Twister? Canada Chile Columbia Egypt Estonia Finland France Germany Great Britain Ireland Italy Japan Qatar Scotland Switzerland Thailand Turkey UAE US 19 Typical System Evolution Finance Application (3rd GL, batch system, no source) Payroll Data (database) Payroll Application (3rd GL) Finance Data (indexed) Marketing Data (external database) Marketing Application (4rd GL, query facilities, no reporting, very large) Personnel Data (database) 20 R & D Data (raw) Personnel App. (20 years old, un-normalized data) R& D Applications (researcher supported, no documentation) Mfg. Data (home grown database) Mfg. Applications (contractor supported)

11 Nicolo Machiavelli ( ) 21 He who doesn t lay his foundations before hand, may by great abilities do so afterward, although with great trouble to the architect and danger to the building. Machiavelli, Niccolo. The Prince. 19 Mar Information Architectures are plans, guiding the transformation of strategic organizational information needs into specific information systems development projects Source: Internet "Information architecture is a foundation discipline describing the theory, principles, guidelines, standards, conventions, and factors for managing information as a resource. It produces drawings, charts, plans, documents, designs, blueprints, and templates, helping everyone make efficient, effective, productive and innovative use of all types of information." Source: Information First by Roger & Elaine Evernden, 2003 ISBN p. 1. Information architecture (IA) is the art of expressing a model or concept of information used in activities that require explicit details of complex systems. (wikipedia.org) All organizations have information architectures Some are better understood and documented (and therefore more useful to the organization) than others. 22

12 Building from the Top 23 Sample Conversation (Developing Constraints) I'd like to build a building. What kind of building - do you want to sleep in it? Eat in it? Work in it? I'd like to sleep in it. Oh, you want to build a house? Yes, I'd like a house. How large a house do you have in mind? Well, my lot size is 100 feet by 300 feet. Then you want a house about 50 feet by 100 feet. Yes, that's about right. How many bedrooms do you need? Well, I have two children, so I'd like three bedrooms... 24

13 GAO Has Identified the Problem 25 Concrete Block & Engineering Continuity 26

14 Look Familiar? 27 Why? 28

15 Finance Example Business Rule: A customer may have one and only one account Bank Manager: The customer is always right... And this one needs multiple accounts! # Account ID 1 peter 2 peter1 3 peter2 4 peter3 5 peter4 6 peter5 7 peter6 8 peter7 9 peter8 10 peter9 11 peter10 Sorted IDs peter peter1 peter10 peter2 peter3 peter4 peter5 peter6 peter7 peter8 peter9 29 Architecture Jargon 30

16 Avoiding Unnecessary Work Using Business Rule Metadata Person BR1) Zero, one, or more EMPLOYEES can be associated with one PERSON Job Class 'Mond-Licht' or 'Mondschein' BR2) Zero, one, or more EMPLOYEES can be associated with one JOB CLASS; BR4) One or more POSITIONS can be associated with one JOB CLASS. Employee Job Sharing Position BR3) Zero, one, or more EMPLOYEES can be associated with one POSITION 31 Student System Data Model 32

17 Proposed Data Model 33 Organizations Surveyed International Organizations 10% Local Government 4% Public Companies 58% State Government Agencies 17% Federal Government 11% Results from more than 400 organizations 32% government Appropriate public company representation Enough data to demonstrate European organization DM practices are generally more mature 34

18 Return! 0 70% Investment <= Return 10% Investment > Return 20% Largely Ineffective DM Investments 35 Approximately, 10% percent of organizations achieve parity and (potential positive returns) on their DM investments. Only 30% of DM investments achieve tangible returns at all. Seventy percent of organizations have very small or no tangible return on their DM investments. Misunderstanding Data Management 36

19 Expanding Scope Approximately Database Administration Database design, operation, monitoring, troubleshooting, etc. Data Administration Data as a strategic resource Requirements analysis/modeling Enterprise-wide Data coordination, integration, stewardship Mandatory Data quality security, privacy, compliance Optional extensions Mashups, 37 DM Origins Which arrives first DM or DBMS? DM 1st DBMS 1st A Key Indicator 70% reacting instead of anticipating Best practices are obvious Simultaneously 38

20 1.00 DM Group Longevity/Maturity 0.75 Measured Estimated Potential mis-labeling information or data management," "information or data services," "enterprise data architecture," and "data administration" Unexplained dip and rise in popularity Shift operations to a much broader range of activities 39 Non-Relational Database Processing 0.05 Percentage of Processing Mission-Critical % 11-20% 21-30% 31-40% 41-50% 51-60% 61-70% 71-80% 81-90% % 68% using hierarchical (typically IMS or Adabase) 20% reporting operational network DBMS "the rumors of the demise of non-relational processing are greatly exaggerated" (from Mark Twain) Virtually no textbook education 40

21 Backup/Recovery Installs New Release Performance Tuning Environmental Controls Database Design Security and Privacy Liaison to Programmers Liaison to Systems Analysts Database Auditing Data Communications Administration Input to Data Dictionary Manages Data Dictionary Education Long Range Planning Liaison to End Users Application Design Future Current DM Responsibilities Biggest perceived increase "Application Design" "Long Range Planning" "Liaison To End Users" Biggest perceived Decrease "environment control" "backup/recovery" "installing new releases" 41 Total FTEs in Data Management

22 DM Organization Footprint 26.25% 2007 DM Footprint size 21.00% Percentage 15.75% 10.50% 5.25% 1981 DM Group Size Small 5 Average 6-7 Large 5-8 Quite large 20's - 30s Very large % Number DM Group Size 43 Data Management Manager s Last Position 1981 systems analyst and programming project manager or leader 2007 database administrator data administrator programmer project manager application manager systems architect data security administrator systems analyst, development manager business line manager 44

23 Data Dictionary Usage Aligning Strategy & Execution Strongly Disagree Somewhat Disagree Neither Agree or Disagree Somewhat Agree Strongly Agree

24 DM Involvement Initiative Leader Initiative Involvement Not Involved Data Warehousing XML Data Quality Customer Relationship Management Master Data Management Customer Data Integration Enterprise Resource Planning Enterprise Application Integration Particpation Percentage Formal or Structured Approach to IQ? Yes 24% No 52% They say they are but they aren't 24% 48

25 % of DM organizations labeled "successful" Successful Partial Success Don't know/too soon to tell Unsuccessful In 25 years: 49 Does not exist Best Practices 50

26 51 Cruiser Collector 52

27 Capability Maturity Model Levels 1996 Council of American Building Officials (COBE) and the 2000 International Code Council recommendations call for unit runs to be not less than 10 inches and unit rises not more than 7! inches. We manage our DM processes so that the whole organization can follow our standard DM guidance We have experience that we have standardized so that all in the organization can follow it Repeatable (2) We have a process for improving our DM capabilities Defined (3) Managed (4) Optimizing (5) We have DM experience and have the ability to implement disciplined processes One concept for process improvement, others include: Norton Stage Theory TQM TQdM TDQM ISO 9000 and focus on understanding current processes and determining where improvements can be made. Initial (1) Our DM practices are ad hoc and dependent upon "heroes" 53 Capability Im-Maturity Model Level Description 0 Negligent/Indifference: Failure to allow successful development process to proceed. All problems are perceived to be technical problems. Managerial and quality assurance activities are deemed to be overhead and superfluous to the development process. Reliance on silver pellets. -1 Obstructive/Counter Productive: Counterproductive processes are imposed. Process are rigidly defined and adherence to the form is stressed. Ritualistic ceremonies abound. Collective management precludes assigning responsibility. Status quo über alles. -2 Contemptuous/Arrogance: Disregard for good software engineering institutionalized. Complete schism between software development activities and software process improvement activities. Complete lack of a training program. -3 Undermining/Sabotage: Total neglect of own charter, conscious discrediting of peer organizations software process improvement efforts. Rewarding failure and poor performance. 54

28 Key Finding: Process Frameworks are not Created Equal With the exception of CMM and ITIL, use of process-efficiency frameworks does not predict higher on-budget project delivery Percentage of Projects on Budget By Process Framework Adoption while the same pattern generally holds true for on-time performance Percentage of Projects on Time By Process Framework Adoption Source: Applications Executive Council, Applications Budget, Spend, and Performance Benchmarks: 2005 Member Survey Results, Washington D.C.: Corporate Executive Board 2006, p Organizational DM Functions and their Inter-relationships Organizational Strategies Implementation Data Program Coordination Goals Guidance Organizational Data Integration Integrated Models Data Stewardship Standard Data Data Development Application Models & Designs Feedback Direction Data Support Operations Business Data Data Asset Use Business Value 56

29 Organizational DM Functions and their Inter-relationships Defining, coordinating, resourcing, implementing, and monitoring organizational data program strategies, policies, plans, etc. as coherent set of activities. Organizational Strategies Data Program Coordination Implementation Identifying, modeling, coordinating, organizing, distributing, and architecting Guidance data shared across business areas or organizational boundaries. Goals Organizational Data Integration Integrated Models Ensuring that specific individuals are assigned the responsibility for the maintenance of specific data as organizational assets, and that those individuals are provided the requisite knowledge, skills, and abilities to accomplish these goals in conjunction with other data stewards in Direction the organization. Data Stewardship Feedback Initiation, operation, tuning, maintenance, backup/ recovery, archiving and disposal of data assets in support of organizational activities. Standard Data Specifying and designing appropriately Application architected data assets that are engineered Models & to Designs be capable of supporting organizational needs. Data Support Operations Business Data Business Value Data Development Data Asset Use 57 Organizational DM Functions and their Inter-relationships Data management processes and infrastructure Data Program Coordination Organizational Strategies Goals Implementation Guidance Combining multiple assets to produce extra value Organizational Data Integration Integrated Models Achieve sharing of data within a business area Organizationalentity subject area data integration Data Stewardship Standard Data Data Development Application Models & Designs Feedback Direction Provide reliable access to data Data Support Operations Business Data Data Asset Use Leverage data in organizational activities Business Value 58

30 How is it done? Follows form of a semistructured interview Approximately one hour is required to complete each interview Examines organizational data management practices in five areas Branched series of questions explores capabilities, execution, and ongoing efforts. Total time to results typically ranges from 1 week to 1 month 59 Council Hill Road Sign roadsign Photo from William J. Manon Jr..pbase.com/g3/91/555491/ 2/ telWKGJG.jpg 60

31 Assessment Benefits Quantitative Benefits Objective determination of baseline BI/Analytic capabilities Gap analysis indicates specific actions required to achieve the "next" level Available comparisons with similar organizations Provides facts useful when prioritizing subsequent investments Qualitative Benefits Highlights strengths, weaknesses, capabilities, and limitations existing BI/ 61 Data Management Practices Measurement (DMPA) Data Program Coordination Organizational Data Integration Data Stewardship Data Development Data Support Operations Initial (I) Repeatable (II) Defined (III) Managed (IV) Focus: Guidance and Facilitation Focus: Implementation and Access Optimizing (V) Collaboration with CMU's Software Engineering Institute (SEI) Results from more than 400 organizations Public Companies State Government Agencies Federal Government International Organizations Defined industry standard Copyright 01/1/08 Copyright and 07/23/08 previous by years Data Blueprint by Data Blueprint - all rights - reserved! all rights reserved!

32 Sample Perception vs. Fact Chart Development Guidance Data Adminstration Support Systems Asset Recovery Capability Development Training Verified Average 63 Comparative Assessment Results Data Program Coordination Challenge Organizational Data Integration Challenge Data Stewardship Data Development Data Support Operations Challenge Nokia Client Industry Competition All Respondents 64

33 High Marks for IFC s Program Data Mgmt Audit 2006 Leadership & Guidance Asset Creation Metadata Management Quality Assurance Change Management "These IFC scores represent the highest aggregate scores in the area of data stewardship recorded in our database of hundreds of assessments that has been recognized as as a representative scientific sample." Data Quality Page Overall Benchmarks Industry Benchmarks TRE IFC ISG The challenge ahead 5.00 The chart represents the average scores presented on the previous slide - interesting that none have apparently reached level

34 After more than a decade Question How many software practices (surveyed) are above level 1 on the CMM? Answer By far most organizations (95%) surveyed are producing software using informal processes Question How many organizations have demonstrated at least some proficiency according to the DM3? (i.e., scored above level 1) Answer One in ten organizations has scored above level

35 Archeology-based Transformations Solve a Puzzle Primary sources of guidance: The edge-pieces are easy to identify Distinct physical piece features exist, such as colors, patterns, pictures, etc. Steps for solving: Physically segregate all identified edge pieces (not always present in existing environment.) Create puzzle framework - connecting edge pieces using the puzzle picture Within frame, physically group remaining pieces by distinct physical features Solve a smaller section of the puzzle containing just a portion of the picture that is focused on similar physical features such as a ball or a puppy as images in the picture. This is an effective approach because the Focus is on a common domain one distinct aspect of the entire picture Because it focuses the analysis on a smaller number of puzzle pieces it is proportionately smaller than attempting to solve the overall puzzle at once. As the components are assembled, combine them to solve the complete puzzle. 69 How was this bridge constructed? 70

36 Flood New River Bridge

37 Bridge Engineering 73 Wally Eastwood Playing Piano 74

38 Evidence Type Evidence System Component System Component Type Component Element Logical Data Attribute Logical Data Entity Location User Type Information Process Model Decomposition Business Rules Business Processes XML-based Portals Business Intelligence Increased business perception of DM value resulting from better business systems including repositories, warehouses, ERP implementations Data Assets XML-based Repositories XML Tomorrow's Contact Information: Data Management 10 Challenge #4 Challenge #3 Challenge #1 Challenge #2 Challenge #1 Data Analysis Technologies Quality Copyright 2004 by Data Blueprint - all rights reserved! Revised Data Management Goals Peter Aiken, Ph.D. Department of Information Systems School of Business Virginia Commonwealth University 1015 Floyd Avenue - Room 4170 Richmond, Virginia Data Blueprint Maggie L. Walker Business & Technology Center 501 East Franklin Street Richmond, VA office : cell: [email protected] 75 Copyright Copyright 01/1/08 and 12/18/07 previous by Data years Blueprint by Data Blueprint - all rights - reserved! all rights reserved! Service Orient or Be Doomed! Service Orient or Be Doomed! How Service Orientation Will Change Your Business (Hardcover) by Jason Bloomberg & Ronald Schmelzer I'm not quite sure what "doom" awaits by not service orienting, other than remaining mired in archaic, calcified and siloed processes which a lot of 76

39 Services Integration Possibilities User Interface Business Process Application Data AV Component Well defined components Self-contained No interdependencies Analogy derived from D. Barry "Web Services" Intelligent Enterprise 10/10/03 pp wiring diagram from sunflowerbroadband.com 77 Contractor Implemented Wiring 78

40 Concise Notes on Software Engineering Published in pages including appendices & references Out of print $1.99 at half.com Principles of Information Hiding (p ) Conceal complex data structures whenever possible Allow only selected service modules to know about the concealed data structures Bind together modules that know about concealed data structures 79 How Does SOA Fit In Existing Architectures? The basketball and golfball slide Bank 80

41 Evolving applications from stove pipe to web-service-based architectures Organizational Portal Sunday, April 27, All systems operational! Organizational News Organizational Early News Press Releases Industry News Newsletters Organizational IT Service Desk Settings 320 new msgs, 14,572 total Send quick 16 million lines of legacy code 2.1 million lines of legacy code Organizational Essentials Knowledge network Employee assistance IT procurement Organizational media design Organizational merchandise Search Stocks Go Reporting Regional Northeast Northwest Southeast Southwest Midnorth Midsouth State Alabama Arkansas Georgia Mississippi Vermont Virginia Full Portfolio XYZ YYZ ZZZ Market Update As of: Sunday, April 27, 2008 Get Quote 81 Legacy Systems Transformed Into Web-services Accessed Through a Portal Legacy Application 1 Legacy Application 2 Legacy Application 3 Legacy Application 4 Legacy Application 5 Web Service 1.1 Web Service 1.2 Web Service 1.3 Web Service 2.1 Web Service 2.2 Web Service 3.1 Web Service 3.2 Web Service 4.1 Web Service 4.2 Web Service 5.1 Web Service 5.2 Web Service 5.3 Organizational Portal Organizational News Monday, November 03, All systems operational! Organizational Early News Press Releases Organizational IT Service Desk Settings Organizational Essentials Knowledge network Employee assistance IT procurement Organizational media design Organizational merchandise Reporting Regional Northeast Northwest Southeast Southwest Midnorth Midsouth State Alabama Arkansas Georgia Mississippi Vermont Virginia Industry News Newsletters 320 new msgs, 14,572 total Send quick Search Stocks Full Portfolio XYZ YYZ ZZZ Go Market Update As of: Monday, November 03, 2008 Get Quote Copyright Copyright 01/1/08 and 11/03/08 previous by Data years Blueprint by Data Blueprint - all rights - reserved! all rights reserved!

42 Solution Framework External Address Validation Processing Channels SOR 1 SOR 2 SOR 3 SOR 4 SOR 5 SORs Indicator Extraction Service (could be segmented by day of week month, system, etc.) Repository Customer Contact Latency Check Service Ch 1 Ch 2 Ch 3 Ch 4 Ch 5 Ch 6 Ch 7 SOR 6 Ch 8 SOR 7 SOR 8 Update Addresses 83 Logical Extension Text 84

43 Logical Extension 85 CONTENTS >> DEFINING THE BUSINESS VALUE OF TECHNOLOGY ISSUE 1,198 AUG. 11, 2008 COVER STORY 28 Simpler Than SOA Stymied by the complexity of SOAs, some IT departments are taking the Web-oriented architecture route > NEWS FILTER 21 Global Problem The indictment of 11 people in five countries in connection with the theft of credit card numbers from U.S. retailers demonstrates how easily cybercrime crosses borders 21 Still Standing IT spending has tightened in the United States, but demand from other parts of the world kept big tech companies growing in the second quarter 22 No Deal Deutsche Post kills a proposed seven-year outsourcing deal with Hewlett-Packard, saying it wouldn t save enough money to be worth the risk 23 Lost Opportunity IBM s e-discovery software offers many useful features, but it misses the mark by not pulling from third-party archives 23 Real Protection SunGard parlays its partnership with VMware into a service that uses virtualization to provide faster disaster-recovery setup 24 Olympic-Sized Task AT&T s new Synaptic Hosting cloud computing service will get its first big test this week, providing temporary Web server capacity for the U.S. Olympic Committee s Web site 25 New Cloud Forms Elastra advances the idea of private clouds, in which corporate data centers use the technologies and practices of public cloud infrastructures from the likes 21 Small world 22 Backpedaling Copyright of Amazon.com 01/1/08 Copyright and 07/23/08 previous and Google by years Data Blueprint by Data Blueprint - all rights - reserved! all rights reserved! Cover photo by Mick Coulas Simpler Than SOA Stymied by the complexity of SOAs, some IT departments are taking the Web-oriented architecture route Smart Web App Development Web-oriented architectures are easier to implement and offer a similar flexibility to SOA informationweek.com Aug. 11,

44 WOA Copyright 01/1/08 Copyright and 07/23/08 previous by years Data Blueprint by Data Blueprint - all rights - reserved! all rights reserved! SOA & Data &??? 88

45 SOA Requirements Data Program Coordination Organizational Data Integration Data Stewardship Data Development Data Support Operations Copyright Copyright 01/1/08 and 11/03/08 previous by Data years Blueprint by Data Blueprint - all rights - reserved! all rights reserved! Predictive Analysis I'm a little surprised, with such extensive experience in predictive analysis, you should've known we would hire you 90

46 What is Analytics? Analytics: Something that is analytic Analytic: Of or relating to analysis; especially; separating or breaking up a whole or a compound into it component parts or constituent elements 91 92

47 Car Maxx in Doha, Qatar 93 BI/Analytic Capabilities Business Intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information and sometimes to the information itself. The purpose of business intelligence--a term that dates at least to is to support better business decision making. Analytics The simplest definition of Analytics is "the science of analysis." A simple and practical definition, however, would be how an entity (i.e., business) arrives at an optimal 94

48 BI/Analytic Capabilities Analytics Business Intelligence Strategy formulation Strategy implementation 95 BI/Analytic Capabilities Wine quality = winter rainfall growing season temperature harvest rainfall (Orley Ashenfelter) Out performs experts specifically Robert Parker ( Most everyone else Clinical Versus Statistical Prediction (Paul Meele) 8/136 studies experts were more accurate 96

49 I didn t have the data 97 Copyright Copyright 01/1/08 and 2004 previous by Data years Blueprint by Data - Blueprint all rights - reserved! all rights reserved! BI Challenges Technical Challenges Poor quality data Poor understanding of architectural constructs Poor quality data management practices New technical expertise is required Non-Technical Challenges Architecture is under appreciated BI perceived as a "technology" project Inability to link technical capabilities to business objectives Putting BI initiatives in context 98

50 Obstacles to Real-Time BI-Lessons from Deployment Business case, high cost or budget issues 60% Non-integrated data sources Education and understanding of real-time BI by business users Lack of infrastructre for handing real-time processing 47% 46% 46% Poor quality data 43% Education and understanding of real-time BI by IT staff Lack of tools for doing real-time processing 36% 35% Immature technology 28% Performance and scalability 24% 99 TDWI The Real Time Enterprise Report, 2003 Cost of Poor Data Quality $600 Billion Annually! 100 Thanks to Bret Champlin

51 Who is Joan Smith? Defining Customer Challenges Purchased an A4 on June Had not done business with the dealership prior "makes them seem sleazy when I get a letter in the mail before I've even made the first payment on the car advertising lower payments than I got" 102

52 Defining Customer Challenges Purchased an A4 on June Had not done business with the dealership prior "makes them seem sleazy when I get a 103 How to solve this data quality problem using just tools? Retail price for the unit was $40 104

53 A congratulations letter from another bank Problems Bank did not know it made an error Tools alone could not have prevented this error Lost confidence in the ability of the bank to manage customer funds 105 From my retirement plan 106

54 Rolling Stone Magazine 107 Quantitative Benefits 108

55 Please Help with A Research Project! Data Management Practices Assessment [email protected] 109