Disparate Data, Disparate Systems, Disparate User Groups (How to Architect The Enterprise Business Needs) Robert Schork, General Dynamics IT

Transcription

1 Disparate Data, Disparate Systems, Disparate User Groups (How to Architect The Enterprise Business Needs) Robert Schork, General Dynamics IT April 27, Waters North American Trading Architecture Summit 1

2 Who am I? Over 25 years of IT Data Mgmt (including consulting) Over 20 yrs of Metadata experience including Metadata Analysis and Design for Fortune 100 corporations Implemented ASG Rochade and CA Platinum Repositories as well as created several Metadata Repositories/Scanners Current President of the Metadata Professional Org Former member of the IBM Data Governance Council and Meet The Boss Presenter at several Metadata and FIMA conferences Former Developer, DBA, Data Modeler, DA and BA Implemented several Enterprise-wide Data and Metadata Management, Data Governance, and Data Modeling Strategies at Fortune 500 companies. April 27, Waters North American Trading Architecture Summit 2

3 Questions for the Group 1. How do you ensure that your data will be available and timely when needed? 2. How do I know that the definitions that I have are accurate and consistent. 3. What is the common denominator between disparate data, disparate systems, and disparate user groups? 4. How do you Architect the Enterprise Business needs so every group will get Their Data? 5. How many of you have your eyes glaze over when you hear the word. METADATA April 27, Waters North American Trading Architecture Summit 3

4 I Know What You re Thinking April 27, Waters North American Trading Architecture Summit 4

5 How much Budget do you have? April 27, Waters North American Trading Architecture Summit 5

6 What is Metadata? Definition: Data about Data (useless definition) Metadata is the definition and usage of the information about the data of your organization. Business Metadata 3 Types Business Technical Operational Few really understand it It is not Pie in the Sky Technical Metadata Operational Metadata April 27, Waters North American Trading Architecture Summit 6

7 Business Metadata Metadata Types Business Terms, Applications Stakeholders and Stewards Business Processes Technical Metadata Databases, Tables, Columns Flat files, XML docs, Spreadsheets Operational Metadata ETL Process jobs Programmatic metadata (Programs, Traceability) April 27, Waters North American Trading Architecture Summit 7

8 AGENDA Case Study: Disparate data, Disparate Systems, Disparate User Groups Application Groupings (Topology) Serving diverse groups of users Extensions and Integration points Examples April 27, Waters North American Trading Architecture Summit 8

9 Citi s Disparate Data Stores April 27, Waters North American Trading Architecture Summit 9

10 Dilemma at Citi Goal was to capture Business Metadata and associate that with the existing Technical and Operational Metadata. Need to incorporate that throughout the Enterprise that has several distinct Business Sectors or LOB. How can we capture and incorporate Data Governance, Business Ownership, and Data Stewardship into the overall strategy. Vendor was too application specific with their approach. April 27, Waters North American Trading Architecture Summit 10

11 First Step Decisions Implement a POC to determine if the ETL Repository was extensible, easily loadable, and reportable. Architect distinct Enterprise and Business Sectors separations of Business Term Metadata. Find a vehicle to capture Business Metadata in various forms. Define a process to associate that Business Metadata with the existing ETL Metadata elements. Determine if the existing Business Process Manager workflow engine could communicate and support Business Ownership and Data Stewardship approvals for Governance. Develop loading methods for each different metadata type. Sell it to a skeptical Sector base who has been burned before. April 27, Waters North American Trading Architecture Summit 11

12 AGENDA Case Study: Disparate data, Disparate Systems, Disparate User Groups Application Groupings(Topology) Serving Diverse Groups of Users Extensions and Integration Points Examples April 27, Waters North American Trading Architecture Summit 12

13 Topology The word topology is used both for the mathematical discipline and for a family of sets with certain properties that are used to define a topological space, a basic object of topology. Of particular importance are homeomorphisms, which can be defined as continuous functions with a continuous inverse. Topology includes many subfields. The most basic and traditional division within topology is point-set topology, which establishes the foundational aspects of topology and investigates concepts inherent to topological spaces Bob s Definition of Topology Categorizing and grouping items in an orderly manor to be used and reported on as to be useful to the Business. Must be aligned to the Business Processes or Business Areas. Must have involved Stakeholders April 27, Waters North American Trading Architecture Summit 13

14 Multi-Tiered Architecture 1. Enterprise - Enterprise-level artifacts such as the highest Level Corporation business terms and processes. 2. Sector/LOB Level Grouping of Applications that perform a vital function 3. Application/System Level - The Local level Business data used by an individual Application. This is a Stewardship level. April 27, Waters North American Trading Architecture Summit 14

15 Extended Metamodel Components Role (Business Owner, IT Steward) Enterprise Business Term Application or System Sector Business Term Linked Sector or Line of Business Linked Data Model Local Business Term Database Logical Entity Table/ File Logical Attribute Column/ Field April 27, Waters North American Trading Architecture Summit 15

16 How do we start? Discovery Inventory of all applications and associated data stores. Record I/O to other systems. Classification - Identify all stores of metadata and assign them types (DBs, data models, data dictionaries, etc). Part of this step is the linking of Business and Technical metadata. Set-up - Determine the architecture for metadata storage to support the business needs. Integration - Compare one system to another and integrate those business terms at the LOB/Sector and/or Enterprise levels. Follow-up Discuss, confirm, and implement the metadata management strategy with the management. Form steering committees and establish standards. April 27, Waters North American Trading Architecture Summit 16

17 AGENDA Case Study: Disparate data, Disparate Systems, Disparate User Groups Application Groupings (Topology) Serving Diverse Groups of Users Extensions and Integration Points Examples April 27, Waters North American Trading Architecture Summit 17

18 Serving Diverse Groups of Users Different Sectors and Applications use data in different ways. This is why integration is so difficult. Setting boundaries and rules makes the reporting and usage of data relevant. Now each application area has their own Sandbox to play in. Inputs and Outputs will need to be defined. If these areas want to get really ambitious, then they can integrate with other applications within their sectors April 27, Waters North American Trading Architecture Summit 18

19 Now that I have a Sandbox, so what? You can define all your Business terms that are used in your application. You can link the Database and Data Model elements to these Business Terms If you bring in your ETL information you can have your traceability defined and useable for impact analysis. You can now reuse reference tables. You can accomplish this in 6 months. April 27, Waters North American Trading Architecture Summit 19

20 AGENDA Case Study: Disparate data, Disparate Systems, Disparate User Groups Application Groupings (Topology) Serving Diverse Groups of Users Extensions and Integration Points Examples April 27, Waters North American Trading Architecture Summit 20

21 Multi-Tiered Architecture April 27, Waters North American Trading Architecture Summit 21

22 What is needed A Metadata Repository tool (Build or Buy) Data Management or Center of Technical Excellence Team that consists of: Project Manager (Hands on) Metadata Architect (Metamodels) Sr. Repository Architect (Tool Admin) Must have Systems Analyst, Business Analyst, and Subject Mater Expert (SME) support Develop a plan to load and report on Metadata April 27, Waters North American Trading Architecture Summit 22

23 Repository Governance Process April 27, Waters North American Trading Architecture Summit 23

24 Approval Sequence Diagram April 27, Waters North American Trading Architecture Summit 24

25 AGENDA Case Study: Disparate data, Disparate Systems, Disparate User Groups Application Groupings (Topology) Serving Diverse Groups of Users Extensions and Integration Points Examples April 27, Waters North American Trading Architecture Summit 25

26 Enterprise Data Asset 2011 Waters North American Trading Architecture Summit 26

27 Business Sector Data Assets 2011 Waters North American Trading Architecture Summit 27

28 Business Data Asset Links 28

29 Change Request Tracking 2011 Waters North American Trading Architecture Summit 29

30 Traceability 2011 Waters North American Trading Architecture Summit 30

31 Questions Contact Information Bob Schork, General Dynamics IT Enterprise Data Architect Defense Medical Logistics, DoD Philadelphia, Pa Waters North American Trading Architecture Summit 31