Enterprise Data Integration

Transcription

1 Enterprise Data in Large Organisations Dr Hamed Jafarzadeh 2 nd National Conference on Information Systems Faculty of Management, University of Tehran, February 2016

2 Overview Why Enterprise Data (EDI)? Challenges Maturity (4 steps) Example of What Data Enables Adventure in EDI

3 Why Enterprise Data Many corporate systems developed/purchased over many years Spaghetti! (Point-to-Point )

4 Is something wrong with the Spaghetti?!

5 Issues with the Spaghetti! Vulnerable conceptually and technically Error prone

6 Issues with the Spaghetti! Unknown impact upon changes, thus owners hesitant to share data

7 Issues with the Spaghetti! Resistance to business process improvement

8 Issues with the Spaghetti! Nightmare for new external vendor when need to integrate with internal systems

9 Issues with the Spaghetti! Lack of organisational control Data security and privacy concerns

10 What is the solution?

11 Data Hub

12 Challenges Amount very many challenges addressed in the literature, these were observed in this case: 1. Lack of digital data 2. Data quality (e.g. absence of key identifiers) 3. Lack of enough business understanding of data within the project team 4. Has to be led by a highly trusted person in the organisation, or you ll fail! Easiest bit: Technical things!

13 Technology Traditional extract-transform-load (ETL) via DMBSs Specialized ETL tools : e.g. ODI

14 Is that enough?

15 Maturity of data and services tend to mature through four overlapping stages or styles of integration. Business Data Exchange Data Application (Phillips, 2014)

16 Maturity (stage 1) of data and services tend 1) Data Exchange: to mature through four overlapping stages Point-to-Point or styles of integration. Business Data Exchange Data Application (Phillips, 2014)

17 Maturity (stage 2) 2) Data of : data and services tend to mature through Hub four overlapping stages or styles of integration. Business Data Exchange Data Application 2) Shortcomings: ETL cycle typically takes 24 hours Delay! (Phillips, 2014)

18 Data Step : Shortcomings 24-hours ETL cycle Hub Students ERP Upstream Collection Integrated Data Hub Enterprise View Downstream Distribution Students Portal Mobile App Direct real-time connection!

19 Data Step : getting timing right! 24-hours ETL cycle Hub Students ERP Upstream Collection Integrated Data Hub Enterprise View Downstream Distribution Students Portal Mobile App Direct real-time connection!

20 Data Step : getting timing right! 24-hours ETL cycle Hub Students ERP Upstream Collection Integrated Data Hub Enterprise View Downstream Distribution Students Portal Mobile App + Change Data Capture (CDC) Timestamps Database triggers Application logic Log based synchronisation

21 Maturity (stage 3) 3) Application : To address real-time data integration and service integration To address integration of functions and services o Enterprise Service Bus (ESB) o Application Programming Interface (API) Gateway Business Data Exchange Data Application

22 Enterprise Service Bus (ESB) Part of SOA family An architecture for integrating numerous applications together over a bus-like infrastructure Systems communicate without dependency or knowledge of other systems Vendors increasingly integrating platforms on the cloud as Integrate Platform as a Service (ipaas)

23 Application Program Interface (API) Gateway Central Data API (CDA)

24 Maturity (stage 4) 4) Business : BPM BPMS Business Data Exchange Data Application

25 Who? How? In-House P2P Data Hub Data Exchange ODI CDC Data Application External Experts API ESB Business BPM So many terms: Brokerage (IB) Hybrid Platform (HIP) DBAs Solely IT Teams Competency Centre (ICC) Centre of Excellence (COE) ICC is a centre of expertise to facilitate integration efforts by providing resources that enable some level of controlled integration self-service

26 Who? How? By 2018, 47% of enterprises will consolidate their data integration and application integration competencies as one team for aligning disciplines and technologies. Source: Gartner (September 2015), n = 317

27 Extremist Approach! oogle Way of Thinking

28 Thank you! Dr Hamed Jafarzadeh 2 nd National Conference on Information Systems Faculty of Management, University of Tehran, February 2016