Advantages of a Layered Architecture for Enterprise Data Warehouse Systems

Transcription

1 1 Advantages of a Layered Architecture for Enterprise Warehouse Systems Thorsten Winsemann, Veit Köppen, Gunter Saake Otto-von-Guericke-Universität, Magdeburg/Germany Complex Systems Design & Management Complex Systems Design & Management 2011 CSD&M, Paris/France, December 7-9

2 Table of contents 2 1. Characteristics of Enterprise Warehouses 2. Traditional Warehouse Architecture 1. Reference Architecture 2. flow-example 3. Architectures for Enterprise Warehouses 1. SAP s Layered, Scalable Architecture 2. Layers in Detail 3. flow-example 4. Simple, but Detailed Example 4. Architectural Differences 1. Overview 2. Advantages of a Layered Architecture Complex Systems Design & Management

3 Characteristics of Enterprise Warehouses (EDW) Business DW, thus covering all business areas basis for several applications, such as BI, planning, CRM, Single Version of Truth of company s data Multiple, heterogeneous source systems Huge amount of data (granular, detailed, old) World-wide scope, different time zones 24*7-hours availability 3 Complex Systems Design & Management

4 4 Traditional Warehouse Architecture: Reference Architecture User Warehouse System Operational Store Marts Basis Base Staging Area Sources Complex Systems Design & Management

5 Traditional Warehouse Architecture: flow-example 5 Marts Year 1 Year 2 Year 3 Basis Base Staging Area DW System Order Header Order Item Sale Invoice Header Sale Invoice Item ERP System Complex Systems Design & Management

6 Architecture for Enterprise Warehouses: SAP s Layered, Scalable Architecture 6 User Reporting & Analysis Layer Business Layer Propagation Layer Quality & Harmonisation Layer Acquisition Layer Corporate Memory Operational Store Warehouse System Sources Complex Systems Design & Management

7 Architecture for Enterprise Warehouses: flow-example 7 Reporting & Analysis DW System Year 1 Year 2 Year 3 Special Business Order Propagation Acquisition + Corporate Memory Sale Invoice Special Sale Sale Order Order Invoice Invoice Header Item Header Item Quality & Harmonization Order Header Order Item Sale Invoice Header Sale Invoice Item Order Header Order Item Sale Invoice Header Sale Invoice Item ERP System Complex Systems Design & Management

8 8 Architecture for Enterprise Warehouses: Layers in Detail (1) Acquisition Layer DW Inbox (temporary) stored immediately without changes Corporate Memory DW Life Insurance (long-term, granular, complete) for non-predictable demands ( master the unkown ) Quality & Harmonization Layer Technical and semantical data integration Usually no data storage Complex Systems Design & Management

9 9 Architecture for Enterprise Warehouses: Layers in Detail (2) Propagation Layer Single Version of Truth Harmonized, integrated data without business logic Business Layer are transformed according to business needs E.g., combination of sales + finance figures Reporting & Analysis Layer are transformed according to requirements for usage and fast access performance Complex Systems Design & Management

10 Architectures for Enterprise Warehouses: Simple, but Detailed Example (1) 10 Harmonization & Quality Layer (no persistence!) Order ORDNR DATEO CUSTOM Char10 Char8 Char Order -Item ORDNR ITMNO MATNR QUASU UNITS AMDCO CURRD Char10 Char4 Char15 Dec10,3 Char3 Dec15,2 Char ABT ,000 BOX 300,00 EUR Aquisition Layer + Corporate Memory Order DOCNR ODATE BUYER SYSID Numc10 Char10 Char10 Char SAO Order - Item DOCNR ITMNR ARTNR QUASU SUNIT AMODC DCURR SYSID Numc10 Numc4 Char15 Dec10,2 Char3 Dec15,2 Char2 Char ABT ,00 BOX 300,00 EU SAO Sale Invoice INVNR DATEI CUSTOM ORDNR DATEP Char10 Char8 Char10 Char10 Char8 IN Sale Invoice- Item INVNR ITMNI MATNR QUABU UNITB AMDCI CURRD Char10 Char4 Char15 Dec10,3 Char3 Dec10,2 Char3 IN ABT ,000 PC 285,00 EUR SaleInvoice DOCNR INVDT PAYER ORDER PDATE SYSID Char7 Char6 Char7 Numc10 Char6 Char3 IN SBI Sale Invoice- Item DOCNR ITMNR ARTNR QUANT BUNIT AMODC DCURR SYSID Char7 Numc3 Char8 Dec10,3 Char2 Dec10,2 Char3 Char3 IN ABT ,000 ST 285,00 EUR SBI types adapted Homonyms split Synonyms merged Field names changed System-ID added Order DOCNR ODATE BUYER System A: Ordering SaleInvoice DOCNR INVDT PAYER ORDER PDATE System B: Invoicing Numc10 Char10 Char10 Char7 Char6 Char7 Numc10 Char IN Order - Item SaleInvoice- Item DOCNR ITMNR ARTNR QUASU SUNIT AMDCI DCURR DOCNR ITMNR MATNR QUANT BUNIT AMDCI DCURR Numc10 Numc4 Char15 Dec10,2 Char3 Dec15,2 Char2 Char7 Numc3 Char8 Dec10,3 Char2 Dec10,2 Char ABT ,00 BOX 300,00 EU IN ABT ,000 ST 285,00 EUR Complex Systems Design & Management

11 Architectures for Enterprise Warehouses: Simple, but Detailed Example (2) 11 Reporting & Analysis Layer Report Execution (no persistence!) CUSTOM MONTH AMLCI CURRL PRPPC Char10 Char6 Dec10,2 Char3 Dec10, ,00 GBP 50,00 Further information added CUSTOM MATNR MATGR DATEO QUABU UNITB DATEP AMDCI CURRD Char10 Char15 Char3 Char8 Dec10,3 Char3 Char8 Dec10,2 Char ABT00471 ABT ,000 PC ,00 EUR Business Layer combined (according to usage) Orders ORDNR ITMNO DATEO CUSTOM MATNR QUASU UNITS QUABU UNITB AMDCO CURRD Char10 Char4 Char8 Char10 Char15 Dec10,3 Char3 Dec10,3 Char3 Dec15,2 Char ABT ,000 BOX 4,000 PC 300,00 EUR Propagation Layer Order ORDNR DATEO CUSTOM Char10 Char8 Char Sale Invoices INVNR ITMNI MATNR DATEI CUSTOM ORDNR DATEP QUABU UNITB AMDCI CURRD Char10 Char4 Char15 Char8 Char10 Char10 Char8 Dec10,3 Char3 Dec10,2 Char3 IN ABT ,000 PC 285,00 EUR SaleInvoice INVNR DATEI CUSTOM ORDNR DATEP Char10 Char8 Char10 Char10 Char8 IN configured (according to business needs) Order - Item ORDNR ITMNO MATNR QUASU UNITS QUABU UNITB AMODC CURRD Char10 Char4 Char15 Dec10,3 Char3 Dec10,3 Char3 Dec15,2 Char ABT ,000 BOX 4,000 PC 300,00 EUR Sale Invoice- Item INVNR ITMNO MATNR QUABU UNITB AMODC CURRD Char10 Char4 Char15 Dec10,3 Char3 Dec10,2 Char3 IN ABT ,000 PC 285,00 EUR Additional information added Harmonization& Quality Layer Complex Systems Design & Management

12 Architectural Differences: Overview 12 Matter Reference Architecture Layered Architecture Complexity Medium High (several layers) volume High Very high Conceptual work Medium (requirement-driven) High (overall concept view) Implementation effort Medium High Enhancement & scalability Possible Supported Complex Systems Design & Management

13 Architectural Differences: Advantages of a Layered Architecture 13 Matter Reference Architecture Layered Architecture Change of transformation rules (e.g., changed keyfigure calculation) Change of data (e.g., new key-figure calculation) Need for new data Reload/-build from source system Reload/-build from source system flow enhancement and reload/-build Single Version of Truth No Yes Decoupling of data load and availability No/limited Rebuild from propagation layer Rebuild from propagation layer Load from propagation layer or corporate memory Yes/supported Detailed previous data Limited Available (corporate memory) Complex Systems Design & Management

14 Appendix CSDM 2011 Poster Layout Slides Arrangement on Panel Panel: 150x125cm Slides: A3 + A Complex Systems Design & Management