A NEW ARCHITECTURE FOR ENTERPRISE APPLICATION SOFTWARE BASED ON IN-MEMORY DATABASES

Transcription

1 A NEW ARCHITECTURE FOR ENTERPRISE APPLICATION SOFTWARE BASED ON IN-MEMORY DATABASES Dr. Jens Krueger SVP, Head of LoB Finance & SAP Innovation Center 45 th Annual Convention of the German Informatics Society September 30, 2015 Cottbus 2015 SAP SE. ALL RI GHT S RESERVED.

2 Large enterprise systems Transaction-maintained aggregates explode in complexity ~ 30,000 Functions } Aggregate Maintenance Development Costs ~ 80,000 Tables ~900,000 Columns } Schema Complexity Consistency Redundancy Data Footprint of up to 150TB } Storage Costs Backup Disaster Recovery 2

3 Hardware development Enabler for a new era of business processing 20 YEARS AGO TODAY NEAR FUTURE MEMORY MEMORY MEMORY 1 GB X TB 48 TB CPU CPU CORES 4 X 50 MHZ X X 3.0 GHZ 480 (8X4X15) 3

4 Workload analysis Read-mostly applies to OLTP and OLAP 100 % 100 % W orkload 90 % 80 % 70 % 60 % 50 % 40 % Delete Modification Insert Read W orkload 90 % 80 % 70 % 60 % 50 % 40 % Delete Modification Insert Read 30 % 30 % 20 % 20 % 10 % 10 % 0 % TPC-C 0 % OLTP OLAP TPC-C Benchmark Workload Analysis Krueger et al. VLDB 11 4

5 Simplification and Fexibility By removing static pre-defined materialized aggregates Static Pre-Defined Aggregates Flexible Aggregation On Demand Static pre-defined aggregates cannot handle structural changes within the organization ON-THE-FLY AGGREGATION FI POSTINGS CUSTOMERS PRODUCTS PROFIT CENTERS SUPPLIERS Dynamic on-the-fly reporting enables analysis & simulation of organizational changes. Impact analysis is immediately available FI POSTINGS CUSTOMERS PRODUCTS PROFIT CENTERS SUPPLIERS Requires Anticipation Does NOT Require Anticipation 5

6 2009: A common database approach For OLTP and OLAP using an in-memory column database Financials Logistics Manufacturing... OLTP & OLAP Applications SQL Interface Stored Procedures Data Management Layer Read-only Replicas Actual Partition (Master) Delta Column Store Attribute Vectors Attribute Vectors Dictionaries Dictionaries Aggregate Cache Row Store Main Historical Partition 1 Main Attribute Vectors Dictionaries Aggregate Cache Main Memory Storage Historical Partition 2 Main Attribute Vectors Dictionaries Aggregate Cache... Logs Checkpoints Non-volatile Storage Source: SIGMOD June

7 A textbook example Wire money from account A to account B Accounts with Transaction-Maintained Balance Raw Account Transactions TRANSACTIONS Insert TRANSACTIONS Primary Key Select BALANCE Update Σ Read (Column Scan) 7

8 Example Posting of a vendor invoice Vendor Invoice Row 1: ~~~~~ Tax: ~~~~~ Total: ~~~~~ Header Account Vendor Cost G/L Account Tax G/L Account 119 Credit 100 Debit 19 Debit 8

9 SAP Financials Invoice Posting Before simplification Index on Primary Key Customer Vendor General Ledger Cost Center Other Indices KNA1 LFA1 SKA1 CSKS Application managed materialized aggregates KNC1 Update LFC1 GLT0 COSP Accounting Document Header Accounting Line Items Split Accounting Line Items Controlling Object Document Header Controlling Object Line Items Tax Documents Vendors Customers G/L Accounts BSAK BSAD BSAS Closed Items Insert BKPF BSEG FAGLFLEXA COBK COEP BSET BSIK BSID BSIS Open Items Application managed materialized views 9

10 SAP Financials Invoice Posting Simplification Customer Vendor General Ledger Cost Center KNA1 LFA1 SKA1 CSKS KNC1 LFC1 GLT0 COSP Accounting Document Header Accounting Line Items Split Accounting Line Items Controlling Object Document Header Controlling Object Line Items Tax Documents Vendors Customers G/L Accounts BSAK BSAD BSAS BKPF BSEG ACDOCA COBK COEP BSET BSIK BSID BSIS 10

11 SAP Financials Invoice Posting After simplification with database views Customer Vendor General Ledger Cost Center KNC1 KNA1 LFA1 SKA1 CSKS LFC1 GLT0 COSP Views with on-the-fly Aggregations Accounting Document Header Accounting Line Items BSAK BSAD BSAS Views with Split Accounting Line Items BKPF BSEG BSIK BSID BSIS Projection and ACDOCA COBK COEP BSET Selection 11

12 Why column-stores are better than row stores for transaction processing - theoretically... Insert duration with and w/o transaction-maintained aggregates BKPF BSEG FAGLFLEXA BSIS BSIS BSIK BSET (PER LEDGER) LFC1 GLT0 COBK COEP COSP Inserts 5 Updates Classic SAP Financials with Transaction-Maintained Aggregates BKPF BSEG ACDOCA (PER LEDGER) 7 Inserts 0 Updates Simplified SAP Financials 12

13 ...and experimentally with respect to workload and runtime Document posting measured on customer data 100 % W orkload 90 % 80 % 70 % 60 % 50 % 40 % 30 % 20 % 10 % 0 % Classic Simplified SAP Financials Delete Modification Insert Read Time (ms) 30,000 25,000 20,000 15,000 10, Sum of CPU time (ms) Suite on HANA S/4HANA Finance Sum of DB reg. time (ms) SAP Financials Workload Analysis SAP Financials Runtime Analysis 13

14 Universal Journey Entries establish a single source of truth and enable a variety of new analyses Processing Analytics Processing Transaktional Analytics Analytisch Totals + Indices Totals + Indices Totals + Indices Totals + Indices Totals + Indices on-the-fly aggregation Financial Accounting Documents Material Ledger Documents Dokumente Controlling der Finanzbuchhaltung Documents Asset Accounting Documents Profitability Documents Universal Journal Entries 14

15 Compatibility views enable a non-disruptive adoption by redundant tables COMPATIBILITY VIEWS Item Views BSID, BSAD, BSIK,, FAGLBSIS, Aggregate Views GLT0, LFC1, KNC1, FAGLFLEXT, Journal Views COBK, COEP, FAGLFLEXA Filter Filter Filter Project Project Project Aggregate Transform CORE DATA MODEL OF SAP S/4HANA FINANCE BKPF BSEG ACDOCA 15

16 Simplified Logistics No aggregates: on-the-fly aggregation of inventories Classical Logistics Data Model Simplified Logistics Data Model MKPF MSTQH MSTHE MSTBH MSSQH MSKUH MSKAH MKOLH MCHBH MATDOC MSEG MSSAH MCHBA MARDH MARCH MSTQ MSTE MSSQ MSSA MCH1 MCHA MSSQ MSSA MCH1 MCHA MSPR MSLB MSKU MSKA MAKT MARM MSKU MSKA MAKT MARM MKOL MCHB MARD MARC MARA MARD MARC MARA TRANSACTIONAL DATA AGGREGATED QUANTITIES MASTER DATA + AGGREGATED QUANTITIES MASTER DATA 16

17 Simplified Logistics Example case with real data from large automotive company Throughput items/sec Delete Classical Data Model Simplified Data Model Backflush real-time, large automotive company # Parallel Processes 17

18 Database footprint reduction With SAP HANA ERP on any DB 7.1 TB CACHE 0.3 TB BACKUP/RECOVERY 1 /35 OF ERP ON ANY DB ERP on HANA 1.8 TB DEVELOPMENT SYSTEM 1 /35 OF FULL DATA No Indices No Aggregates No Redundancy «serp» on HANA (estimate) 3 : TB HOT 0.2 TB TEST SYSTEM 1 /35 OF FULL DATA Hot = Actuals, status at beginning of current year + all transactions during current year to conduct business and fulfill statuary reporting Cold = Historical, up to 10 years 18

19 Transparent aggregate cache Database-maintained aggregates on read-only main partition On-the-fly Aggregation Transparent Aggregate Cache COLD N COLD 1 COLD N COLD 1 MAIN MAIN DELTA Inserts AGGREGATES Aggregation Σ DELTA Inserts Aggregation Σ 19

20 Read-Only Replication Leveraging business workload characteristics Executives Sales Managers Accountants OLAP, Search, and Read-only Applications on Transactional Schema Read-only Replicas Mixed Workload Processing Master Node Asynchronous, Transactionally Consistent Replication OLTP Operational Reporting & New Applications Data Entry 20

21 Boardroom Redefined TRENDS Review the state of the business at any time, examining data in real-time Prioritize where we spend time reviewing Apply analytics to identify solutions Higher speed leads to more intelligence 21

22 Thank you.

23 Disclaimer This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.