SAP BW 7.40 Near-Line Storage for SAP IQ What's New?

Transcription

1 SAP BW 7.40 Near-Line Storage for SAP IQ What's New? Rainer Uhle Product Management SAP EDW (BW / HANA), SAP SE Public

2 Disclaimer This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent SAP SE or an SAP affiliate company. All rights reserved. Public 2

3 Process Orchestration Process Orchestration SAP BW on HANA Smarter, simpler, more efficient How Does BW running on RDBMS differ from BW running on HANA? Customer value of SAP BW powered by SAP HANA SAP NetWeaver BW Data Modeling Planning OLAP Data Management SAP BW on HANA Data Modeling Planning OLAP Data Management Excellent query performance for improved decision making Performance boost for Data Load processes for decreased data latency Accelerated In-Memory planning capabilities for faster planning scenarios Flexible combine EDW with HANA-native data for real-time insights and decision making Data persistency layers are cut off and reduced administration efforts Simplified data modeling and remodeling any NLS RDBMS Traditional Stack Data Schema & Data SAP IQ-NLS HANA Stack Push Down Data intensive functions are pushed down from BW to HANA HANA as the Primary Database for BW and Foundation for new Applications 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 3

4 Differentiations: Multi Temperature Data Approach hot warm cold Data is read and/or written frequently In HANA DB, Full In-memory handling Accessibility: No restrictions, all features available Infrequent access HANA Dynamic Tiering (fka Extended Storage) In HANA DB, on disk, Main memory for caching and processing Accessibility: No restrictions, all features available Sporadic access BW-NLS Management for read-only data Not stored in HANA DB; stored in SAP IQ Near-line DB Accessibility: Restricted to BW-NLS capabilities, read-only Providing lower TCO by optimized data volume management 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 4

5 BW powered by HANA and Sybase IQ Near-Line Storage (NLS) Architecture - Overview MultiProvider SAP HANA BI Clients SAP Netweaver BW 7.3x Transient Provider NLS- Partner OEM InfoCube/DSO Near-Line SDK Architecture BW NLS4IQ SAP Native SAP owned alternative to existing NLS-Partner Solutions An SAP - owned BW NLS implementation for Sybase IQ offers a fully integrated solution for combined primary & secondary database management: Primary Database (preferable SAP HANA) active data, maximum performance, transactional changes, high frequent access, very short MTTR* Secondary Database (always SAP IQ) Slave (detouchable without breaking the whole system), non-active data, lower performance accepted, only bulk loads at scheduled PITs*, entirely readonly, low frequent access, longer MTTR acceptable *) MTTR Mean Time To Recover PIT - Point in Time 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 5

6 Introducing SAP HANA dynamic tiering Map data priorities to data management Hot data SAP HANA Database Hot Store Warm data Dynamic Tiering Primary Image in memory Durability RAM All in one database Warm Store Cache / Processing Primary Image on disk Hot Store Classical HANA tables Primary data image in memory DB algorithms optimized for in-memory data Persistence on disk to guarantee durability Warm Store Extended Tables Primary data image on disk Data processing using algorithms optimized for diskbased data Main memory used for caching and processing SAP SE or an SAP affiliate company. All rights reserved. Public 6

7 Decision Matrix Extended Table vs. NLS on InfoProvider Types Supported InfoProvider Types per temperature InfoCube Standard DSO w/o DSO PSA HOT WARM currently not supported Advanced DSO will replace InfoCubes and Standard DSOs supporting Dynamic Tiering/ Extended Tables currently not supported not recommended not recommended HANA COLD not supported by SAP (PBS) Archive 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 7

8 NLS Scenario Design and Runtime Aspects BW DB InfoProvider DBMS Younger Q1/2011 Q4/2010 Q3/2010 Mass Maintenance Tool for DAPs, Variants, Process Chains available (SP08) NLS - Interface DAP Design Time Define partitioning criteria Generate DAP DAP Scheduling / Monitoring / Run-Time: Describe partition to be extracted Generate request InfoProvider Manage Menu Process Chain Archive / Extract (DAP menu, Process Chain Restore (DAP menu) Add. Features Query access to NLS DTP access to NLS Look up during Transforms Data Archiving Process (DAP) NLS Implementation Q2/2010 Q1/ SAP SE or an SAP affiliate company. All rights reserved. Public 8

9 Example for 3(4)-fold Archiving Process on single Object Data Relocation to NLS takes place in three phases Copy Phase: All data of a request is copied in individually committed packages to SAP IQ Data is pulled from BW database Data is transferred across the network and loaded into NLS using IQ client side load After the last data package is loaded the final transaction commit id is retrieved IQ Backup Event is Triggered to capture all the data that was archived Verification Phase: Load Performance Technical key of archived records is retrieved and verified for completeness Improvements under continous Deletion Phase: investigation Before data is deleted from primary database (still potential for Deletion proceeds only if the latest backup includes all the data from the copy improvements) phase Data is deleted from the BW database and Archived Data in NLS becomes active IQ Backup Event is Triggered to capture the end state of the NLS database Throughput Up to 250 Mio. rows/h (100 GByte/h) 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 9

10 Main Principle: Relocation - Cold NLS Data is write-protected BW InfoProvider Newer Q1/2011 Q4/2010 Locked Q3/2010 for update Source RDBMS NLS - Interface Updates for archived records in the period Q1 to Q can be managed using DTP Error Handling after NLS reload NLS Implementation Q3/2010 Q2/2010 Q1/2010 Update capabilities coming soon!! ( Straggler Management ) 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 10

11 BW NLS in action - Runtime Experience read NLS as well Further NLS Read Performance Improvements based on HANA SDA available (SP08) Acceptable NLS Latency Factor 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 11

12 SAP NetWeaver BW Product Roadmap Focusing BW NLS Implementation for Sybase IQ BW on HANA Query Optimizations Optimized performance for NLS read purposes (HANA SDA with BW 7.4 SP05) Supported Platforms Server/Client Sybase IQ 16.0 SP3 Sybase IQ ODBC NLS client for BW AppServer Windows on x64 64bit, Linux (x86_64 64bit), AIX 64bit, HP-UX, Solaris, OS/400 Error DTP Handling in case of NLS caused lock violations Support for Standard DSO with BW 7.3 SP11 (resp. Note ) IQ Backup Automation Event trigger for incremental backup in IQ after data relocation to NLS (BW 7.0 SP32) BW NLS Downport to BW 7.0 BW 7.0 SP32 (CW2/2014) BW 7.0 SP28 with note BW NLS Enhancements NLS support for BW Inventory Cubes Generated HANA Models of an InfoProvider can access data in NLS BW on HANA Query Optimizations Further improvement of NLS Query performance due to SDA optimizations (SDA Virtual Tables in HANA CalcViews) Extended Pruning for time characteristics BW NLS Implementation Support Mass maintenance Support (BW 7.40 SP 8) for generating DAPs, DAP Variants, Process Chains Further improvements on NLS operations DBA Cockpit for integrated Sybase IQ Monitoring (BW 7.40 SP 8 + downports up to NW 7.02) New nearline connection mode Optimized handling of attached NLS in BW system copy or migration process to HANA Improved operations when nearline data is unavailable Lookup Performance (SP10) Optimizations for lookup to archived data using the Nearline Interface API Throughput optimizations in archiving (SP11) New parallelization option copy, verification and deletion phase of a Data Archiving Process (DAP) High Performance Data Relocation through multi-threaded execution of the LOAD statement within the SAP IQ server NLS support for additional BW InfoProviders (planned for next BW Feature Pack) Supporting the Advanced DataStore Object BW NLS Enhancements (planned for next BW Feature Pack) Integrated handling of exceptional updates to archived nearline time slices within BW Processes (e.g. Advanced DataStore Object Activation) Today Planned Innovations Future Direction 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 12

13 SAP HANA Smart Data Access and SAP BW NLS Query Enhanced Business Flexibility by providing the logical EDW Data Federation in diverse EDW landscapes BW Virtualization Layer Composite Provider, Open ODS View Smart data access read access to relational and non-relational sources via ODBC Supports data location agnostic development No special syntax to access heterogeneous data sources BW based Analytic Services on external data Implemented in BW on HANA for NLS in IQ HANA Tables Virtual Tables Scenario HANA Smart Data Access Layer Make other DWHs transparent to HANA Non-disruptive evolution from virtual table to persistent structure by establishing ETL without major effort Consolidating / rationalizing the DWH landscape Oracle MSSQL Teradata SAP HANA ASE IQ Hadoop Consumption of HANA datamart scenarios from second HANA database Optimization of NLS Queries with filters on Navigational Attributes 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 13

14 HANA Smart Data Access for BW NLS with SAP IQ BW7.40 SP08 (RTC Sept 2014) BW HANA Facts (online) Analytic Query API NLS-Split Master data API CalcScenario VIRTUAL_TABLE VirtualTable can be used in HANA CalcView Initial version w/ HANA SP07 Full plan optimizations w/ HANA SP08 BW Query execution can push OLAP calculations to HANA (Restricted KeyFigures, Hierarchies, ) Further improvement of NLS Query performance due to optimized access SAP IQ Facts (nearline) 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 14

15 Runtime Example from E-Procurement with 40% NLS plus SDA Less than 500 ms runtime with 40% NLS 669 ms drill down into NavAttr Region Reduced network traffic for NavAttr Region 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 15

19 SAP NetWeaver BW Product Roadmap Focusing BW NLS Implementation for Sybase IQ BW on HANA Query Optimizations Optimized performance for NLS read purposes (HANA SDA with BW 7.4 SP05) Supported Platforms Server/Client Sybase IQ 16.0 SP3 Sybase IQ ODBC NLS client for BW AppServer Windows on x64 64bit, Linux (x86_64 64bit), AIX 64bit, HP-UX, Solaris, OS/400 Error DTP Handling in case of NLS caused lock violations Support for Standard DSO with BW 7.3 SP11 (resp. Note ) IQ Backup Automation Event trigger for incremental backup in IQ after data relocation to NLS (BW 7.0 SP32) BW NLS Downport to BW 7.0 BW 7.0 SP32 (CW2/2014) BW 7.0 SP28 with note Today BW NLS Enhancements NLS support for BW Inventory Cubes Generated HANA Models of an InfoProvider can access data in NLS BW on HANA Query Optimizations Further improvement of NLS Query performance due to SDA optimizations (SDA Virtual Tables in HANA CalcViews) Extended Pruning for time characteristics BW NLS Implementation Support Mass maintenance Support (BW 7.40 SP 8) for generating DAPs, DAP Variants, Process Chains based on query selection statistics Further improvements on NLS operations DBA Cockpit for integrated Sybase IQ Monitoring (BW 7.40 SP8 + downports up to NW 7.02) New nearline connection mode Optimized handling of attached NLS in BW system copy or migration process to HANA Improved operations when nearline data is unavailable Planned Innovations Lookup Performance (SP10) Optimizations for lookup to archived data using the Nearline Interface API Throughput optimizations in archiving (SP11) New parallelization option copy, verification and deletion phase of a Data Archiving Process (DAP) High Performance Data Relocation through optional multi-threaded execution of the LOAD statement within the SAP IQ server NLS support for additional BW InfoProviders (planned for next BW Feature Pack) Supporting the Advanced DataStore Object BW NLS Enhancements (planned for next BW Feature Pack) Integrated handling of exceptional updates to archived nearline time slices within BW Processes (e.g. Advanced DataStore Object Activation) Future Direction 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 19

20 Key Points to take home 1. Near-line Storage helps to reduce TCO and gives seamless access to massive amounts of aged resp. inactive data to reveal trends, patterns, correlations 2. With its columnar store and compression mechanisms, SAP IQ ideally complements SAP HANA as NLS store 3. Finally NLS also supports any kind of inventory handling in BW and opens up a new category of high volume InfoProviders for NLS based volume optimization 4. SAP HANAs Smart Data Access capabilities guarantee optimized performance for NLS query access even in case of complex filter criteria. 5. The read-only restriction for NLS data can soon be temporarily suspended for exceptional update operations. 6. With the latest enhancements to their NLS solution SAP competes against all existing NLS solutions of our certified NLS Development Partners. SAP NLS is optimized for BW on HANA SAP SE or an SAP affiliate company. All rights reserved. Public 20

21 SAP BW Near-Line Storage RDS Solution scope covered The following diagram shows the scope and process flow of the solution: Install and configure SAP Sybase IQ Configure Near-Line storage Implement data aging strategy in SAP NetWeaver BW for historical data. Update support package to SAP BW 7.3x or higher Install/configure SAP Sybase IQ Configure SAP NW BW to use SAP Sybase IQ as near-line storage DAP 1 DAP N Implementation of data aging strategy in SAP NW BW for historical data Archive historical data to near-line storage Delete already archived data from online SAP NetWeaver BW system Process Repeatable result DAP = Data archiving process 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 21

22 Services Scope Overview Go live in as little as 12 weeks based on implementation options Service Name: Rapid deployment of SAP NetWeaver BW Near-Line Storage Implementation Duration: 12 Weeks* Core Offerings Implementation of SAP NetWeaver BW Near-Line Storage Identification of objects to be archived Installation and Configuration of SAP Sybase IQ database Installation of SAP Sybase IQ Client and configuration of SAP NetWeaver BW to Near-Line Storage Create up to 10 DAPs (Data Archiving Processes) Create up to 2 Process Chains Adjust settings for up to 10 Queries and 2 multiproviders Optional Task** Support Package Update * Based on scope ** Provided as option and impacts overall efforts of the service A special step-by-step guide describes each activity during the deployment 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 22

23 SAP BW NLS RDS Implementation Partners Former SAP BW NLS Development Partners also offer implementation services for SAP BW Near-line rapid-deployment solution Partner specific add-ons available on top (e.g. NLS storage management, migration options) have a look into SAP Store for more details under store.sap.com 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 23

25 NLS Interface Nearline Storage (NLS) with SAP IQ Specific features of SAP s Nearline Solution for BW based on SAP IQ InfoCubes (Advanced) DataStore Objects SAP BW SAP HANA SAP IQ Interface Available Optimized query performance when accessing NLS IQ data through HANA Smart Data Access (SDA) NLS support for BW Inventory Cubes Automated mass generation of Data Archiving Processes based on query selection statistics Generated HANA Models of an InfoProvider can access data in NLS Optimized lookup to archived data in NLS IQ using the Nearline Interface API DBA Cockpit for integrated Sybase IQ Monitoring Backup Automation: Event trigger for incremental backup in IQ after data relocation to NLS Planned High Performance Data Relocation through optional multi-threaded execution of the LOAD statement within the SAP IQ server (SP11) Integrated handling of exceptional updates to archived nearline time slices within BW Processes (e.g. Advanced DataStore Object Activation) 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 25

26 2014 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see for additional trademark information and notices. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE s or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions SAP SE or an SAP affiliate company. All rights reserved. Public 26