Bruce Labbate Non-SAP Data Warehousing in SAP HANA Session 2897

Transcription

1 Bruce Labbate Non-SAP Data Warehousing in SAP HANA Session 2897

2 INTRODUCTION Bruce Labbate Decision First Technologies Business Intelligence, EIM, and HANA expertise SAP Gold Partner 7x Business Objects Partner of the Year

3 LEARNING POINTS Understand the options for loading data into HANA on an enterprise scale Explore the differences between Data Modeling and Analytic Modeling Learn how to optimize Data Modeling and Analytic Modeling on the HANA platform

4 RETURN ON INVESTMENT Best practices to minimize development time Faster and lighter models to more efficiently leverage your HANA investment Extensive and responsive analytics allowing swifter and smarter decision-making

5 AGENDA What is HANA? Loading Data into HANA Physical Data Modeling vs Analytic Modeling Optimizing Physical Data Modeling in HANA Optimizing Analytic Modeling in HANA General Best Practices Information View-Specific Best Practices Visualization

6 WHAT IS HANA? The Basics High-Performance ANalytic Appliance Hardware and Software DBMS In-Memory, Compressed, Columnar Table Store Tables, views, stored procs, etc ACID compliant Multi-tenancy Hybrid Transactional/Analytical Processing Join Engine OLAP Engine Calculation Engine

7 WHAT IS HANA? HANA is a development and modeling platform Information Views Text Analysis Geospatial Modeling Graph Analysis Predictive Analysis Smart Data Integration

8 AGENDA What is HANA? Loading Data into HANA Physical Data Modeling vs Analytic Modeling Optimizing Physical Data Modeling in HANA Optimizing Analytic Modeling in HANA General Best Practices Information View-Specific Best Practices Visualization

9 LOADING DATA INTO HANA Enterprise data load requirements Robust High data volume Error handling and recovery Fast DBMS-specific optimization High throughput Flexible Transformations and filtering Scripting Conditional logic

10 LOADING DATA INTO HANA SLT Great for SAP ECC replication Limited ABAP-based transformation Non-SAP replication is possible Limited DBMS support Strict value and type requirements SAP Note

11 LOADING DATA INTO HANA SAP Data Services Extensive native DBMS support Web services, flat files, xml, and custom adapters Robust transformation capabilities HANA-specific optimization SQL pushdown Bulk upsert

12 LOADING DATA INTO HANA HANA Smart Data Integration Native to HANA Leverages Smart Data Access Allows real-time replication Brand new, not yet mature Not recommended yet Good information in the SAP HANA Developer Guide at

13 AGENDA What is HANA? Loading Data into HANA Physical Data Modeling vs Analytic Modeling Optimizing Physical Data Modeling in HANA Optimizing Analytic Modeling in HANA General Best Practices Information View-Specific Best Practices Visualization

14 PHYSICAL DATA MODELING vs ANALYTIC MODELING Physical Data Modeling Physical database tables, views, and indexes Persistent transformed data, sometimes duplicated Concerned with data storage, partitioning, and data types and sizes Generated by ETL Analytic Modeling Logical views and structures Non-persistent datasets, minimal data duplication Concerned with memory usage Entirely within HANA Both are necessary in HANA Complementary and interrelated

15 AGENDA What is HANA? Loading Data into HANA Physical Data Modeling vs Analytic Modeling Optimizing Physical Data Modeling in HANA Optimizing Analytic Modeling in HANA General Best Practices Information View-Specific Best Practices Visualization

16 OPTIMIZING PHYSICAL DATA MODELING Star Schema is still the core model for analysis No need to create indexes Primary keys indexed by default Secondary indexes rarely helpful Must be highly selective and narrow Index Advisor $DIR_INSTANCE/exe/python_support/indexAdvisor.py Avoid surrogate keys Use unique, single-column natural keys instead if possible

17 OPTIMIZING PHYSICAL DATA MODELING Keep tables narrow More columns create more expensive INSERTs Wider tables have lower throughput Choose efficient data types Smallest appropriate type E.g. DATE vs TIMESTAMP Avoid character-based join columns Dynamic Tiering Keep hot data in memory Warm data on disk

18 OPTIMIZING PHYSICAL DATA MODELING Partitioning Types Round-robin Hash Range Date-based aging Multi-level Secondary level relaxes key column restriction E.g. Hash-Range Partitioning Consider ~3 partitions per node Improves delta merge Hash-Range Partitioning

19 AGENDA What is HANA? Loading Data into HANA Physical Data Modeling vs Analytic Modeling Optimizing Physical Data Modeling in HANA Optimizing Analytic Modeling in HANA General Best Practices Information View-Specific Best Practices Visualization

20 OPTIMIZING ANALYTIC MODELING - GENERAL Joins Can be very expensive Optimize join columns Avoid NULLs, character fields, calculated columns Ensuring matching data types Referential Join Assumes referential integrity! If RI is not intact, results may be inconsistent Text Join Requires Language column Consider replacing with Unions

21 OPTIMIZING ANALYTIC MODELING - GENERAL Avoid calculated columns Require calculation engine Can delay joins Consider auto-generated columns Calculation in DB instead of at run-time Reduce data size as early as possible Filter at the lowest level Prune columns Transfer minimal data between views/engines Don t cross engines!

22 OPTIMIZING ANALYTIC MODELING - GENERAL Build incrementally Test and backup often Reworking views is time-intensive Don t be afraid to edit views in XML Flatten hierarchies and pivot in ETL When in doubt, Visualize Plan Exposes detailed plan with execution times and data sizes Shows engine usage

23 OPTIMIZING ANALYTIC MODELING VIEWS Attribute Views Analytic Views Calculation Views

24 OPTIMIZING ANALYTIC MODELING VIEWS Attribute Views Join Engine Dimensions Few tables Denormalized Keep as simple as possible Evaluated and flattened when included in views

25 OPTIMIZING ANALYTIC MODELING VIEWS Analytic Views OLAP Engine Star Schema/Cubes Avoid complex or high-volume joins Use calculated attributes/columns sparingly Stay out of the calculation engine Consider restricted measures instead Calculate after aggregation

26 OPTIMIZING ANALYTIC MODELING VIEWS Calculation Views Calculation Engine Can I avoid via ETL? Use projection nodes for every source Immediate pruning and filtering Easy to insert nodes Avoid joins between analytic views/fact tables Use union node with constant values and aggregate Move attribute joins to analytic views

27 OPTIMIZING ANALYTIC MODELING VIEWS Calculation Views Scripted Calculation Views CE_* Often faster than graphical Optimize and parallelize well Difficult to write efficiently DO NOT Mix with plain SQL Use cursors Use dynamic SQL

28 AGENDA What is HANA? Loading Data into HANA Physical Data Modeling vs Analytic Modeling Optimizing Physical Data Modeling in HANA Optimizing Analytic Modeling in HANA General Best Practices Information View-Specific Best Practices Visualization

29 VISUALIZATION Lumira Simple, quick, and pretty Build charts, graphs, clouds, maps, and more Combine with text, pictures, etc into documents and stories Easy to share datasets and visualizations Connects to live HANA data or static datasets

30 VISUALIZATION Web Intelligence Requires Universe layer Additional development Reusable Provides semantic abstraction, filtering, parameters Do not join HANA views Self-service power user reporting tool Web-based or desktop Robust matrix analysis with visualizations

31 VISUALIZATION Explorer Self-service data discovery Slice and dice deep into datasets View exposed data in visualizations and charts Connect directly to HANA Design Studio Powerful HTML5 dashboards Extensible and customizable Connect directly to HANA

32 KEY LEARNINGS HANA requires a combination of physical data modeling and analytic modeling for best performance Start with star schema and tweak Don t ignore partitioning Be careful with joins Stay in one engine, preferably not calculation Choose the right visualization tool to expose data

33 STAY INFORMED Follow the ASUGNews team: Tom Chris Craig

34 SESSION CODE 2897