Session 0202: Big Data in action with SAP HANA and Hadoop Platforms Prasad Illapani Product Management & Strategy (SAP HANA & Big Data) SAP Labs LLC, Bellevue, WA
Legal disclaimer The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP. This presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation and SAP's strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this document is not a commitment, promise or legal obligation to deliver any material, code or functionality. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This document is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP s willful misconduct or gross negligence. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions. 2014 SAP AG or an SAP affiliate company. All rights reserved. 2
Agenda Overview SAP HANA Platform & Big Data SAP HANA and Hadoop Partnership Partner Hadoop Distribution s Overview Big Data Use Case Scenario Retail Industry CAR (Customer Activity Repository) - Demo 2014 SAP AG or an SAP affiliate company. All rights reserved. 3
H2 The Power of SAP HANA and Hadoop Big Data Platform Big Data Analytics & Apps Big Data Science Real Time Real Value Real Results SAP: Real-time, with real results 4
SAP makes Big Data Real & Actionable Big Data Platform Big Data Analytics & Apps Big Data Science Real Time Real Value Real Results 5
SAP HANA: Big Data Real Time Any Apps Any App Server SAP Business Suite and BW ABAP App Server SQL MDX R JSON Open Connectivity Supports any Device SAP HANA Platform Extended Application Services App Server UI Integration Services Web Server Processing Engine Development OLTP OLAP Search Text Analysis Predictive Events Spatial Rules Planning Calculators Database Services Application Function Libraries & Data Models Administration Predictive Analysis Libraries Business Function Libraries Data Models & Stored Procedures Integration Services Data Virtualization Replication ETL/ELT Mobile Synch Streaming Deployment: On-Premise Hybrid On-Demand SAP HANA platform converges Database, Data Processing and Application Platform capabilities & provides Libraries for predictive, planning, text, spatial, and business analytics so businesses can operate in real-time. 6
PR5 Big Data is Strategic to SAP (eg: Partners) http://www.cloudera.com/content/cloudera/en/so lutions/partner/sap.html http://hortonworks.com/partner/sap/ 7
Slide 7 PR5 Retain for all industries Padmini R, 10/22/2013
HDP 2.1: A Modern Data Architecture APPLICATIONS Business Analytics Custom Applications Packaged Applications DEV & DATA TOOLS Build & Test DATA SYSTEM RDBMS EDW MPP REPOSITORIES Governance & Integration Enterprise Hadoop Data Access Data Management Security Operations OPERATIONS TOOLS Provision, Manage & Monitor SOURCES OLTP, ERP, CRM Systems Documents, Emails Web Logs, Click Streams Social Networks Machine Generated Sensor Data Geolocation Data 2014 SAP AG or an SAP affiliate company. All rights reserved. 8
HDP 2.1: Enterprise Hadoop HDP 2.1 Hortonworks Data Platform GOVERNANCE & INTEGRATION DATA ACCESS SECURITY OPERATIONS Data Workflow, Lifecycle & Governance Falcon Sqoop Flume NFS WebHDFS Batch Map Reduce Script Pig SQL Hive/Tez, HCatalog NoSQL HBase Accumulo Stream Storm YARN : Data Operating System Search Solr 1 HDFS (Hadoop Distributed File System) Others In-Memory Analytics, ISV engines N Authentication Authorization Accounting Data Protection Storage: HDFS Resources: YARN Access: Hive, Pipeline: Falcon Cluster: Knox Provision, Manage & Monitor Ambari Zookeeper Scheduling Oozie DATA MANAGEMENT Deployment Choice Linux Windows On-Premise Cloud 2014 SAP AG or an SAP affiliate company. All rights reserved. 9
HDP: Open, Reliable, & Current HDP certifies most recent & stable community innovation HDP 2.1 April 2014 2.4.0 0.4 0.12.1 0.13.0 0.12.0 0.98.1 0.9.1 0.9.0 4.8.0 1.4.5 1.4.0 0.5 1.5.1 4.0.0 0.4 HDP 2.0 October 2013 HDP 1.3 May 2013 2.2.0 1.1.2* Hadoop &YARN Tez 0.12.0 0.11 Pig 0.11.0 Hive & HCatalog 0.96.0 0.94.6 HBase Storm 0.8.0 0.7.0 Mahout Solr 1.4.4 1.4.3 Sqoop 1.3.1 Flume Falcon 1.4.1 1.2.3 Ambari 3.3.2 Oozie 3.4.5 Zookeeper Knox Data Management Data Access Governance & Integration Hortonworks Data Platform Operations Security 2014 SAP AG or an SAP affiliate company. All rights reserved. 10
Cloudera Powered by Hadoop 2014 SAP AG or an SAP affiliate company. All rights reserved. 11
Cloudera s Enterprise Data Hub Integration with Over 200 ISVs Self-Service BI Data Exploration Visualization Enterprise Data Hub Flexible Deployment Options On-Premise or Cloud Appliances Engineered Systems Powerful Security Solution Risk Analysis Fraud Prevention Compliance Infinite Analytic Storage Multi-Structured Data In-place Analytics Active Archive Advanced Analytics Engine 360 Customer View Recommendation Engines Processing & Analytics Improve IT Operations ETL Acceleration EDW Rationalization Mainframe Offload 2014 SAP AG or an SAP affiliate company. All rights reserved. 12
Cloudera: Hadoop and The Enterprise Data Hub Open Source Scalable Flexible Cost-Effective Managed CLOUDERA S ENTERPRISE DATA HUB BATCH PROCESSING ANALYTIC SQL SEARCH ENGINE MACHINE LEARNING WORKLOAD MANAGEMENT STREAM PROCESSING 3 RD PARTY APPS DATA MANAGEMENT Open Architecture Secure and Governed STORAGE FOR ANY TYPE OF DATA UNIFIED, ELASTIC, RESILIENT, SECURE Filesystem Online NoSQL SYSTEM MANAGEMENT 2014 SAP AG or an SAP affiliate company. All rights reserved. 13
SAP HANA Platform - Partner Hadoop Distributions SOURCES ERP Apps Mobile Apps Mobile Apps SAP Analytics Sensor Geo Logs Text Structured Weather Social Data Acquisition, Ingestion & Provisioning SAP HANA PLATFORM In-memory processing platform for real-time transactions + end-to-end analytics Application Development Extended Application Services Processing Engine HANA Processing Engine Database Services Application Function Libraries & Data Models (OLTP + OLAP) Application Function Libraries & Data Models Integration Services Unified Administration HANA Other Other Partner Partner Distributions Other Hortonworks Data Platform (HDP) 2014 SAP AG or an SAP affiliate company. All rights reserved. 14
CAR Customer Activity Repository: Demo Story A Product Manager checks merchandise categories regularly. They use Lumira visualization tool to identify High / Low performing categories In this demo they are: Looking at a sales report for products in the shoe category They see total sales for the current year as well as the High / Low performers in this category The need is to explore the low performers and look for trends over time to see if / how to the issue can be corrected the results for the 3 Low performers over the last 5 years is examined Examining other categories of in conjunction can reveal if the trends seen are isolated to the low preforming categories or generalize across categories indicating more global factors such as global economy, global climate change etc. 2014 SAP AG or an SAP affiliate company. All rights reserved. 15
CAR (Customer Activity Repository): Retail Scenario Correlate with other categories Examine historical trends Discover High / Low performers Under performing: Flats Tall Boots Leather bags Shoe Category Challenges 2014 SAP AG or an SAP affiliate company. All rights reserved. 16
Car Landscape - SAP HANA (Cell 2.0) Consume Store & Process Ingest SAP HANA Platform Analytics Exploration, Dashboards, Reports, Charting, Visualization HANA In Memory Planning & Simulation Graph Spatial Consume Process ESP Transactional Analytical Extended Storage (IQ) Text, Social Media Processing Replication Framework Applications Machine Learning & Predictive Smart Data Access Data Services Native HANA Apps & Services Tiered Storage (Hot-warm-cold) IM H A D O O P Cell 2.0 Spare-03 Arista 7048 (48x1G) Spare-04 CFM-01 CFM-02 Peristent Flash Memory Node-29 Node-27 Node-25 Node-23 Node-21 Node-19 Node-17 Node-15 Node-13 Node-11 Node-09 Node-07 Node-05 Node-03 Node-01 H A N A 40G Switch 40G Switch Node-30 Node-28 Node-26 Node-24 Node-22 Node-20 Node-18 Node-16 Node-14 Node-12 Node-10 Node-08 Node-06 Node-04 Node-02 2014 SAP AG or an SAP affiliate company. All rights reserved. 17
HANA Cell 2.0 Key Features Technology innovations bring dramatic benefits to the HANA Cloud Platform Compute 8 Nodes (4 sockets Ivybridge-EX per node) 32 CPU sockets (480 cores/960 Threads) 3TB DRAM per node, 24TB of DRAM per rack 3x memory capacity compared to Cell 1.0 Shared Persistence High Performance flash memory 100-500 TB of shared flash Switches & Networks PCIe3.0 (upto 384 Gb/ per node) 40Gb Ethernet (80 Gb/s per node) 2.9 Tb/s Bisection Bandwidth Cloud management software SAP Cloud Frame Manager for infrastructure lifecycle management 2014 SAP AG or an SAP affiliate company. All rights reserved. 18 HANA Network SAP Cloud Management Compute Memory C M Flash Array (S) C M C M Storage Network CELL 2.0
CAR Demo - Data Volume 1000 Stores Total Records Total Transactions Total Line items Total Line Items records Date Range HANA 4,560,258,738 518,207,094 1,554,651,590 4,042,051,644 Jan 2013 - Jan 2014 HADOOP 14,590,124,890 1,696,526,316 5,089,558,899 12,893,598,493 Jan 2008 - Dec 2012 2014 SAP AG or an SAP affiliate company. All rights reserved. 19
CAR Demo Calc Views: Z_POSLOG_QUERY Join on Material Group to Get Merchandise Category Name Join on RetailLocation to Get Store Attributes Bring Together POS Data from Hortonworks and Hana Merge this year with last year in HANA Retrieve HANA POS Transactions Data Type conversion from Hadoop to CAR and POS Transactions retrival from Hortonworks 2014 SAP AG or an SAP affiliate company. All rights reserved. 20
Demo: CAR 1. Integrated Lumira Analysis: Category managers and Marketing colleagues want to tackle declining sales with insights into the merchandise category. They must leverage transactional data and look for trends across multiple years to better understand trends, plan what to buy and organize upcoming promotions and campaigns. 2. Category Analysis Current and Historic Data: As category managers analyze performance of the merchandise categories with real-time insights, they need to understand trends and performance of categories in order to achieve targets and find new ways to add lift and exceed budget. 2013 SAP AG or an SAP affiliate company. All rights reserved.
Key Takeaways SAP HANA platform for big data Partner Hadoop distribution platform overview Retail industry use case scenario for big data 2014 SAP AG or an SAP affiliate company. All rights reserved. 22
Further Information Experience SAP Big Data http://www.sapbigdata.com/ SAP HANA and Hadoop http://www.sap.com/solution/big-data/software/overview.html Hortonworks Data Platform www.hortonworks.com Hortonworks & SAP www.hortonworks.com/partners/sap Cloudera Enterprise Hadoop http://www.cloudera.com/content/cloudera/en/home.html Cloudera & SAP http://www.cloudera.com/content/cloudera/en/solutions/partner/sap.html 2014 SAP AG or an SAP affiliate company. All rights reserved. 23
2014 SAP AG or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG or an SAP affiliate company. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG (or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP AG or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP AG or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP AG or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP AG or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP AG s or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP AG or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forwardlooking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions. 2014 SAP AG or an SAP affiliate company. All rights reserved. 24