SAS FOR BIG DATA PRESENTED BY: BRAD HATHAWAY

Transcription

1 SAS FOR BIG DATA PRESENTED BY: BRAD HATHAWAY

2

3 SAS AND BIG DATA SOME KEY TAKEAWAYS FROM THE VIDEO Combining Big Data and Analytics Hadoop allows capturing unlimited amounts of diverse data many companies are using this to create a Data Lake Extracting value from the lake requires analytics which makes SAS a natural complement to Hadoop One thing the video didn t mention: the longer the data stays in the Data Lake, the better your performance and overall experience will be. It is critical to have as much processing in Hadoop as possible.

4 SAS BUSINESS ANALYTICS FRAMEWORK... GIVES SAS CUSTOMERS THE POWER TO KNOW! Each area is a market on its own! SAS is ranked as a leader in pretty much all of them! Our customers are now shifting their attention to how each of these areas interact with Hadoop!

5 AGENDA What is Hadoop? (a quick refresher) Two Hadoop Approaches Data Platforms with Hadoop BI & Analytics on Hadoop SAS on Hadoop a taste of technology Data Quality Accelerator on Hadoop Self-Service DI on Hadoop SAS Visual Statistics

6 WHAT IS HADOOP? A QUICK REFRESHER

7 WHAT IS HADOOP? DICTIONARY DEFINITION Hadoop is one way of using a set of cheap computers to store an enormous amount of data and then to process that data in parallel."

8 WHAT IS HADOOP? MAKING HADOOP EASY AND ENTERPRISE READY

9 Total Cost WHAT IS HADOOP? AS A DATA PLATFORM, STORAGE COSTS ARE MUCH LOWER $ ,00 $ ,00 $ ,00 $ ,00 $ ,00 $ ,00 $ ,00 Hadoop Teradata Warehouse Appliance Oracle Exadata IBM Netezza $ ,00 $ ,00 $0, Number of Gigabytes

10 WHAT IS HADOOP? PROJECTS FOR THE HADOOP STACK

11 SUPPORTING EVIDENCE THE TREND IS UP! Source: SandHill Group, Do You Hadoop? A Survey of Big Data Practitioners October 29, 2013

12 TWO HADOOP APPROACHES

13 TWO STARTING POINTS NOT MUTUALLY EXCLUSIVE BUT OFTEN NOT SEEN TOGETHER! Hadoop as a Data Platform (standalone or as part of a broader ecosystem) Hadoop as a core component of the next generation of BI and Analytics EVALUATE / MONITOR RESULTS IDENTIFY / FORMULATE PROBLEM DATA PREPARATION DEPLOY MODEL DATA EXPLORATION VALIDATE MODEL BUILD MODEL TRANSFORM & SELECT.. to support an IT Transformation.. to support innovative business usage

14 DATA PLATFORMS WITH HADOOP

15 WHERE WE ARE TODAY? SETTING THE SCENE Operational Data Sources Operational Data Sources: Traditional sources include ERP, CRM and financial systems amongst others. Evolving sources that include unstructured data from places like Twitter, LinkedIn etc. and streaming data from the Internet of Things (sensors etc.)

16 WHERE WE ARE TODAY? SETTING THE SCENE BI and Analytics Operational Data Sources Data Mart Data Mart EDW Analytic Mart Analytic Mart Unstructured, Semi-structured and Streaming data (i.e. sensor data) often handled outside the Warehouse flow

17 WHERE DOES HADOOP FIT? HADOOP AS A NEW DATA STORE BI and Analytics Operational Data Sources Data Mart Data Mart EDW Analytic Mart Analytic Mart

18 WHERE DOES HADOOP FIT? HADOOP AS AN ADDITIONAL INPUT TO THE EDW BI and Analytics Operational Data Sources Data Mart Data Mart EDW Analytic Mart Analytic Mart Analytic Mart Data Mart

19 WHERE DOES HADOOP FIT? HADOOP DATA PLATFORM AS A BASIS FOR BI AND ANALYTICS BI and Analytics Operational Data Sources EDW Analytic Mart Data Mart Data Mart Data Mart Analytic Mart Analytic Mart

20 WHERE DOES HADOOP FIT? HADOOP DATA PLATFORM AS A STAGING LAYER AS PART OF A DATA LAKE Downstream stores could be Hadoop, data appliances or an RDBMS Operational Data Sources EDW BI and Analytics Data Mart Data Mart Analytic Mart Analytic Mart

21 HIGH LEVEL VIEW WHAT YOU CAN DO WITH SAS AND HADOOP WHEN IT COMES TO USING HADOOP AS A DATA PLATFORM Today... Coming very soon BASE SAS Map Reduce + Pig Scripting + HDFS Commands SAS/Access to Hadoop Hive, Hive2 + Direct file access SAS/Access to Impala (Cloudera only) SAS Data Integration Studio (Transforms) in Data Management Standard / Advanced: Read/Write HDFS files Submit HiveQL code Execute Map/Reduce code Submit Pig Latin Transfer data to/from Hadoop using Hadoop utilities SQL transforms pushed down with Access to Hadoop engine SAS Federation Server Virtual and secure access to Hadoop and more traditional sources Everything we have today plus... SAS Data Quality Accelerator for Hadoop - Execute selected DQ routines in Hadoop SAS Code Accelerator for Hadoop - Execute SAS DS2 code in Hadoop New Web Based Business User Interface Point and click data management routines where data stays in Hadoop HTML 5 Web based interface SAS Event Stream Processing Engine To bring streaming data from Sensors into Hadoop

22 BI & ANALYTICS ON HADOOP

23 WHEN IT COMES TO BI / REPORTING TWO SIMPLE THINGS TO REMEMBER Data for data visualization, and reporting sourced from Hadoop but the actual visualization / reporting is not running on Hadoop Hadoop cluster processors used for data visualization, exploration and reporting SAS/Access just like we do with an RDBMS A In-Memory B More or less business as usual Transformational

24 WHEN IT COMES TO BI / REPORTING WHAT YOU CAN DO WITH SAS AND HADOOP WHEN IT COMES TO USING HADOOP AS PART OF BI Data for data visualization, and reporting sourced from Hadoop but the actual visualization / reporting is not running on Hadoop Hadoop cluster processors used for data visualization, exploration and reporting Any SAS BI Product: SAS Visual Analytics SAS Office Analytics SAS Enterprise Guide SAS BI/EBI Server SAS Stored Processes and batch programs for reporting A In-Memory Exploration, Visualization & Reporting SAS Visual Analytics B

25 WHEN IT COMES TO ANALYTICS THREE SIMPLE THINGS TO REMEMBER Data for Analytics sourced from Hadoop but no Analytics running on Hadoop Hadoop cluster processors used for Analytical Computation Analytics deployed for batch execution in Hadoop Think SAS/Access just like we do with an RDBMS Think In-Memory Analytics Think In-Database just like with an RDBMS C D E More or less business as usual Transformational Operational

26 WHEN IT COMES TO ANALYTICS WHAT YOU CAN DO WITH SAS AND HADOOP WHEN IT COMES TO USING HADOOP AS PART OF ANALYTICS Data for Analytics sourced from Hadoop but no Analytics running on Hadoop Hadoop cluster processors used for Analytical Computation Analytics deployed for batch execution on Hadoop Any SAS Analytics Product: SAS Enterprise Miner SAS Forecast Server SAS/STAT etc. C D E In-Memory Interactive Analytics SAS Visual Statistics SAS In-Memory Statistics for Hadoop Operational Analytics SAS Scoring Accelerator for Hadoop SAS Code Accelerator for Hadoop

27 THE ANALYTICS LIFECYCLE STRATEGY: ENABLE THE ENTIRE LIFECYCLE ON HADOOP SAS Visual Analytics EVALUATE / MONITOR RESULTS IDENTIFY / FORMULATE PROBLEM DATA PREPARATION SAS Visual Analytics SAS Visual Statistics SAS In-Memory Statistics for Hadoop SAS Scoring Accelerator for Hadoop SAS Code Accelerator for Hadoop DEPLOY MODEL DATA EXPLORATION VALIDATE MODEL TRANSFORM & SELECT Done using either the Data Preparation, Data Exploration or Build Model Tools Done using the Build Model Tools and other checks BUILD MODEL SAS High Performance Analytics Offerings supported by relevant clients like SAS Enterprise Miner, SAS/STAT etc.

28 SAS ON HADOOP A TASTE OF TECHNOLOGY

29 SAS DI STUDIO FLOW INCLUDING HADOOP DATA HADOOP Access Hadoop SAP Combine with other data, Transform & Load TERADATA ORACLE SAS FEDERATION SERVER SAS DB2

30 SAS DI STUDIO MANAGE DATA IN HADOOP STANDALONE Access data in Hadoop Transform data inside Hadoop using HiveQL Creating new data in Hadoop

31 DATA MANAGEMENT FOR HADOOP HIGH PERFORMANCE IN-HADOOP DATA PROCESSING SAS Servers Hadoop Cluster MapReduce, Pig, HiveQL SAS Code Accelerator SAS Data Quality Accelerator Harness the power of the Hadoop distributed platform, big data, and SAS data management capabilities High performance in-database processing Native capabilities (HiveQL, Pig, MR) + Value-Added capabilities SAS Code Accelerator SAS Data Quality Accelerator Embedded into Hadoop HDFS / Raw Files SAS LEVERAGES HADOOP FOR MAXIMUM PERFORMANCE

32 DATA MANAGEMENT FOR HADOOP SELF-SERVICE DATA QUERY AND TRANSFORMATION New SAS Web-Based Business User Interface Users are able to manage big data Query, Select, Filter, Summarize & Transform data Use data quality Load data into SAS LASR Hadoop Cluster HiveQL, Pig, MapReduce SAS Code Accelerator SAS Data Quality Accelerator Feature preview:

33 SAS DATA DIRECTOR

34 SAS DATA DIRECTOR

35 SAS DATA DIRECTOR

36 SAS DATA DIRECTOR

37 SAS DATA DIRECTOR

38 SAS DATA DIRECTOR

39 SAS DATA DIRECTOR

40 SAS DATA DIRECTOR

41 SAS VISUAL STATISTICS 6.4 EXTENDING SAS VISUAL ANALYTICS FOR MORE ANALYTIC CONTROL AND TARGETED ACTIONS

42 Advanced Modeling Techniques

43 IN SUMMARY SAS BUSINESS ANALYTICS FRAMEWORK... GIVES OUR CUSTOMERS THE POWER TO KNOW! SAS does this with support of Hadoop in all core areas this is unique to SAS! Any business use case you can think of will need all of these!

44 THANK YOU!