Predictive Analytics for the Non-Data Scientist: What You Really Need to Know. Ashish C. Morzaria SAP

Transcription

1 Predictive Analytics for the Non-Data Scientist: What You Really Need to Know Ashish C. Morzaria SAP

2 Who Is This Webinar For? Do you know what these terms mean? Triple Exponential Smoothing Simple Linear Regression K-Means Analysis This session is really focused on: Business analysts, data analysts, IT people Basically everybody but the Data Scientists! Understanding the basic concepts to demystify Advanced Analytics Bringing advanced capabilities to the rest of us 2015 SAP SE or an SAP affiliate company. All rights reserved. 1

3 Why an Advanced Analytics Session for BI People? The problem is most BI customers don t focus on Advanced Analytics We are still dealing with the basics of BI first (or upgrading to BI 4.1) We need to get a handle on our existing data before we create more! We don t have any Data Scientists and our BI people can t do it. Only 21% of organizations have fully or partially implemented predictive analytics. 61% are still exploring or have no plan! 2015 SAP SE or an SAP affiliate company. All rights reserved. 2

4 The Big Data Problem Web, mobile, social & machine generated data explosion Faster Decision Cycles Advanced Analytics Skills Gap Demand for deep analytical talent in the United States could be 50 to 60% greater than its projected supply by 2018.

5 The Solution: Automated Algorithmic Analysis Descriptive BI Diagnostic BI Predictive BI Prescriptive BI Describe Understand Predict Recommend A human using a computer A computer providing info to a human

7 Spectrum of Users for Analytics Data Scientist Data Analysts Executives/ Business Users Create complex predictive models and simulations Validate predictive business requirements Publish results back to source Transform and enrich data source(s) Create simple predictive models and simulations Visualize results and publish to BI Platform Interact with published predictive analysis Visualize results in context of use case Collaborate with colleagues toward closure/action.1% ~3% 97% Representative User Base 2015 SAP SE or an SAP affiliate company. All rights reserved. 6

8 Specialization Today, Predictive Analytics Is an Island ETL LOB Chasm PM PA DM ETL Cleaned Data EIM Raw Data Reporting Analysis BI Discovery Dashboards Sophistication / Skill Set There is a disconnect between the BI and Advanced Analytics world 2015 SAP SE or an SAP affiliate company. All rights reserved. 7

9 Specialization Bringing Predictive Analytics to Business Users Is Key ETL LOB Automated Predictive PM PA DM Cleaned Data EIM Raw Data Reporting Analysis BI Discovery BI Dashboards Invisible Analytics in BI Sophistication / Skill Set 2015 SAP SE or an SAP affiliate company. All rights reserved. 8

14 Predictive Analytics: Very Visible Results Make more money: Walmart discovered prior to hurricanes, customers bought flashlights, batteries and Pop-Tarts 1 Best Buy discovered 7% of its customers account for 43% of its sales 1 Reduce costs: A major Canadian bank: Increased campaign response rates by 600% Cut acquisition cost by 50% Boosted ROI by 100% 2 A European telecom reduced customer churn from 20% to 5% using predictive analysis 1 Airlines better estimate the number of passengers who won t show up for a flight 2 Save Lives: Health care: finding emerging symptoms for Ebola before the pattern is obvious to the naked eye Route optimization to reduce response times: how much is 5 minutes worth to a dying patient? 1 The Economist, The Data Deluge, Data, data everywhere, February 27,2010, pages Wayne W. Eckerson, Predictive Analytics: Extending the Value of Your Data Warehousing Investment, TDWI Best Practices Report, 2007, page SAP SE or an SAP affiliate company. All rights reserved. 13

15 Advanced Analytics*: An Introduction * Advanced Analytics and Predictive Analytics will be used relatively interchangeably in this presentation 2015 SAP SE or an SAP affiliate company. All rights reserved. 14

16 What Is Predictive Analysis? High level: Using algorithmic analysis to recognize data relationships that influence likely outcomes and identify potential risks and opportunities before they occur to make better decisions in the future. Approximate the relationship between variables and their outputs and represent it as an algorithm (set of rules) Use the algorithm against future data to predict the response with the least amount of error In the BI context: PA is another toolset that helps us uncover patterns and relationships algorithmically that can be used against future data sets instead of relying solely on visual representations that require the analyst to infer the future 2015 SAP SE or an SAP affiliate company. All rights reserved. 15

17 How Does Predictive Analysis Work? Trivial Example: Analysis of an entire year s worth of store sales data reveals the following patterns: Soda is bought 40% of the time when potato chips are purchased Purchases of umbrellas increase 125% during the months of March and April When baby formula is purchased by a male: 54% of the time, so is at least one case of beer As a store manager, what would you do with this data? Promote sales of soda pop in the same isle as potato chips (Cross sell) Increase stock of umbrellas in the spring (Forecasting) Put the beer closer to the baby isle (Trending) SAP SE or an SAP affiliate company. All rights reserved. 16

19 What s a Predictive Model? A predictive model algorithmically represents the desired target within the dataset. This model can then be applied to other (or future) datasets to identify elements that should be targeted. Predictive Models: This last transaction could be fraudulent Descriptive Models: Male purchasers of baby formula are 56% likely to also buy a case of beer, but female purchasers are likely to have a higher number of items and a larger total bill Decision Models: Your credit score is pretty low and you have few assets, giving you a mortgage would be pretty risky 2015 SAP SE or an SAP affiliate company. All rights reserved. 18

20 What Is the Predictive Process? How much time is typically spent in each area? Business understanding: 5 15 percent Data understanding: 5 10 percent Data preparation: percent Modeling: 5 15 percent Evaluation: 5 10 percent Deployment: percent The secret: Good analytics require good data. Any problems in the data can directly affect the (accuracy of) predicted outcomes SAP SE or an SAP affiliate company. All rights reserved. 19

21 How Are Models Created? A model can be created by a Data Scientist: Requires strong understanding of the data as every observation/assumption defines the model Resulting model can be validated by applying to historical data to determine Predictive Power SAP Predictive Analytics Automated Algorithms are adaptive and are self-training Model can be validated by applying against the testing set of historical data to determine Predictive Power System iterates continuously until predictive power is high enough Constant validation to determine when model needs to be modified (Model Manager) Data Scientists can typically create better models: A trained Data Scientist who has spent days or weeks creating and validating models can have a more logical model because it is based on semantic understanding of the data A programmatically created model relies on algorithmic pattern recognition, which typically cannot have semantics added afterwards BUT: It takes them *much* longer 2015 SAP SE or an SAP affiliate company. All rights reserved. 20

22 Where Do We Get The Information For Predictive? Information about who they are: Profile information Location Friends / Associations Comments / Feedback Interaction history Comments from calls Information about their behavior: What they searched for What they put in their cart What they actually bought When they bought How they bought How much they paid What else they bought What do they buy regularly? 2015 SAP SE or an SAP affiliate company. All rights reserved. 21

24 Name Gender Age Marital Recent Activity C-Sat Renewed Before Churn Predictive Modelling Actual Churn Historical Data Predicted Churn Training Set Smith, John M 45 M Y 6.7 Y N Brown, Erin F 23 S N 4.5 N Y Jones, Kim F 36 D N 9.0 Y N Testing Set Freely, IP M 56 M Y 8.5 N N? Howe, Jim F 23 S N 3.2 N Y? Hack, Bob M 36 M Y 7.6 Y N? Model Algorithms CSat < 5.2 Freely, IP M 56 M Y 8.5 N N N Howe, Jim F 23 S N 3.2 N Y Y Hack, Bob M 36 M Y 7.6 Y N Y Recent Activity = N Recent Activity = Y Churn=Y 78% Churn=N 22% Churn = Y 30% Actual Churn Accurate Predictions Total Records = Accuracy Rate

26 Name Gender Age Marital Recent Activity C-Sat Renewed Before Predicted Churn Batch Scoring NEW Data (Current Customers) Hancock, John M 38 D Y 4.2 N? Doe, Jane F 45 M Y 9.4 N? Red, Simply F 18 S N 2.1 N? Significantly increase ROI through dataset reduction: Lower campaign costs by targeting those most likely to leave Increase response rate by targeting even more specifically on other attributes Increase C-Sat by not hassling loyal customers Model Customer not expected to churn, so don t bother them! Hancock, John M 38 D Y 4.2 N Y Doe, Jane F 45 M Y 9.4 N N Red, Simply F 18 S N 2.1 N Y Hancock, John M 38 D Y 4.2 N Y Red, Simply F 18 S N 2.1 N Y Targeted List

27 Name Gender Age Marital Recent Activity C-Sat Renewed Before Churn Real-Time Scoring NEW Data (Single Customer) Hancock, John M 38 D Y 4.2 N? Real-time scoring enables automated decision-making: Make an offer while customer is still on the phone/web site Proactively address situations by recognizing patterns (i.e. shopping carts) Cross sell by offering items that are highly likely to be purchased Model Hancock, John M 38 D Y 4.2 N Y Special Offer: Renew now and save 25%!

28 Embedding Predictive Analytics Into BI Workflows Model Embedded as Stored Procedure Customer Database Hancock, John M 38 D Y 4.2 N Y SQL Doe, Jane F 45 M Y 9.4 N N Business Users can get on-the-fly scoring without even knowing they are using predictive algorithms Red, Simply F 18 S N 2.1 N Y Lumira Dataset w/ Scoring Lumira Server for Teams Lumira Storyboard Lumiar Server for BI Platform SAP Lumira Cloud

30 Shhh! The Secret to Advanced Analytics for BI People You don t need to know HOW the algorithms work, but you do need to know WHAT they are for * Advanced Analysis Descriptive Algorithms Predictive Algorithms Clustering Association Classification Time Series * Actually, it does help to know how they work, but it is not absolutely required to get started 2015 SAP SE or an SAP affiliate company. All rights reserved. 29

32 SAP Solutions for the Entire Spectrum of Users No Low High Level Of Skill Set - Analytics Business User Data Analyst Data Scientist HANA Application Developer Embedded Analytics Industry & Business Process Analytics Custom Analytics Application Embedded PA SAP Predictive Analytics Lumira Automated Analytics Expert Analytics Application Function Modeler PAL, APL SAP ANALYTICS R Integration 2015 SAP SE or an SAP affiliate company. All rights reserved. 31

33 SAP Solutions for the Entire Spectrum of Users No Low High Level Of Skill Set - Analytics Business User Data Analyst Data Scientist HANA Application Developer Embedded Analytics Industry & Business Process Analytics Custom Analytics Application Embedded PA Lumira Automated Analytics Expert Analytics Application Function Modeler PAL, APL SAP ANALYTICS R Integration 2015 SAP SE or an SAP affiliate company. All rights reserved. 32

34 SAP Predictive Analytics: Expert Analytics Self-Service for Business Analysts and Data Scientists Provide Data Scientists and Business Analysts with sophisticated algorithms to take the next step in understanding their business and modeling outcomes Perform statistical analysis on your data to understand trends and detect outliers in your business Build models and apply to scenarios to forecast potential future outcomes Breadth of connectivity to access almost any data Optimized for SAP HANA to support huge data volumes and in-memory processing 2015 SAP SE or an SAP affiliate company. All rights reserved. 33

35 SAP Predictive Analytics: Automated Analytics Data Scientist in a Box Provide Business Analysts and Data Scientists with a fully automated process Data preparation Create 1000s of derived attributes Define metadata once Builds analytic dataset automatically Predictive modeling/data mining Regression/Classification Segmentation Forecasting Association rules Social Network Analysis Advanced model deployment and management 2015 SAP SE or an SAP affiliate company. All rights reserved. 34

36 Predictive Analytics Solutions from SAP Any Data Source Relational Databases Big Data Sources CRM/ERP Stores Applications Cloud Services SAP Predictive Analytics 2.x Data Preparation Visualization Automated Analysis SDK/API Model Management Scoring Social Recommendation Expert Analysis Connectors SAP HANA In-Memory Processing Engine Predictive Applications 25+ Industries 11+ LoBs R Transactions Predictive Analysis Library Automated Predictive Library R-Scripts Financial & Insurance Services Retail & Consumer Products Telecommunications Public Sector & Healthcare O&G, Manufacturing & Utilities Cloud / On-Premise 2015 SAP SE or an SAP affiliate company. All rights reserved. 35

38 Detecting Fraud from auto insurance data Past Auto Claims with a Yes/No Fraud flag New Claims Train Model Apply Model Model equation New Claims Fraud Scores 2015 SAP SE or an SAP affiliate company. All rights reserved. 37

40 Where to Start? SAP Predictive Analytics is available for 30-day Trial! What questions are you trying to answer for your company? Where do my customers live, and what do they do for a living? How much will my wealthiest customers spend for my product or service? If my customers buy product A, what percentage will also buy product B? If I offer an online customer a discount for a product related to their shopping cart, will they buy it? Understand your data before you apply any algorithms! It is easier to ask questions if you know what type of data you have Even seemingly simple questions have value: Your questions will get more complex and detailed over time, but don t underestimate Yes/No results What if you could increase your customer response rate by even 20%? Would that be worth it? Good News! There s no 100% right answer There s just different levels of right 2015 SAP SE or an SAP affiliate company. All rights reserved. 39

41 Key Messages SAP Predictive Analytics is designed to: Help companies make better use of their data and better decisions Make more meaningful and proactive sense of new data coming in Answer difficult questions that BI is not equipped for Predictive Analytics is the next step in Business Intelligence: PA is a natural progression that customers don t realize yet PA Return on Investment (ROI) not TCO: Answer the ROI question first Almost every analytics use case could benefit from predictive algorithms Increasingly, Big Data = Predictive Analytics Algorithms are the key to finding needles in the Big Data (Hay)stack Automated Analytics enables more people to work with Big Data without data science experience 2015 SAP SE or an SAP affiliate company. All rights reserved. 40

42 Additional Predictive Sessions at ASUG BOUC 2706: Roadmap: Predictive Analytics: What's New and What's Next 2765: Beginner's Guide to Harnessing Big Data and Internet of Things For Real-World Use Cases 3526: ASUG Predictive Analytics Influence Council 3627: SAP Predictive Analytics Solution Hands-On Session 2703: Top 10 Predictive Use Cases and Customer Case Studies 2513: Predictive Maintenance & Service: Customer Lessons Learned 2015 SAP SE or an SAP affiliate company. All rights reserved. 41

43 Where to Find More Information Predictive on SCN: Predictive Official Product Tutorials: Predictive Blog: Predictive 30-day Trial Predictive Product Roadmap * (then Analytics > Predictive) * Requires login credentials to the SAP Service Marketplace 2015 SAP SE or an SAP affiliate company. All rights reserved. 42

45 2015 SAP SE or an SAP affiliate company. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see for additional trademark information and notices. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary. These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE s or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forwardlooking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions SAP SE or an SAP affiliate company. All rights reserved. 44