Big Data and Predictive Analytics: Fiserv Data Mining Case Study [CON8631] Data Warehouse and Big Data

Transcription

1 Big Data and Predictive Analytics: Fiserv Data Mining Case Study [CON8631] Data Warehouse and Big Data Miguel Barrera - Director, Risk Analytics, Fiserv, Inc. Julia Minkowski - Risk Manager, Fiserv, Inc. Charlie Berger, MS Eng, MBA, Sr. Director Product Management, Data Mining and Advanced Analytics charlie.berger@oracle.com

2 Agenda R Big Data and Predictive Analytics: Fiserv Data Mining Case Study [CON8631] 1. Oracle Advanced Analytics Overview Charlie Berger, Sr. Director Product Management, Data Mining and Advanced Analytics, Oracle Corporation 2. Fiserv Data Mining Case Study Miguel Barrera - Director, Risk Analytics, Fiserv, Inc. Julia Minkowski - Risk Manager, Fiserv, Inc. Copyright 2014 Oracle and/or its affiliates. All rights reserved. 2

3 Planning for Future Growth of Data Exponentially Greater than Growth of Data Analysts! Conclusion Data Analysis platforms need to be Extremely Easy to Learn, yet.. Extremely Powerful and Automated as much as possible! Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle Confidential Internal/Restricted/Highly Restricted 3

4 Oracle Advanced Analytics Database Evolution 7 Data Mining Partners Analytical SQL in the Database Oracle acquires Thinking Machine Corp s dev. team + Darwin data mining software Oracle Data Mining 9.2i launched 2 algorithms (NB and AR) via Java API Oracle Data Mining 10g & 10gR2 introduces SQL dm functions, 7 new SQL dm algorithms and new Oracle Data Miner Classic wizards driven GUI ODM 11g & 11gR2 adds AutoDataPrep (ADP), text New algorithms (EM, PCA, SVD) Predictive Queries SQLDEV/Oracle Data Miner 4.1 SQL script generation, JSON Query node & SQL Query node (R integration) mining, perf. improvements SQLDEV/Oracle Data Miner OAA/ORE work flow GUI adds NN, Stepwise, launched scalable R algorithms Integration with R and Oracle Adv. Analytics introduction/addition of for Hadoop Connector Oracle R Enterprise launched with parallel implementations of R Product renamed Oracle algorithms (NN, LM, NMF) Advanced Analytics (ODM + ORE) Copyright 2014 Oracle and/or its affiliates. All rights reserved.

5 Oracle Advanced Analytics Database Option History & High Level Architecture Development has spend 15 years stem-celling analytics and workhorse machine learning algorithms into Oracle Database 1999: Thinking Machines acquisition of Darwin data mining software Darwin killed Instead developed new within the SQL kernel, in-database implementations of popular & cutting edge data mining algorithms that leverage the strengths of the database counting, conditional probabilities, sort, rank, partition, group-by, collections, etc. Today, in 12c, the Database has become an Analytical Database Nearly 20 cutting edge machine learning algorithms and 50+ statistical functions implemented as true SQL functions When building models, leverage existing scalable technology (e.g., parallel execution, bitmap indexes, aggregation techniques) and add new core database technology (e.g., recursion within the parallel infrastructure, IEEE float, etc.) A data mining model is a schema object in the database, built via a PL/SQL API and scored via built-in SQL functions. True power is evident when scoring models using built-in SQL functions e.g. Exadata smart scan scoring Also expose data mining functions via OAA/Oracle R Enterprise integration. OAA/ORE also extends OAA with ability to run and call R packages. R Copyright 2014 Oracle and/or its affiliates. All rights reserved.

6 Oracle Advanced Analytics Performance and Scalability with Low Total Cost of Ownership Traditional Analytics Data Import Data Mining Model Scoring Data Prep. & Transformation Data Mining Model Building Data Prep & Transformation Data Extraction Hours, Days or Weeks Oracle Advanced Analytics avings Model Scoring Embedded Data Prep Model Building Data Preparation Secs, Mins or Hours Data remains in the Database Scalable, parallel Data Mining algorithms in SQL kernel Fast parallelized native SQL data mining functions, SQL data preparation and efficient execution of R open-source packages High-performance parallel scoring of SQL data mining functions and R open-source models Fastest way to deliver enterprise-wide predictive analytics Integrated GUI for Predictive Analytics Database scoring engine Lowest TCO Eliminate data duplication Eliminate separate analytical servers Leverage investment in Oracle IT Copyright 2014 Oracle and/or its affiliates. All rights reserved.

7 Oracle Advanced Analytics Database Architecture Component of Oracle Database SQL Functions SQL Developer R Client OBIEE Applications Oracle Database Enterprise Edition Oracle Advanced Analytics Native SQL Data Mining/Analytic Functions + High-performance R Integration for Scalable, Distributed, Parallel Execution Copyright 2014 Oracle and/or its affiliates. All rights reserved.

8 Oracle Advanced Analytics Database Option Fastest Way to Deliver Scalable Enterprise-wide Predictive Analytics Key Features In-database data mining algorithms and open source R algorithms SQL, PL/SQL, R languages Scalable, parallel in-database execution Workflow GUI and IDEs Integrated component of Database Enables enterprise analytical applications Copyright 2014 Oracle and/or its affiliates. All rights reserved.

9 Be Specific in Problem Statement Poorly Defined Better Data Mining Technique Predict employees that leave Based on past employees that voluntarily left: Create New Attribute EmplTurnover O/1 Predict customers that churn Target best customers How can I make more $$? Which customers are likely to buy? Who are my best customers? How can I combat fraud? Based on past customers that have churned: Create New Attribute Churn YES/NO Recency, Frequency Monetary (RFM) Analysis Specific Dollar Amount over Time Window: Who has spent $500+ in most recent 18 months What helps me sell soft drinks & coffee? How much is each customer likely to spend? What descriptive rules describe best customers? Which transactions are the most anomalous? Then roll-up to physician, claimant, employee, etc. Copyright 2014 Oracle and/or its affiliates. All rights reserved.

10 Responders More Data Variety Better Predictive Models Increasing sources of relevant data can boost model accuracy 100% 100% Model with Big Data and hundreds -- thousands of input variables including: Demographic data Purchase POS transactional data Unstructured data, text & comments Spatial location data Long term vs. recent historical behavior Web visits Sensor data etc. 0% Population Size Naïve Guess or Random Model with 20 variables Model with 75 variables Model with 250 variables Copyright 2014 Oracle and/or its affiliates. All rights reserved.

11 Predicting Behavior Identify Likely Behavior and their Profiles Consider: Demographics Past purchases Recent purchases Customer comments & tweets Copyright 2014 Oracle and/or its affiliates. All rights reserved.

12 SOURCES Oracle Big Data Management System Oracle Big Data SQL Oracle Database Cloudera Hadoop Oracle NoSQL Database Oracle R Advanced Analytics for Hadoop Oracle R Distribution Big Data Appliance B Oracle Big Data Connectors Oracle Data Integrator Oracle Oracle Industry Database Models Oracle Advanced Oracle Advanced Security Analytics Oracle Advanced Oracle Analytics Spatial & Graph Oracle Spatial & Graph Oracle Exadata select cust_id from customers where region = US and prediction_probability(churnmod, Y using *) > 0.8; Copyright 2014 Oracle and/or its affiliates. All rights reserved.

13 R Widely Popular R is a statistics language similar to Base SAS or SPSS statistics R environment Strengths Powerful & Extensible Graphical & Extensive statistics Free open source Challenges Memory constrained Single threaded Outer loop slows down process Not industrial strength Copyright 2014 Oracle and/or its affiliates. All rights reserved.

14 Oracle Advanced Analytics Oracle R Enterprise Compute Engines 1 R Engine Other R packages Oracle R Enterprise packages SQL Results 2 3 Oracle Database R Open Source User tables?x R Results R Engine Other R packages Oracle R Enterprise packages User R Engine on desktop Database Compute Engine R Engine(s) spawned by Oracle DB R-SQL Transparency Framework intercepts R functions for scalable in-database execution Function intercept for data transforms, statistical functions and advanced analytics Interactive display of graphical results and flow control as in standard R Submit entire R scripts for execution by database Scale to large datasets Access tables, views, and external tables, as well as data through DB LINKS Leverage database SQL parallelism Leverage new and existing in-database statistical and data mining capabilities Database can spawn multiple R engines for database-managed parallelism Efficient data transfer to spawned R engines Emulate map-reduce style algorithms and applications Enables lights-out execution of R scripts Copyright 2014 Oracle and/or its affiliates. All rights reserved.

15 Integrated Business Intelligence Enhance Dashboards with Predictions and Data Mining Insights In-database predictive models mine customer data and predict their behavior OBIEE s integrated spatial mapping shows location All OAA results and predictions available in Database via OBIEE Admin to enhance dashboards Oracle BI EE defines results for end user presentation Oracle Advanced Analytics data mining results available to Oracle BI EE Copyright 2014 Oracle and/or its affiliates. All rights reserved.

16 Oracle Communications Industry Data Model Predictive Analytics Applications Pre-Built Predictive Models Fastest Way to Deliver Scalable Enterprise-wide Predictive Analytics OAA s clustering and predictions available in-db for OBIEE Automatic Customer Segmentation, Churn Predictions, and Sentiment Analysis Copyright 2014 Oracle and/or its affiliates. All rights reserved.

17 Fusion HCM Predictive Workforce Predictive Analytics Applications Fusion Human Capital Management Powered by OAA Oracle Advanced Analytics factoryinstalled predictive analytics Employees likely to leave and predicted performance Top reasons, expected behavior Real-time "What if?" analysis Copyright 2014 Oracle and/or its affiliates. All rights reserved.

18 Oracle Advanced Analytics Database Option Oracle Data Miner 4.0 Summary New Features Oracle Data Miner/SQLDEV 4.1 (for Oracle Database 11g and 12c) New Graph node (box, scatter, bar, histograms) SQL Query node + integration of R scripts Automatic SQL script generation for deployment JSON Query node (added in 4.1) Oracle Advanced Analytics 12c features exposed in Oracle Data Miner New SQL data mining algorithms/enhancements Expectation Maximization clustering algorithm PCA & Singular Vector Decomposition algorithms Improved/automated Text Mining, Prediction Details and other algorithm improvements) Predictive SQL Queries automatic build, apply within SQL query Copyright 2014 Oracle and/or its affiliates. All rights reserved.

19 12c New Features New Server Functionality Predictive Queries Immediate build/apply of ODM models in SQL query Classification & regression Multi-target (nested) problems Clustering query Anomaly query Feature extraction query Results/Predictions! R OAA automatically creates multiple anomaly detection models Grouped_By and scores by partition via powerful SQL query Copyright 2014 Oracle and/or its affiliates. All rights reserved.

20 Oracle Advanced Analytics OAA/ORACLE DATA MINER QUICK 2 MINUTE DEMO Copyright 2014 Oracle and/or its affiliates. All rights reserved.

21 Take a Test Drive! Vlamis Software, Oracle Partner Offers FREE Test Drives on the Amazon Cloud Step 1 Fill out request Go to Step 2 Connect Connect with Remote Desktop Step 3 Start Test Drive! Oracle Database + Oracle Advanced Analytics Option SQL Developer/Oracle Data Miner GUI Demo data for learning Follow Tutorials Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle Confidential Internal/Restricted/Highly Restricted 21

22 New book on Oracle Advanced Analytics available Book available on Amazon Predictive Analytics Using Oracle Data Miner: Develop for ODM in SQL & PL/SQL Copyright 2014 Oracle and/or its affiliates. All rights reserved. Oracle Confidential Internal/Restricted/Highly Restricted 22

23 OAA Links and Resources Oracle Advanced Analytics Overview: Link to presentation Big Data Analytics using Oracle Advanced Analytics In-Database Option OAA data sheet on OTN Oracle Internal OAA Product Management Wiki and Workspace YouTube recorded OAA Presentations and Demos: Oracle Advanced Analytics and Data Mining at the YouTube Movies (6 + OAA live Demos on ODM r 4.0 New Features, Retail, Fraud, Loyalty, Overview, etc.) Getting Started: Link to Getting Started w/ ODM blog entry Link to New OAA/Oracle Data Mining 2-Day Instructor Led Oracle University course. Link to OAA/Oracle Data Mining 4.0 Oracle by Examples (free) Tutorials on OTN Take a Free Test Drive of Oracle Advanced Analytics (Oracle Data Miner GUI) on the Amazon Cloud Link to SQL Developer Days Virtual Event w/ downloadable VM of Oracle Database + ODM/ODMr and e-training for Hands on Labs Link to OAA/Oracle R Enterprise (free) Tutorial Series on OTN Additional Resources: Oracle Advanced Analytics Option on OTN page OAA/Oracle Data Mining on OTN page, ODM Documentation & ODM Blog OAA/Oracle R Enterprise page on OTN page, ORE Documentation & ORE Blog Oracle SQL based Basic Statistical functions on OTN Business Intelligence, Warehousing & Analytics BIWA Summit 15, Jan 27-29, 2015 at Oracle HQ Conference Center Copyright 2014 Oracle and/or its affiliates. All rights reserved.

24 BIWA Summit January 27-29, 2015 Oracle HQ Conference Center Copyright 2014 Oracle and/or its affiliates. All rights reserved.

25 Faster than a Mouse: Turn Data Mining Strategy into Action Miguel Barrera, Director of Risk Analytics, Fiserv Inc. Julia Minkowski, Risk Manager, Fiserv Inc.

26 Use Case : Fraud Prevention in Online Payments Agenda Best Practices : Turning Data Mining Strategy into Action Fiserv, Inc. or its affiliates.

27 27 Use Case : Fraud Prevention in Online Payments

28 Risk Fiserv Electronic Payments We prevent $200M in losses every year using data to monitor, understand and anticipate fraud We manage risk for $24BB in transfers, servicing 2,000+ US financial institutions, including the 5 of the top 10 banks A department of 5 people, we operated in start-up mode until we were acquired in 2011 by Fiserv We build our risk models, supervise their installation & develop the next-generation of strategies for risk mitigation 2014 Fiserv, Inc. or its affiliates.

29 What is special about Fraud Prevention? Fraud is performed by organized criminal groups using sophisticated technologies and logistics Hard to detect: target has low frequency (2 in 10,000) The cost of mistakes is very high $ Losses if you fail to detect fraud 60% increase in customer attrition if you miss-classify The environment changes fast, so you need to adapt quickly Fraud prevention is a great field for the application of predictive analytics 2014 Fiserv, Inc. or its affiliates.

30 Analytics for Fraud Prevention Explore & Understand Anticipate & Control Monitor 2014 Fiserv, Inc. or its affiliates.

31 Risk Management: Goals and Constraints Goals: Help the business to expand to more profitable markets (on-line booking, real time payments), while keeping loss rates constant, and customers happy Constraints: Build a flexible system that adapts to new fraud patterns Service the existing client base Minimize the time that the production systems will be off-line or reset Build the next-generation of strategies with very limited resources 2014 Fiserv, Inc. or its affiliates.

32 Data Miner Survey 2013 by Rexer Analytics While 6 out 10 data miners report the data is available for analysis within days of capture, the time to deploy the models takes substantially longer. For 60% of the respondents the deployment time will range between 3 weeks and 1year Fiserv, Inc. or its affiliates.

33 The problem we faced We had the Algorithms (SAS + Angoss) Decision Trees + Gradient Boosting GLM and Logistic Regression SVM Bayesian Estimates But implementation took too long 3 months turn-around to estimate + deploy Logistic Regression (2008) 1 month to estimate and deploy Trees and GLM (2010) 1 week to estimate, 1 week to install rules in online application (2012) 1 day to estimate and deploy Trees + GLM models (2014) 2014 Fiserv, Inc. or its affiliates.

34 In Fraud-Mitigation Speed is the Key How long can you wait to deploy a solution? 2014 Fiserv, Inc. or its affiliates.

35 Why we liked Oracle Advance Analytics? Accuracy Agility Scalability The algorithms fit are as good as more complex algorithms The loss reduction from timely deployment (hours) compensates for model fit No data transfer needed (in-database) New opportunities to combine structured data with unstructured data The integration with our DB replication makes re-fit inexpensive The same algorithm can scale-up for all other clients 2014 Fiserv, Inc. or its affiliates.

36 Oracle Advanced Analytics Time Value 2012, Oracle Corporation 2014 Fiserv, Inc. or its affiliates.

37 Accuracy + Agility vs. Cost to Deploy Pick the best combination of: Less days to deployment High model accuracy Lower Cost Application Deploy (Days) Accuracy Total Cost SAS Server x5 ODM SAS Ba s e % Angos s % 2014 Fiserv, Inc. or its affiliates.

38 ODM Oracle Data Miner GUI Built-In in Oracle SQL Developer Tool Downloadable free on OTN version 4.0 or latest Easy to use GUI; explore data; work-flows Powerful multiple algorithms and data transformations; 100% in-db; build, evaluate and apply data mining models Deployable Shared analytical workflows; Generates PMML and SQL scripts for automation 2012, Oracle Corporation 2014 Fiserv, Inc. or its affiliates.

39 ODM Oracle Data Miner GUI Oracle Data Miner Nodes 2012, Oracle Corporation 2012, Oracle Corporation 2014 Fiserv, Inc. or its affiliates.

40 Oracle Data Miner Algorithms Identify most important risk factors (Attribute Importance) Predict Fraudsters behavior (Classification) Find profiles of bad transfers (Decision Trees) Predict Fraud Risk Probability (Regression) Segment overall population (Clustering) Find fraudulent transactions (Anomaly detection) Determine co-occuring items in baskets (Associations) Reduce a large dataset into representative new attributes (Feature Extraction) 2012, Oracle Corporation 2014 Fiserv, Inc. or its affiliates.

41 Turning Data Mining Strategy into Action

42 Best Practices in Analytics 1. Identify Benefits & Constraints Involve Key Stakeholders 2. Develop the Strategy Collect & Process Data Run Descriptive Analytics Identify patterns 3.Turn Strategy into Action Select Best Option(s) Business Manager Data Scientist IT Manager Success Factors and Constraints Align your Team s Incentives Provide Actionable Insights Test Predictive Models Simulate scenarios (Monte Carlo) Score models on KPI Install into Production Run A/B testing Start Small and Increase Gradually Select the Appropriate Infrastructure DB Architecture Modeling techniques Estimate Impact for the Business ROI /Cost Profitability Operations Track Benefits and KPI 2014 Fiserv, Inc. or its affiliates.

43 Involve the Right Stakeholders Business Manager IT Manager Data Scientist Preserve Service Level Agreement Reduce Operational Risk Preserve Budget 2014 Fiserv, Inc. or its affiliates.

44 Conflict of Interests? Cannot agree on success factors? Wonder why? 2014 Fiserv, Inc. or its affiliates.

45 IT Manager s Strategy Preserve Service Level Agreements Stable systems Ease of roll-back Minimize Operational risk Control Costs 2014 Fiserv, Inc. or its affiliates.

46 Data Scientist s Mind Estimate the Best Model Possible Improve Detection Rates Better Algorithms, Faster Hardware Big(ger) Data! Explore New Algorithms Source: Ayasdi 2014 Put some power behind it!! 2014 Fiserv, Inc. or its affiliates.

47 Business Manager s Mind Maximize Productivity: Build for specific needs What is the cost? Why does it take so long? What is the impact on customer experience? And: Don t talk to me in Tech-Speak! First we ran a chi- square test, and then we converted the categorical data to ordinal, next we ran a logistic regression, and then we lagged the economic data by a year Source: Davenport, Tom (2013), Keep up with your quants", Harvard Business Review, Issue July-August Fiserv, Inc. or its affiliates.

48 Communication issue 2014 Fiserv, Inc. or its affiliates.

49 Managing the Quants Define clearly the objective and constraints Implement SMART* goal setting Get familiar with basic analytics concepts Establish a time-line for delivery then multiply x 2 Make sure you understand enough to explain to other executives you will champion this initiative and negotiate the budgets Source: Davenport, Tom (2013), Keep up with your quants", Harvard Business Review, Issue July-August Fiserv, Inc. or its affiliates.

50 Taking Care of Business (tips for Data Scientists) Communicate clearly business level information When and what is the expected result Present the key concept in 2 phrases Avoid technical language for communication If asked for more details, then present the How Provide a Business Dashboard Provide the $$ metrics profit/loss reduction Show the impact of algorithms deployed / provided Current vs. Historical Pick the right model - the model that maximizes the ROI Source: Davenport, Tom (2013), Keep up with your quants", Harvard Business Review, Issue July-August Fiserv, Inc. or its affiliates.

51 From Paper to Execution Bring that Escher guy here NOW!! 2014 Fiserv, Inc. or its affiliates.

52 Success Factors in Fraud Mitigation Accuracy: Low False Negative Rate (How much fraud $ you miss) Low False Positive Rates (How many people you bother with additional identity verification) Agility Minimize reaction-time to fraud attacks Make your updates easy to implement Scalability: Automate processes to keep variable cost down Invest in infrastructure and replicate across clients 2014 Fiserv, Inc. or its affiliates.

53 Selecting the Right Tools Easy to Use and Deploy Can combine structured data with unstructured data - new trend Tools that integrate in the DB Allow for Fast Model Fitting and Re-estimation Minimize data transport across systems In-House Algorithms Develop algorithms that can run directly in DB for fast estimation and execution 2014 Fiserv, Inc. or its affiliates.

54 Tracking Performance: Dashboard Our dashboards tracked the key performance metrics: Historical Trends for Fraud Rates and Losses (Business KPI) Percentage of Transfers affected by Risk Mitigation (Business KPI) % of population affected by policy and % of fraud prevented (KPI for Analytics) Fraud detection rates for rules installed (KPI for Analytics) 2014 Fiserv, Inc. or its affiliates.

55 Key Takeaways On Fraud Modeling When dealing with fraud, the speed to implement a new model is the most important factor Improvements in accuracy may be lost due to delays in deployment; systems with fast turnaround have better ROI than complex algorithms with long implementation times. Select the right tools that will enable fast analysis and deployment. Turning Strategy into Action Involving the key stakeholders early in the process maximizes your chance for success. Once you have aligned the incentives for the team, selecting the appropriate techniques, tools and infrastructure becomes much simpler It is crucial for business managers to correctly define the problems and objectives, asking the right questions and learning the basic analytical concepts For data scientists it is important to select their models and projects based on the expected business impact and to translate their findings into the relevant metrics 2014 Fiserv, Inc. or its affiliates.

56 If you have further questions or comments, please contact: Julia Minkowski Lead Risk Analyst, Fiserv Inc Source: Kdnuggets, Fiserv, Inc. or its affiliates.

57 Copyright 2014 Oracle and/or its affiliates. All rights reserved.

58