Oracle Business Intelligence and Analytics Platform SFOUG March 22, 2006 Shyam Varan Nath Oracle Corporation 1
Agenda Introduction to Business Intelligence A brief look into Oracle Integrated BI platform Need for Real-time Analytics Overview of Oracle Data Mining platform Real-time Analytical Application DM demo (time-permitting) Q&A 2
About me Principal Consultant in Business Intelligence and Reporting group at Oracle, also called the BI Practice Oracle Certified Professional, DBA track, since 1998 Oracle database version 7.3 onwards Worked in industries like Finance industry, Law Enforcement Telecomm, Healthcare, etc. President and Founder of Oracle BIWA SIG, awarded IOUG Oracle Contribution Award for 2007 Speaker in Oracle Open world (2003, 2006), IOUG/Collaborate (2005-06), NYOUG (June/Sep),NOUG, SEOUC, IEEE conferences etc 3
O V E R V I E W Business Intelligence 4
What is Business Intelligence? Business intelligence (BI) is a broad category of application programs and technologies for gathering, storing, analyzing, and providing access to data to help enterprise users make better business decisions. BI applications include the activities of decision support, query and reporting, online analytical processing (OLAP), statistical analysis, forecasting, and data mining. 5
BI WIKI 6
Buzz about the Biz 7
Business Intelligence-Driven Enterprise Insight Driven Action Intelligent Interaction What should I do now, at the moment of contact Increasing Value Query, OLAP / DW Explore my data Performance Management How am I doing vs. goals? What is my best opportunity? What should I do next? Transactional Reporting Here are your reports Generational Steps 8
What Is Greater Business Insight? A level of business insight where every user throughout the organization is guided by relevant, timely, consistent, and accurate information to make the most informed decision at the moment that action should be taken. 9
Greater Business Insight in Action Example: Production Management - Cars Question Insight Action Common part quantities are limited. How should AI allocate? HYBRID Product Product Profitability A B Re-allocate parts used in C to produce more A A Common Part SUV B C C -8-4 0 4 8 Margin (%) Common Part B C 10
Delivering Greater Business Insight Aligning Operational Decisions with Strategic Objectives Embedding Actionable Intelligence within Transactions Creating a Trusted Source of Business Information 11
Role-Based Dashboards BAM Business Insight Delivery CPM Dashboards Analytic Foundation BI Technology Fusion Middleware Applications that enable users to monitor business performance, and drill down on exceptions to diagnose problems and take action Detect Business Issues and Opportunities Diagnose Problems Decide Course of Action Sales R&D Mfg & Maintenance Purchasing Service Human Resources Finance Continually Monitor Performance 12
INTEGRATED ENTERPRISE PLATFORM FOR Business Intelligence 13
Common Customer Environment Multi-Vendor, Un-integrated ETL Tool OLAP Engine Analytic Apps Security? Name/Address Scrubbing Transformation Engine Transformation Engine Database Mining Engine Protracted and complex implementation Escalating maintenance costs Poor and incomplete BI solution Promotes information silos Query & Analysis Reporting Engine Enterprise Reporting P o r t a l 14
Query and Reporting OLAP Data Mining Extraction of detailed and roll up data Information How is the interest rate changing in last 2 years? Summaries, trends and forecasts Analysis What is the average home price of the condos, by region, by year of construction? Knowledge discovery of hidden patterns Insights & Prediction Who will reconsolidate loans in the next 6 months and why? 15
Components of Oracle BI Platform Oracle Warehouse Builder OLAP Data Mining Oracle Discoverer/Portal/Reports (SE) Oracle BI (EE) BI Beans 16
What is Oracle Warehouse Builder? 17
Integrated options for OWB 10gR2 Enterprise ETL Option Reuse Productivity Performance Data Quality Option Data Profiling Data Rules Data Auditors 18
Why is OLAP such a fast Data Access Method How do Expenses compare this Quarter versus Last Quarter What is an Item s Expense contribution to its Category? Data stored in dense arrays Category Hotel Lunch Food Expenses Q1 Q2 Q3 Northeast SF West Market Offset addressing no joins More powerful analysis Better performance Time 19
Oracle BI Road MAP 20
Oracle BI EE and Advanced Analytics (OLAP/ODM) Rich analytics, completely integrated 21
A Case Study of Business Intelligence based Solution 22
Oracle Protect Architecture Overview PROTECT Dashboards Portal / Disco Based PROTECT Crime Analysis Law Enforcement Exec Dashboard Regional Crime Analysis Detailed Crime Report Counter Terrorism Dashboard CompStat Dashboard Discoverer Ad-Hoc Excel based OLAP XML Publisher Dashboard / PROTECT Warehouse Integration Layer OLAP Support for Dashboard Metrics Compstat Workflow for Alerts Approvals ETL-OWB Processes PROTECT Justice Data Repository Dimensional Model Flexible Extensible Advanced Analytics Grid / RAC for Scalability High Availability Manageability Industry Compliant Integration Layer XDB interface Data Mining Support for Trends, Patterns, Hot Spots Narrative Text Mining Spatial Support for Visualization Proximity Searches XML Message Layer ETL Repository Source Data RMS Pawn Shop DMV Criminal History Incident Report Property & Evidence Field Interviews Other 23
Transactional (Records Management) v/s BI System 24
Police Report of the Incident 25
26
27
28
Similar Application in BI EE 29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
Recap so far. Looked at Intro to BI A sample of BI application What s ahead. Real time analytics Data Mining etc
Need for Real Time Analytics 47
The Amazon Example (the familiar example) Market Basket Analysis 48
What is Real-time Analytics? Real-time analytics is the use of, or the capacity to use, all available enterprise data and resources when they are needed. It consists of dynamic analysis, drawing inferences and reporting, based on data entered into a system up to the actual time of use. Real-time analytics is also known as real-time data analytics, real-time data integration, and realtime business intelligence. 49
50
Real-Time Applications based on Data Mining Why do we need Real time decisions? Recent industry changes like DO NOT CALL LIST, TIVO, spam blockers, aversion to junk post mail etc Importance of In-Bound Opportunity Finance-banking industry example 51
O V E R V I E W Data Mining 52
Data Mining Process of sifting through massive amounts of data to find hidden patterns and discover new insights Data Mining can provide valuable results: Identify factors more associated with a target attribute (Attribute Importance) Predict individual behavior (Classification) Find profiles of targeted people or items (Decision Trees) Segment a population (Clustering) Determine important relationships with the population (Associations) Find fraud or rare events (Anomaly Detection) 53
Data Mining Techniques(10g,10gR2) Unsupervised (clustering) K-means O-Cluster Non-Negative Matrix Factorization Anomaly Detection Supervised (classification and prediction) Adaptive Bayes Network (ABN) Naïve Bayes Support Vector Machine Decision Trees (PL/SQL code generator) 54
Oracle Data Mining Algorithms & Example Applications Attribute Importance Identify most influential attributes for a target attribute Factors associated with high costs, responding to an offer, etc. Classification and Prediction Predict customers most likely to: Respond to a campaign or offer Incur the highest costs Target your best customers Develop customer profiles Married >$50K Gender Income Status Gender HH Size A1 A2 A3 A4 A5 A6 A7 <=$50K Age M F >35 <=35 Single F M >4 Buy = 0 Buy = 1 Buy = 0 Buy = 1 Buy = 0 <=4 Buy = 1 55
Oracle Data Mining 10g R2 Decision Trees Decision Trees Classification Prediction Customer profiling Owns foreign car = yes Status Income Age >55 <55 Foreign car = no 45 > Age Problem: Find customers likely to buy a Buick and their profiles <=45 Gender Num children <100K >100K F M <=4 >4 Buick = 0 Buick = 1 Buick = 0 Buick = 1 Buick = 0 Buick = 1 IF (Age >55 AND Owns foreign car=no AND Income >100K ) THEN P(Buy Buick=1) =.77 Support = 250 56
Oracle Data Mining Algorithms & Example Applications Clustering Find naturally occurring groups Market segmentation Find disease subgroups Distinguish normal from non-normal behavior Association Rules Find co-occurring items in a market basket Suggest product combinations Design better item placement on shelves 57
Data Mining Flow Source Data Oracle Data Mining Host Application Source Data Extract Clean Deploy Evaluate Predict Source Data Transform Model Predict Production Data Source Data Mining Table Repository (MTR) 58
PL/SQL code sample /* pl/sql code gen1 */ CREATE PACKAGE "DATAMININGACTIVITY1" AUTHID DEFINER AS PROCEDURE "MINING_BUILD_TEST"(case_table IN VARCHAR2 DEFAULT '"DMUSER1"."MINING_BUILD_TEXT"', additional_table_1 IN VARCHAR2 DEFAULT NULL, model_name IN VARCHAR2 DEFAULT 'MINING_BUILD_75202_DT', test_metric_name IN VARCHAR2 DEFAULT '"DM4J$MINING_TEST"', END; / /* pl/sql code gen1 */ CREATE PACKAGE BODY "DATAMININGACTIVITY1" AS c_long_sql_statement_length CONSTANT INTEGER := 32767;. 59
Model Scoring Approach Gartner's Scoring Approach Classification 60
Oracle Spreadsheet Add-In for Predictive Analytics 61
Industry Specific Cases/Applications of Data Mining 62
The in-bound window of opportunity In-bound calls present a great opportunity for upsell and cross sell Data mining scoring for recommendations in Real time can provide CSR s with most likely loan product to offer Such an application was built using Oracle Data Mining for finance industry 63
Demo Scenario: Business Challenges About National Bank Fictional financial services provider Customer base: 5 million Large volume Call Center Business Challenges High customer turnover rate of 14% per year Associated replacement cost in millions per year Average cost of new customer acquisition: $250 Currently 2 products per customer, goal of achieving 4 per customer 64
Demo Scenario: Call Center Solution Predict in real-time customers propensity to attrite and to respond to various retention treatments Offer relevant and timely retention offers such as free online bill payment only to customers most likely to leave Predict in real-time customers propensity to respond to various cross- & up sell offers Target customers with relevant and timely cross sell offers at the time of call instead of Running costly, less relevant and less timely outbound retention and cross- & up-sell campaigns 65
Call Scenario A: Intelligent Cross- Sell Profile of caller (Linda Johnson): Female, 28 years old, single Holds checking and savings account at National Bank Medium-value customer Calls to change address (due to new job after grad school) Objectives of National Bank: Expand customer relationship through real-time intelligent cross- and up-sell offers 66
Call Scenario A: Upon caller identification, Linda Linda Johnson Johnson is is recognized recognized as as a a student student holding holding two two accounts accounts with with National. National. 67
In addition, upon caller identification, based based on on Linda s Linda s customer customer profile, profile, the the mining mining engine engine predicts predicts that that Linda Linda currently currently has has no no significant significant risk risk of of churning, churning, and and therefore therefore no no retention retention treatment treatment is is warranted, warranted, and and that that Auto Auto Insurance Insurance is is the the marketing marketing offer offer that that is is most most likely likely to to be be accepted accepted by by Linda. Linda. 68
Upon noting Linda s call reason, change of address, this this new new in-context in-context information information is is communicated communicated to to the the mining mining engine engine in in real-time. real-time. 69
Based on the new in-context information, the the mining mining engine engine predicts predicts in in real-time real-time that that Linda s Linda s churn churn risk risk has has not not increased increased but but that that the the most most appropriate appropriate offer offer now now is is Overdraft Overdraft Protection, Protection, addressing addressing Linda s Linda s likely likely increased increased financial financial needs. needs. 70
Linda s response to the extended offer is is noted noted by by the the agent agent using using the the offer offer response response buttons. buttons. The The response response information information is is communicated communicated to to the the mining mining engine engine in in real-time real-time for for self-learning. self-learning. 71
Linda s response to the extended offer is is also also recorded recorded in in the the database, database, enabling enabling offer offer response response tracking tracking and and cross cross channel channel Marketing Marketing Analytics Analytics reporting. reporting. 72
Recap of the Analytics Application An example of data mining driving the BI Analytical application The Bank develops Greater Business Insight into its customer Converts that into actionable items The CSR can act on this in real-time (during the call) 73
Installing Oracle Data Mining
Architecture for Oracle Data Mining Installing Oracle Data Mining How to optimize the DB configuration? How to understand the schemas necessary to support Data Mining? 75
Installation Test Check to see if the product exists 76
Install Option Default install list for Oracle 9i and 10g RDBMS include ODM It s a separate license, free for development, fee for production If ODM not installed, can be added by custom install using Oracle Universal Installer (OUI) Companion disk has samples programs for ODM ODMr the GUI ODM website 77
Add ODM 78
ODM 79
Installing ODM GUI The GUI is called ODMr Uses JDBC connection to the database Can be used for most data mining tasks such as importing data from flat files, running models, scoring, visually viewing the results etc. 80
Schema for Data Mining
Creating mining users CREATE TABLESPACE "ODMPERM" DATAFILE 'C:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL \odm1.dbf SIZE 20M REUSE AUTOEXTEND ON NEXT 20M; The next SQL command creates a new temporary tablespace. CREATE TEMPORARY TABLESPACE "ODMTEMP" TEMPFILE 'C:\ORACLE\PRODUCT\10.2.0\ORADATA\ORCL \odmtemp.tmp SIZE 20M REUSE AUTOEXTEND ON NEXT 20M; 82
SQL for Create user CREATE USER dmuser1 IDENTIFIED BY change_now DEFAULT TABLESPACE odmperm TEMPORARY TABLESPACE odmtemp QUOTA UNLIMITED on odmperm; SQL for Grants GRANT create procedure to DMUSER1; GRANT create session to DMUSER1; GRANT create table to DMUSER1; GRANT create sequence to DMUSER1; Export / import data mining models SQL> EXECUTE DBMS_DATA_MINING.EXPORT_MODEL('allmodels.dmp','D MTEST'); 83
DB Privileges to the DM user Access Rights: Data mining users require several CREATE privileges. For text mining, users must also have access to the Oracle Text package ctxsys.ctx_ddl. The following privileges are required. CREATE PROCEDURE CREATE SESSION CREATE TABLE CREATE SEQUENCE CREATE VIEW CREATE JOB CREATE TYPE CREATE SYNONYM EXECUTE ON ctxsys.ctx_ddl 84
Contact information: Shyam.Nath@Oracle.com (954) 609 2402 cell DW & BI Special Interest Group http://dwbisig.oracle.ioug.org 86