extreme Datamining mit Oracle R Enterprise

Size: px

Start display at page:

Download "extreme Datamining mit Oracle R Enterprise"

Linette Goodman
9 years ago
Views:

1 extreme Datamining mit Oracle R Enterprise Oliver Bracht Managing Director eoda Matthias Fuchs Senior Consultant ISE Information Systems Engineering GmbH

2 extreme Datamining with Oracle R Enterprise About R In database data mining R with Oracle database R on Oracle Exadata R Example implementation Outlook R Copyright (C) ISE GmbH - All Rights Reserved 2

ISE & eoda - Oracle partner since 1992 - Test center

- R expertice since 2009 - analysing of structured and

3 ISE & eoda - Oracle partner since Test center Exadata Exalogic Exalytics - Gräfenberg Nürnberg - München - R expertice since analysing of structured and unstructured data - Kassel Copyright (C) ISE GmbH - All Rights Reserved 3

10 In Database data mining Traditonal Analytics Data Import Model Scoring Data Preparation Transformation Oracle Datamining Savings Results Faster time for Data to Insights Lower TCO Eliminates Data Movement Data Duplication Maintains Security Model Building Data Preparation Transformation Data Extraction Model Scoring Embedded Data Preparation Model Building Data Preperation Cutting edge machine learning algorithms inside the SQL kernel of Database Copyright (C) ISE GmbH - All Rights Reserved 10

11 R with Oracle database Using Oracle DB Calculation Local Using Oracle DB Calculation in Oracle DB Using Oracle DB Calculation on DB Server R Engine Oracle R Packages SQL In Database statistical and data mining R emebedded Oracle R Packages R Engine Calculating on R Client Data out of DB transfer to client Calculating in DB Oracle Data mining Data stay in database Use of cell storage Calculating on DB server Data out of DB Spawning several R Processes Copyright (C) ISE GmbH - All Rights Reserved 11

12 R with Oracle database Comaprison between the Oracle R database methods R Client R in Database R in DB Server Cran Packages Yes Ore packages and ODM Parallel No, only in R In Packages, spawn parralel R Processes Yes Spawn parallel R Processes performance limitation Network, CPU, RAM on Client I/O, CPU, RAM on DB Server I/O, CPU, RAM of DB Server Parallel in R Start R client Out of SQL, R client Out of SQL, R client Copyright (C) ISE GmbH - All Rights Reserved 12

13 R with Oracle database - Oracle Data Mining Mapping Cran RODM Packages Mapping of ODM Packages to R RODM Function RODM_create_ai_model RODM_create_assoc_model RODM_create_dt_model RODM_create_glm_model RODM_create_kmeans_model RODM_create_nb_model RODM_create_nmf_model RODM_create_oc_model RODM_create_svm_model Description Attribute Importance Association Rules Decision Tree Generalized Linear Model Hierarchical k-means Naive Bayes Non-Negative Matrix Factorization O-cluster Support Vector Machine Copyright (C) ISE GmbH - All Rights Reserved 13

14 R with Oracle database Routines in package ore Significance Tests Chi-square, McNemar, Bowker Simple and weighted kappas Cochran-Mantel-Haenzel correlation Cramer's V Binomial, KS, t, F, Wilcox Distribution Functions Beta distribution Binomial distribution Cauchy distribution Chi-square distribution Exponential distribution F-distribution Gamma distribution Geometric distribution Log Normal distribution Logistic distribution Negative Binomial distribution Normal distribution Poisson distribution Sign Rank distribution Student t distribution Uniform distribution Weibull distribution Density Function Probability Function Quantile distribution Other Functions Gamma function Natural logarithm of the Gamma function Digamma function Trigamma function Error function Complementary error function Base SAS Equivalents Freq, Summary, Sort Rank, Corr, Univariate Copyright (C) ISE GmbH - All Rights Reserved 14

15 R on Oracle Exadata Oracle Exadata Storage Server Oracle Database Server Compute Intensive Processing Oracle Database Server Compute Intensive Processing Data Intensive Processing Oracle Exadata Storage Server Data Intensive Processing Oracle Exadata Storage Server Data Intensive Processing Oracle Database Server Compute Intensive Processing Oracle Exadata Storage Server Data Intensive Processing Clustered Database Servers High Bandwidth Interconnect Massively Parallel Storage Copyright (C) ISE GmbH - All Rights Reserved 15

16 R on Oracle Exadata Database server Up to 256 GB memory Up to 2x8cores 8 times in full rack Exadata Cell Servers R Falsh cache up to 1,6 TB per cell Infiniband connections to DB Server Offloading 14 times in full rack Offloading for ore and ODM packages, cell use Spawing many R processes over all database servers Copyright (C) ISE GmbH - All Rights Reserved 16

17 R example implementation is one 100% child of the Axel Springer corporation and forms part of the media concern s extremely successful digital strategy is one of the three major digital markets for real estate in Germany has a complety oracle solution with exadata and exalytics Copyright (C) ISE GmbH - All Rights Reserved 17

18 R Example implementation Starting on R client op <- options(digits.secs=2) Sys.time() #Loading libraries require(party) #connecting to exadata.exa() #Loading data out of database dat <- ore.pull(immonet_data) #Building regression tree ct <- ctree(data = dat, control = ctree_control(maxdepth = 3), formula = rexa.calc ~ rpqm.calc + auss2.calc + flaechen.wohnflaeche + flaechen.anzahl_zimmer + freitexte.objekttitel.nchar) #Plot tree plot(ct, terminal_panel = node_boxplot(ct, id = FALSE, cex = 0)) " 21:33:03.85 CET" - " 21:33:58.31 CET" Copyright (C) ISE GmbH - All Rights Reserved 18

R Example implementation Working with R on

21 R Example implementation Starting on R remote op <- options(digits.secs=2) Sys.time() #connect.exa() #Calc mod <- ore.doeval( function(param) { require(party) dat <- ore.pull(immonet_data) ct <- ctree(data = dat, control = ctree_control(maxdepth = 3), formula = rexa.calc ~ rpqm.calc + auss2.calc + flaechen.wohnflaeche + flaechen.anzahl_zimmer + freitexte.objekttitel.nchar) pdf("2_client.pdf") plot(ct) dev.off() ct}) op <- options(digits.secs=2) Sys.time() Copyright (C) ISE GmbH - All Rights Reserved 21

23 R Example implementation Working embedded - Detail rq*eval() Table Functions rqeval(), rqtableeval(), rqgroupeval(), rqroweval() Output only parts of the calculation, num rows Output table definition a query specifying the format of the result If NULL, output is a serialized BLOB Group name (optional) Name of the grouping column Number of rows (optional) number of rows to provide to function at one time Copyright (C) ISE GmbH - All Rights Reserved 23

Oracle Advanced Analytics Oracle R Enterprise & Oracle Data Mining

Oracle Advanced Analytics Oracle R Enterprise & Oracle Data Mining R The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated