Introduction Predictive Analytics Tools: Weka, R!

Transcription

1 Introduction Predictive Analytics Tools: Weka, R! Predictive Analytics Center of Excellence San Diego Supercomputer Center University of California, San Diego!

2 Available Data Mining Tools! COTs:! n IBM Intelligent Miner! n SAS Enterprise Miner! n Oracle ODM! n Microstrategy! n Microsoft DBMiner! n Pentaho! n Matlab! n Teradata! Open Source:! n WEKA! n KNIME! n Orange! n RapidMiner! n NLTK! n R! n Rattle! 2

3 Agenda! WEKA! Intro and background" Data Preparation" Creating Models/ Applying Algorithms" Evaluating Results" R! R Background" R Basics" Outline" R-Studio Overview" Hands On (homework)"

4 WEKA!

5 Download and Install WEKA! Website: 5 7/1/14

6 What is WEKA?! Waikato Environment for Knowledge Analysis! WEKA is a data mining/machine learning application developed by Department of Computer Science, University of Waikato, New Zealand" WEKA is open source software in JAVA " WEKA is a collection machine learning algorithms and tools for data mining tasks" data pre-processing, classification, regression, clustering, association, and visualization. " WEKA is well-suited for developing new machine learning schemes " WEKA is a bird found only in New Zealand.! 6 7/1/14

7 Advantages of Weka! Free availability! under the GNU General Public License" Portability! fully implemented in the Java programming language and thus runs on almost any modern computing platforms" Windows, Mac OS X and Linux" Comprehensive collection of data preprocessing and modeling techniques! Supports standard data mining tasks: data preprocessing, clustering, classification, regression, visualization, and feature selection." Easy to use GUI! Provides access to SQL databases! using Java Database Connectivity and can process the result returned by a database query."

8 Disadvantages!! Sequence modeling is not covered by the algorithms included in the Weka distribution! Not capable of multi-relational data mining!

9 WEKA Walk Through: Main GUI! Three graphical user interfaces! The Explorer (exploratory data analysis)" pre-process data" build classifiers " cluster data" find associations" attribute selection" data visualization" The Experimenter (experimental environment)" used to compare performance of different learning schemes " The KnowledgeFlow (new process model inspired interface) " Java-Beans-based interface for setting up and running machine learning experiments." Command line Interface ( Simple CLI )! More at: 9 7/1/14

10 1 0 7/1/14

11 WEKA:: Explorer: Preprocess! Importing data! Data format" Uses flat text files to describe the data" Data can be imported from a file in various formats: " ARFF, CSV, C4.5, binary" Data can also be read from a URL or from an SQL database (using JDBC)"

12 WEKA:: ARFF file age sex { female, chest_pain_type { typ_angina, asympt, non_anginal, cholesterol exercise_induced_angina { no, class { present, 63,male,typ_angina,233,no,not_present 67,male,asympt,286,yes,present 67,male,asympt,229,yes,present 38,female,non_anginal,?,no,not_present...! A more thorough description is available here

13 University of Waikato 7/1/14 1 3

14 University of Waikato 7/1/14 1 4

15 Weka: Explorer:Preprocess! Preprocessing data! Visualization" Filtering algorithms " filters can be used to transform the data (e.g., turning numeric attributes into discrete ones) and make it possible to delete instances and attributes according to specific criteria." Removing Noisy Data" Adding Additional Attributes" Remove Attributes"

16

17

18 WEKA:: Explorer: Preprocess! Used to define filters to transform Data.! WEKA contains filters for:! Discretization, normalization, resampling, attribute selection, transforming, combining attributes, etc"

19 University of Waikato 7/1/14 1 9

20

21

22 Explorer: Visualize! Visualization very useful in practice! help determine difficulty of the learning problem" WEKA can visualize single attributes (1-d) and pairs of attributes (2-d)! Color-coded class values! Jitter option to deal with nominal attributes (and to detect hidden data points)! Zoom-in function! 7/1/14 22

23 University of Waikato 7/1/14 2 3

24 University of Waikato 7/1/14 2 4

25 Explorer: Attribute Selection! Panel that can be used to investigate which (subsets of) attributes are the most predictive ones! Attribute selection methods contain two parts:! A search method: best-first, forward selection, random, exhaustive, genetic algorithm, ranking! An evaluation method: correlation-based, wrapper, information gain, chi-squared, " Very flexible: WEKA allows (almost) arbitrary combinations of these two! 2 5 7/1/14

26 WEKA:: Explorer: building classifiers! Classifiers in WEKA are models for predicting nominal or numeric quantities! Implemented learning schemes include:! Decision trees and lists, instance-based classifiers, support vector machines, multi-layer perceptrons, logistic regression, Bayes nets, " Meta -classifiers include:! Bagging, boosting, stacking, error-correcting output codes, locally weighted learning, "

27 University of Waikato 7/1/14 2 7

28 University of Waikato 7/1/14 2 8

29 WEKA:: Explorer: building Cluster! WEKA contains clusters for finding groups of similar instances in a dataset! Implemented schemes are:! k-means, EM, Cobweb, X-means, FarthestFirst" Clusters can be visualized and compared to true clusters (if given)! Evaluation based on loglikelihood if clustering scheme produces a probability distribution!

30 Explorer: Finding associations! WEKA contains an implementation of the Apriori algorithm for learning association rules! Works only with discrete data" Can identify statistical dependencies between groups of attributes:! milk, butter bread, eggs (with confidence 0.9 and support 2000)" Apriori can compute all rules that have a given minimum support and exceed a given confidence! 7/1/14 30

31 References and Resources! References:! WEKA website: WEKA Tutorial:" Machine Learning with WEKA: A presentation demonstrating all graphical user interfaces (GUI) in Weka. " A presentation which explains how to use Weka for exploratory data mining. " WEKA Data Mining Book:" Ian H. Witten and Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques (Second Edition)" WEKA Wiki: Main_Page"

32 R Environment: R Studio!

33 Downloading R/ R Studio!

34 What is R?! An Environment! R is an integrated suite of software facilities for data manipulation, calculation and graphical facilities for data analysis and display. " Effective data handling and storage" Suite of operators for calculations on arrays" Large, coherent, integrated collection of intermediate tools for data analysis " Programming language, run time environment" Developed at Bell Labs! GNU open source software! Under the terms of the Free Software Foundation's GNU General Public License" Open Source implementation of S-Plus language! Well-developed, simple and effective programming language" Highly extensible!

35 R Features! Software package designed for data analysis and graphical representation! Interactive, but may also be used programmatically! Platform independence! Compiles and runs on a wide variety platforms, Unix base, Windows and MacOS. " Free, open source code! Engaged community! over 4,200 user-contributed packages" Extendable! User defined functions" > 4000 packages available in the CRAN package repository" Supports extensions / add-ons (i.e. rapache)" Compatible with other languages (i.e. SQL, perl, C)" Data Import" Pre-processing data from different sources" Scalability! Parallel R packages "

36 Clustering! Classification! Association Rules! Sequential patterns! Time Series! Statistics! Graphics! Data manipulation! R packages for DM!

37 Data Mining! linear models (lm)! generalized linear models(glm)! generalized additive models (gam)! linear mixed effects models(lme)! quantile regression (qr)! vector general additive models(vgam)! lasso, ridge, and elastic net models (glmnet)! non-linear models (nlm)! linear mixed effects models (nlmer)! linear discriminant analysis (lda)! quadratic discriminate analysis (qda)! trees (tree)! random forests (randomforrest)! support vector machines (svm)! neural networks (nnet)! k-nearest neighbors (knn)! kmeans!

38 Big Data Options! lapply-based parallelism! multicore library" snow library" foreach-based parallelism! domc backend" dosnow backend" dompi backend" Map/Reduce- (Hadoop-) based parallelism! Hadoop streaming with R mappers/reducers" Rhadoop (rmr, rhdfs, rhbase)" RHIPE" Poor-man's Parallelism! lots of Rs running" lots of input files" Hands-off Parallelism! OpenMP support compiled into R build" Dangerous!"

39 R Considerations/Limitations! Command Line Interface! Performance! Memory Limits! memory limits dependent on the build, (32-bit vs. 64-bit)" 32-bit build of R on Windows is dependent on the underlying OS version" Syntax curiosities! Learning curve!

40 R-Studio Overview! R-Studio is an integrated development environment to support R code. R-Studio runs in two ways: Desktop version for Linux, Mac, Windows: Single user, perfect for laptop or desktop machine Server Version for Linux: Allows an number of remote users to run R-Studio within a web-browser, facilitates sharing of code and data among team members

41 General View of R-Studio Editor Window! Project Window:! Currently loaded! Workspace, and! history! pop-up :! Multi-tab display:! Shows graphics,! Current directory and! loaded packages! Console: Run R! Commands!

42 The Fundamentals! Launch R! Quit R! q() " Getting Help! help(package_name) or?(package_name) or help start()" example(package_name)"??(keyword)" library(help= package_name )"

43 R environmental commands! list objects" ls() " objects()" list files in current directory" list.files()" list current directory" getwd()" set working directory" setwd()" remove objects" rm()" Workspace versus console! Clear workspace" rm(list=ls())" Clear console" (control, L)" The Basics!

44 The Basics (Naming Variables)! Requirements! Case sensitive, names must start with letter or '. " Only letters, numbers, underscores and. s" Special keywords! break, else, FALSE, for, function, if, Inf, NA, NaN, next, repeat, return, TRUE, while" Names not limited in length!

45 The Basics All entities in are called objects! arrays, vectors, matrices, functions, lists, data frames, factors" Expressions vs. assignments! 10+10" my.age <- 23" my.age < - 23 (note the added space)" age<- c(my.age, 14, 59, 32)" my.age == 40" Data Types! Numeric, Integer, Complex, Logical, Character" Function call!!> mean(weight)"!

46 Summary of Data Structures! Linear! Rectangular! Homogeneous" Vectors" Matrices" Heterogeneous" Lists" Data Frames" " Vectors and Matrices must contain same data type! Character Type will trump numeric: Values will be forced into characters!

47 The Basics (Functions)! Basic functions! mean(age)" sd(age)" sqrt(var(age))" TIP: to list all function in search path" sapply(search(), ls, all.names = TRUE) User Defined functions! Score <- age * 10;" Using the correct functions for the given data type! apply() family "

48 Function Components! writelines(text= text, con = stdout(), sep = "\n", usebytes = FALSE)! function name: writelines( 146.6, poprate.txt, sep = "\n )" parentheses: writelines( 146.6, poprate.txt, sep = "\n )! commas: writelines( 146.6, poprate.txt, sep = "\n )" first argument: writelines( 146.6, poprate.txt, sep = "\n )" second argument: writelines( 146.6, poprate.txt, sep = "\n )"" optional argument: writelines( 146.6, poprate.txt, "\n )"

49 Importing Data/Exporting Data! Flat Files! Import: > AHW <- read.csv( AHW_1.csv, header=true)" >weatherdata <- read.table(file="c:/work/dm1/weather.csv", header=true, sep=",") " Export: > USTemps=read.table(file=file.choose(),header=TRUE)" Databases! Import" connection <- dbconnect(driver, user, password, host, dbname)" > AHW <- dbsendquery(connection, SELECT * FROM AHW ) Export" > connnection <- dbconnect(driver, user, password, host,dbname)" > dbwritetable (con, AHW, AHW) R objects! Import: > load( AHW.Rdata )" Export: > save(ahw, file= New_AHW.Rdata )" Web! connection <-url( )" AHW <- read.csv(con, header=true)" Plots! png(filename="c:/r/figure.png", height=295, width=300, bg="white")" pdf(file="c:/r/figure.pdf", height=3.5, width=5)" Dev.off() #turn off device driver (to flush output to png/pdf)"

50 Loading dataset to R-Studio (Simple text file) Name of data frame! to be created with! imported data! Options for parsing! the text data into! fields and values! How data frame will! look once the data! are imported!

51 Extending R! Install a package! from command line" "> install.package( name_of_package )" from GUI" Packages & Data > Package Installer" Load Library (to use installed package)" > library(name_of_package)" Example " > library(markdown)" Use Library Function! > function_name(parameters)" Example " > markdowntohtml("example.md")" "

52 More Information! The R Manuals! And Introduction to R! Books!

53 Other Resources! /server irc.freenode.net/join #R!"

54 the end!