# Decision Trees (Sections )

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Decision Trees (Sections ) Non-metric methods CART (Classification & regression Trees) Number of splits Query selection & node impurity Multiway splits When to stop splitting? Pruning Assignment of leaf node labels Feature choice Multivariate decision trees Missing attributes

2 Data Type and Scale Data type refers to the degree of quantization in the data binary feature: have exactly two values (Yes-No response) discrete feature: finite, usually small, number of possible values (gray values in a digital image) continuous feature: any real value in a fixed range of values Data scale indicates the relative significance of numbers qualitative scales Nominal (categorical): not really a scale since numerical values are simply used as names; e.g., (yes, no) response can be coded as (0,1) or (1,0) or (50,100); numerical values are meaningless in a quantitative sense Ordinal: numbers have meaning only in relation to one another (e.g., one value is larger than the other); e.g., scales (1, 2, 3), (10, 20, 30), and (1, 20, 300) are all equivalent from an ordinal viewpoint (military rank) quantitative scales Intervalscale: separation between numbers has meaning; a unit of measurement exists, and the interpretation of the numbers depends on this unit (Fahrenheit scale for temperature. Equal differences on this scale represent equal differences in temperature, but a temperature of 30 degrees is not twice as warm as one of 15 degrees. Ratioscale: numbers have an absolute meaning; an absolute zero exists along with a unit of measurement, so the ratio between two numbers has meaning (height)

3 Also known as Hierarchical classifiers Tree classifiers Multistage classification Divide & conquer strategy Decision Trees A single-stage classifier assigns a test pattern X to one of C classes in a single step: compute the posteriori probability for each class & choose the class with the max posteriori Limitations of single-stage classifier A common set of features is used for distinguishing all the classes; for large no. of classes, this common feature set may not be the best for specific pairs of classes Requires a large no. of features when the no. of classes is large For each test pattern, C posteriori probs. need to be computed Does not perform well when classes have multimodal distributions Not easy to handle nominal data (discrete features without any natural notion of similarity or even ordering)

4 Decision Trees Most pattern recognition methods address problems where feature vectors are real valued and there exists some notion of metric Suppose the classification involves nominal data attributes that are discrete & without any natural notion of similarity or even ordering Describe patterns by using a list of attributes rather than by vectors of real numbers Describe a fruit by {color, texture, taste, size} {red, shiny, sweet, small} How to learn categories using such non-metric data?

5 Decision Trees Classify a pattern through a sequence of questions (20-question game); next question asked depends on the answer to the current question This approach is particularly useful for non-metric data; questions can be asked in a yes-no or true-false style that do not require any notion of metric Sequence of questions is displayed in a directed decision tree Root node, links or branches, leaf or terminal nodes Classification of a pattern begins at the root node until we reach the leaf node; pattern is assigned the category of the leaf node Benefit of decision tee: Interpretability: a tree can be expressed as a logical expression Rapid classification: a sequence of simple queries Higher accuracy & speed:

6 Decision Tree Seven-class, 4-feature classification problem Apple= (green AND medium) OR (red AND medium) = (Medium AND NOT yellow)

7 How to Grow A Tree? Given a set Dof labeled training samples and a set of features How to organize the testsinto a tree? Each test or question involves a single feature or subset of features A decision tree progressively splits the training set into smaller and smaller subsets Pure node:all the samples at that node have the same class label; no need to further split a pure node Recursive tree-growing process: Given data at a node, decide the node as a leaf node or find another feature to split the node This generic procedure is called CART (Classification & Regression Trees)

8 Classification & Regression Tree (CART) Six types of questions Should the attributes (answers to questions) be binary or multivalued? In other words, how many splits should be made at a node? Which feature or feature combination should be tested at a node? When should a node be declared a leaf node? If the tree becomes too large, can it be prunedto make it smaller and simpler? If a leaf node is impure, how should category label be assigned to it? How should missing data be handled?

9 Number of Splits Binary tree: every decision can be represented using just binaryoutcome; tree of Fig 8.1 can be equivalently written as

10 Query Selection & Node Impurity Which attribute test or query should be performed at each node? Fundamental principle of simplicity: obtain simple, compact trees with few nodes Seek a query T at node N to make the descendent nodes as pure as possible Query of the form x i <= x is leads to hyperplanar decision boundaries that are perpendicular to the coordinate axes (monothetic tree; one feature/node))

11 Query Selection and Node Impurity P(ω j ): fraction of patterns at node N in category ω j Node impurity is 0 when all patterns at the node are of the same category Impurity becomes maximum when all the classes at node N are equally likely Entropy impurity Gini impurity Expected error rate at node N if the category label is selected randomly from the class distribution present at N 1 i N P ω P ω P ω 2 ( ) = ( i ) ( j ) = 1 ( j ) (3) i j 2 j Misclassification impurity Minimum probability that a training pattern will be misclassified at N

12 Query Selection and Node Impurity Given a partial tree down to node N, what value sto choose for property test T? Choose the query at node N to decrease the impurity as much as possible Drop in impurity is defined as Best query value s is the choice for test T that maximizes the drop in impurity Optimization in Eq. (5) is local done at a single node. This greedy method does not assure that global optimum of impurity will be reached

13 When to Stop Splitting? If we continue to grow the tree fully until each leaf node corresponds to the lowest impurity, then the data have typically been overfit; in the limit, each leaf node has only one pattern! If splitting is stopped too early, error on training data is not sufficiently low and performance will suffer Validation and cross-validation Continue splitting until error on validation set is minimum Cross-validation relies on several independently chosen subsets Stop splitting when the best candidate split at a node reduces the impurity by less than the preset amount (threshold) How to set the threshold? Stop when a node has small no. of points or some fixed percentage of total training set (say 5%) Trade off between tree complexity or size vs. test set accuracy

14 Pruning Occasionally, stopping tree splitting suffers from the lack of sufficient look ahead A stopping condition may be met too early for overall optimal recognition accuracy Pruning is the inverse of splitting Grow the tree fully until leaf nodes have minimum impurity. Then all pairs of leaf nodes (with a common antecedent node) are considered for elimination Any pair whose elimination yields a satisfactory (small) increase in impurity is eliminated, and the common antecedent node is declared as leaf node

15 Example 1: A Simple Tree

16 Example 1. Simple Tree Entropy impurity at nonterminal nodes is shown in red and impurity at each leaf node is 0 Instability or sensitivity of tree to training points; alteration of a single point leads to a very different tree; due to discrete & greedy nature of CART

17 Decision Tree

18 Feature Choice As with most pattern recognition methods, tree-based methods work best if proper features are used; preprocessing by PCA can be effective because it finds important axes

19 Multivariate Decision Trees Allow splits that are not parallel to feature axes

20 Popular Tree Algorithms ID3: Third in a series of Interactive dichotomizer procedures. Intended to be used for nominal (unordered) inputs only. If the problem involves real-valued variables, they are first binned into intervals, each interval being treated as an unordered nominal attribute C4.5: the successor and refinement of ID3 is the most popular classification tree method. Real valued variables are treated the same as in CART. Multiway (B>2) splits are used with nominal data

21 Missing Attributes Missing attributes during training, during classification, or both Naïve approach; delete any such deficient patterns Calculate impurities at anode N using only the attribute information present

22 Decision Tree IRIS data Used first 25 samples from each category Two of the four features x1 and x2 do not appear in the tree feature selection capability Sethi and Sarvaraydu, IEEE Trans. PAMI, July 1982

23 Decision Tree for IRIS data 2-D Feature space representation of the decision boundaries X 4 (Petal width) Setosa Virginica 1.65 Versicolor X 3 (Petal length)

24 Decision Tree Hand printed digits dimensional patterns from 10 classes; 16 patterns/class. Independent test set of 40 samples

25 Properties of a Metric A metric D(.,.) is merely a function that gives a generalized scalar distance between two argument patterns A metric must have four properties: For all vectors a,b,and c, the properties are: Non-negativity: D(a, b) >= 0 reflexivity: D(a, b) = 0 if and only if a= b symmetry: D(a,b) = D(b,a) triangle inequality: D(a, b) + D(b, c) >= D(a, c) It is easy to verify that the Euclidean formula for distance in d dimensions possesses the properties of metric D d 1/ 2 2 ( a, b) ( ) = k = 1 a k b k

26 Scaling the Data Although one can always compute the Euclidean distance between two vectors, the results may or may not be meaningful If the space is transformed by multiplying each coordinate by an arbitrary constant, the Euclidean distance in the transformed space is different from original distance relationship; such scale changes can have a major impact on NN classifiers

27 General Class of Metrics Minkowski metric L d 1/ k k k ( a, b) = ai bi i= 1 Manhattan distance L d 1 ( a, b) = a i b i i= 1

28 Example 2: Surrogate Splits and Missing Attributes

### COMP3420: Advanced Databases and Data Mining. Classification and prediction: Introduction and Decision Tree Induction

COMP3420: Advanced Databases and Data Mining Classification and prediction: Introduction and Decision Tree Induction Lecture outline Classification versus prediction Classification A two step process Supervised

### Data Mining for Knowledge Management. Classification

1 Data Mining for Knowledge Management Classification Themis Palpanas University of Trento http://disi.unitn.eu/~themis Data Mining for Knowledge Management 1 Thanks for slides to: Jiawei Han Eamonn Keogh

### Chapter 4: Non-Parametric Classification

Chapter 4: Non-Parametric Classification Introduction Density Estimation Parzen Windows Kn-Nearest Neighbor Density Estimation K-Nearest Neighbor (KNN) Decision Rule Gaussian Mixture Model A weighted combination

### Classification/Decision Trees (II)

Classification/Decision Trees (II) Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Right Sized Trees Let the expected misclassification rate of a tree T be R (T ).

### Data Mining Classification: Decision Trees

Data Mining Classification: Decision Trees Classification Decision Trees: what they are and how they work Hunt s (TDIDT) algorithm How to select the best split How to handle Inconsistent data Continuous

### Gerry Hobbs, Department of Statistics, West Virginia University

Decision Trees as a Predictive Modeling Method Gerry Hobbs, Department of Statistics, West Virginia University Abstract Predictive modeling has become an important area of interest in tasks such as credit

### Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

### Lecture 10: Regression Trees

Lecture 10: Regression Trees 36-350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,

### Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation. Lecture Notes for Chapter 4. Introduction to Data Mining

Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data

### Medical Information Management & Mining. You Chen Jan,15, 2013 You.chen@vanderbilt.edu

Medical Information Management & Mining You Chen Jan,15, 2013 You.chen@vanderbilt.edu 1 Trees Building Materials Trees cannot be used to build a house directly. How can we transform trees to building materials?

### Data mining techniques: decision trees

Data mining techniques: decision trees 1/39 Agenda Rule systems Building rule systems vs rule systems Quick reference 2/39 1 Agenda Rule systems Building rule systems vs rule systems Quick reference 3/39

### Data Mining with R. Decision Trees and Random Forests. Hugh Murrell

Data Mining with R Decision Trees and Random Forests Hugh Murrell reference books These slides are based on a book by Graham Williams: Data Mining with Rattle and R, The Art of Excavating Data for Knowledge

### An Introduction to Data Mining. Big Data World. Related Fields and Disciplines. What is Data Mining? 2/12/2015

An Introduction to Data Mining for Wind Power Management Spring 2015 Big Data World Every minute: Google receives over 4 million search queries Facebook users share almost 2.5 million pieces of content

### Learning Example. Machine learning and our focus. Another Example. An example: data (loan application) The data and the goal

Learning Example Chapter 18: Learning from Examples 22c:145 An emergency room in a hospital measures 17 variables (e.g., blood pressure, age, etc) of newly admitted patients. A decision is needed: whether

### Decision-Tree Learning

Decision-Tree Learning Introduction ID3 Attribute selection Entropy, Information, Information Gain Gain Ratio C4.5 Decision Trees TDIDT: Top-Down Induction of Decision Trees Numeric Values Missing Values

### Data Mining Techniques Chapter 6: Decision Trees

Data Mining Techniques Chapter 6: Decision Trees What is a classification decision tree?.......................................... 2 Visualizing decision trees...................................................

### Data Mining 5. Cluster Analysis

Data Mining 5. Cluster Analysis 5.2 Fall 2009 Instructor: Dr. Masoud Yaghini Outline Data Structures Interval-Valued (Numeric) Variables Binary Variables Categorical Variables Ordinal Variables Variables

### Knowledge Discovery and Data Mining. Structured vs. Non-Structured Data

Knowledge Discovery and Data Mining Unit # 2 1 Structured vs. Non-Structured Data Most business databases contain structured data consisting of well-defined fields with numeric or alphanumeric values.

### Decision Trees from large Databases: SLIQ

Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values

### Data Mining. Practical Machine Learning Tools and Techniques. Classification, association, clustering, numeric prediction

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 2 of Data Mining by I. H. Witten and E. Frank Input: Concepts, instances, attributes Terminology What s a concept? Classification,

### Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 2. Tid Refund Marital Status

Data Mining Classification: Basic Concepts, Decision Trees, and Evaluation Lecture tes for Chapter 4 Introduction to Data Mining by Tan, Steinbach, Kumar Classification: Definition Given a collection of

### Introduction to Statistical Machine Learning

CHAPTER Introduction to Statistical Machine Learning We start with a gentle introduction to statistical machine learning. Readers familiar with machine learning may wish to skip directly to Section 2,

### Classification and Prediction

Classification and Prediction Slides for Data Mining: Concepts and Techniques Chapter 7 Jiawei Han and Micheline Kamber Intelligent Database Systems Research Lab School of Computing Science Simon Fraser

### Data Mining Practical Machine Learning Tools and Techniques

Some Core Learning Representations Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter of Data Mining by I. H. Witten and E. Frank Decision trees Learning Rules Association rules

### A variable is the mathematical model for a characteristic observed or measured in an experiment, survey, or observational study.

Types of Variables This material is covered partially by Sall, Lehman and Creighton (2001) Chapters 1 and 2, by Milton (1992) Chapters 1 and 4, by Sokal and Rohlf (1995) Chapters 1 and 2, by Koopmans (1987),

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Data, Measurements, Features

Data, Measurements, Features Middle East Technical University Dep. of Computer Engineering 2009 compiled by V. Atalay What do you think of when someone says Data? We might abstract the idea that data are

### Data Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining

Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar What is data exploration? A preliminary exploration of the data to better understand its characteristics.

### Email Spam Detection A Machine Learning Approach

Email Spam Detection A Machine Learning Approach Ge Song, Lauren Steimle ABSTRACT Machine learning is a branch of artificial intelligence concerned with the creation and study of systems that can learn

### Less naive Bayes spam detection

Less naive Bayes spam detection Hongming Yang Eindhoven University of Technology Dept. EE, Rm PT 3.27, P.O.Box 53, 5600MB Eindhoven The Netherlands. E-mail:h.m.yang@tue.nl also CoSiNe Connectivity Systems

### Data Exploration Data Visualization

Data Exploration Data Visualization What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping to select

### Data Mining Part 5. Prediction

Data Mining Part 5. Prediction 5.1 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Classification vs. Numeric Prediction Prediction Process Data Preparation Comparing Prediction Methods References Classification

### Decision Trees. JERZY STEFANOWSKI Institute of Computing Science Poznań University of Technology. Doctoral School, Catania-Troina, April, 2008

Decision Trees JERZY STEFANOWSKI Institute of Computing Science Poznań University of Technology Doctoral School, Catania-Troina, April, 2008 Aims of this module The decision tree representation. The basic

### Data Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining

Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 8/05/2005 1 What is data exploration? A preliminary

### Big Data Analytics CSCI 4030

High dim. data Graph data Infinite data Machine learning Apps Locality sensitive hashing PageRank, SimRank Filtering data streams SVM Recommen der systems Clustering Community Detection Web advertising

### Decision Trees What Are They?

Decision Trees What Are They? Introduction...1 Using Decision Trees with Other Modeling Approaches...5 Why Are Decision Trees So Useful?...8 Level of Measurement... 11 Introduction Decision trees are a

### Data Mining: Exploring Data. Lecture Notes for Chapter 3. Slides by Tan, Steinbach, Kumar adapted by Michael Hahsler

Data Mining: Exploring Data Lecture Notes for Chapter 3 Slides by Tan, Steinbach, Kumar adapted by Michael Hahsler Topics Exploratory Data Analysis Summary Statistics Visualization What is data exploration?

### 6 Classification and Regression Trees, 7 Bagging, and Boosting

hs24 v.2004/01/03 Prn:23/02/2005; 14:41 F:hs24011.tex; VTEX/ES p. 1 1 Handbook of Statistics, Vol. 24 ISSN: 0169-7161 2005 Elsevier B.V. All rights reserved. DOI 10.1016/S0169-7161(04)24011-1 1 6 Classification

### Metric Spaces. Chapter 7. 7.1. Metrics

Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some

### TOWARDS SIMPLE, EASY TO UNDERSTAND, AN INTERACTIVE DECISION TREE ALGORITHM

TOWARDS SIMPLE, EASY TO UNDERSTAND, AN INTERACTIVE DECISION TREE ALGORITHM Thanh-Nghi Do College of Information Technology, Cantho University 1 Ly Tu Trong Street, Ninh Kieu District Cantho City, Vietnam

### Smart Grid Data Analytics for Decision Support

1 Smart Grid Data Analytics for Decision Support Prakash Ranganathan, Department of Electrical Engineering, University of North Dakota, Grand Forks, ND, USA Prakash.Ranganathan@engr.und.edu, 701-777-4431

### T3: A Classification Algorithm for Data Mining

T3: A Classification Algorithm for Data Mining Christos Tjortjis and John Keane Department of Computation, UMIST, P.O. Box 88, Manchester, M60 1QD, UK {christos, jak}@co.umist.ac.uk Abstract. This paper

### Measurement and Measurement Scales

Measurement and Measurement Scales Measurement is the foundation of any scientific investigation Everything we do begins with the measurement of whatever it is we want to study Definition: measurement

### Chapter 20: Data Analysis

Chapter 20: Data Analysis Database System Concepts, 6 th Ed. See www.db-book.com for conditions on re-use Chapter 20: Data Analysis Decision Support Systems Data Warehousing Data Mining Classification

### Machine Learning using MapReduce

Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

### Data Mining - Evaluation of Classifiers

Data Mining - Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010

### Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

### Classification algorithm in Data mining: An Overview

Classification algorithm in Data mining: An Overview S.Neelamegam #1, Dr.E.Ramaraj *2 #1 M.phil Scholar, Department of Computer Science and Engineering, Alagappa University, Karaikudi. *2 Professor, Department

### !"!!"#\$\$%&'()*+\$(,%!"#\$%\$&'()*""%(+,'-*&./#-\$&'(-&(0*".\$#-\$1"(2&."3\$'45"

!"!!"#\$\$%&'()*+\$(,%!"#\$%\$&'()*""%(+,'-*&./#-\$&'(-&(0*".\$#-\$1"(2&."3\$'45"!"#"\$%&#'()*+',\$\$-.&#',/"-0%.12'32./4'5,5'6/%&)\$).2&'7./&)8'5,5'9/2%.%3%&8':")08';:

### Introduction to Statistics

1 1.1 Introduction Introduction to Statistics Statistics is a collection of methods for planning experiments, obtaining data, and then organizing, summarizing, presenting, analyzing, interpreting and drawing

### Data Preprocessing. Week 2

Data Preprocessing Week 2 Topics Data Types Data Repositories Data Preprocessing Present homework assignment #1 Team Homework Assignment #2 Read pp. 227 240, pp. 250 250, and pp. 259 263 the text book.

### Connecting Segments for Visual Data Exploration and Interactive Mining of Decision Rules

Journal of Universal Computer Science, vol. 11, no. 11(2005), 1835-1848 submitted: 1/9/05, accepted: 1/10/05, appeared: 28/11/05 J.UCS Connecting Segments for Visual Data Exploration and Interactive Mining

### Continuous Metrics for Efficient and Effective Testing Laura J. Freeman & Bram Lillard NDIA National Test and Evaluation Conference March 15, 2012

Continuous Metrics for Efficient and Effective Testing Laura J. Freeman & Bram Lillard NDIA National Test and Evaluation Conference March 15, 2012 3/20/2012-1 Everyday Example Gallup Poll: Americans Divided

### Methods for statistical data analysis with decision trees

Methods for statistical data analysis with decision trees Problems of the multivariate statistical analysis In realizing the statistical analysis, first of all it is necessary to define which obects and

### An Overview and Evaluation of Decision Tree Methodology

An Overview and Evaluation of Decision Tree Methodology ASA Quality and Productivity Conference Terri Moore Motorola Austin, TX terri.moore@motorola.com Carole Jesse Cargill, Inc. Wayzata, MN carole_jesse@cargill.com

### Data quality in Accounting Information Systems

Data quality in Accounting Information Systems Comparing Several Data Mining Techniques Erjon Zoto Department of Statistics and Applied Informatics Faculty of Economy, University of Tirana Tirana, Albania

### Psychology 312: Lecture 6 Scales of Measurement. Slide #1. Scales of Measurement Reliability, validity, and scales of measurement.

Psychology 312: Lecture 6 Scales of Measurement Slide #1 Scales of Measurement Reliability, validity, and scales of measurement. In this lecture we will discuss scales of measurement. Slide #2 Outline

### Data Mining Practical Machine Learning Tools and Techniques

Data ining Practical achine Learning Tools and Techniques Slides for Chapter 2 of Data ining by I. H. Witten and E. rank Outline Terminology What s a concept Classification, association, clustering, numeric

### Knowledge Discovery and Data Mining

Knowledge Discovery and Data Mining Unit # 6 Sajjad Haider Fall 2014 1 Evaluating the Accuracy of a Classifier Holdout, random subsampling, crossvalidation, and the bootstrap are common techniques for

### Sub-class Error-Correcting Output Codes

Sub-class Error-Correcting Output Codes Sergio Escalera, Oriol Pujol and Petia Radeva Computer Vision Center, Campus UAB, Edifici O, 08193, Bellaterra, Spain. Dept. Matemàtica Aplicada i Anàlisi, Universitat

### Statistics Review PSY379

Statistics Review PSY379 Basic concepts Measurement scales Populations vs. samples Continuous vs. discrete variable Independent vs. dependent variable Descriptive vs. inferential stats Common analyses

### Extend Table Lens for High-Dimensional Data Visualization and Classification Mining

Extend Table Lens for High-Dimensional Data Visualization and Classification Mining CPSC 533c, Information Visualization Course Project, Term 2 2003 Fengdong Du fdu@cs.ubc.ca University of British Columbia

### Introduction to Learning & Decision Trees

Artificial Intelligence: Representation and Problem Solving 5-38 April 0, 2007 Introduction to Learning & Decision Trees Learning and Decision Trees to learning What is learning? - more than just memorizing

### Quantitative Research Methods II. Vera E. Troeger Office: Office Hours: by appointment

Quantitative Research Methods II Vera E. Troeger Office: 0.67 E-mail: v.e.troeger@warwick.ac.uk Office Hours: by appointment Quantitative Data Analysis Descriptive statistics: description of central variables

### Professor Anita Wasilewska. Classification Lecture Notes

Professor Anita Wasilewska Classification Lecture Notes Classification (Data Mining Book Chapters 5 and 7) PART ONE: Supervised learning and Classification Data format: training and test data Concept,

### EXPLORING & MODELING USING INTERACTIVE DECISION TREES IN SAS ENTERPRISE MINER. Copyr i g ht 2013, SAS Ins titut e Inc. All rights res er ve d.

EXPLORING & MODELING USING INTERACTIVE DECISION TREES IN SAS ENTERPRISE MINER ANALYTICS LIFECYCLE Evaluate & Monitor Model Formulate Problem Data Preparation Deploy Model Data Exploration Validate Models

### Statistical Data Mining. Practical Assignment 3 Discriminant Analysis and Decision Trees

Statistical Data Mining Practical Assignment 3 Discriminant Analysis and Decision Trees In this practical we discuss linear and quadratic discriminant analysis and tree-based classification techniques.

### 131-1. Adding New Level in KDD to Make the Web Usage Mining More Efficient. Abstract. 1. Introduction [1]. 1/10

1/10 131-1 Adding New Level in KDD to Make the Web Usage Mining More Efficient Mohammad Ala a AL_Hamami PHD Student, Lecturer m_ah_1@yahoocom Soukaena Hassan Hashem PHD Student, Lecturer soukaena_hassan@yahoocom

### COM CO P 5318 Da t Da a t Explora Explor t a ion and Analysis y Chapte Chapt r e 3

COMP 5318 Data Exploration and Analysis Chapter 3 What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping

### Web Search Engines: Solutions

Web Search Engines: Solutions Problem 1: A. How can the owner of a web site design a spider trap? Answer: He can set up his web server so that, whenever a client requests a URL in a particular directory

### Model Combination. 24 Novembre 2009

Model Combination 24 Novembre 2009 Datamining 1 2009-2010 Plan 1 Principles of model combination 2 Resampling methods Bagging Random Forests Boosting 3 Hybrid methods Stacking Generic algorithm for mulistrategy

### Feature Selection vs. Extraction

Feature Selection In many applications, we often encounter a very large number of potential features that can be used Which subset of features should be used for the best classification? Need for a small

### COMPARISON OF DISCRETIZATION METHODS FOR PREPROCESSING DATA FOR PYRAMIDAL GROWING NETWORK CLASSIFICATION METHOD

COMPARISON OF DISCRETIZATION METHODS FOR PREPROCESSING DATA FOR PYRAMIDAL GROWING NETWORK CLASSIFICATION METHOD Ilia Mitov 1, Krassimira Ivanova 1, Krassimir Markov 1, Vitalii Velychko 2, Peter Stanchev

### Data Mining Practical Machine Learning Tools and Techniques. Slides for Chapter 7 of Data Mining by I. H. Witten and E. Frank

Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 7 of Data Mining by I. H. Witten and E. Frank Engineering the input and output Attribute selection Scheme independent, scheme

### Concepts of Variables. Levels of Measurement. The Four Levels of Measurement. Nominal Scale. Greg C Elvers, Ph.D.

Concepts of Variables Greg C Elvers, Ph.D. 1 Levels of Measurement When we observe and record a variable, it has characteristics that influence the type of statistical analysis that we can perform on it

### Cluster Analysis. Isabel M. Rodrigues. Lisboa, 2014. Instituto Superior Técnico

Instituto Superior Técnico Lisboa, 2014 Introduction: Cluster analysis What is? Finding groups of objects such that the objects in a group will be similar (or related) to one another and different from

### Iris Sample Data Set. Basic Visualization Techniques: Charts, Graphs and Maps. Summary Statistics. Frequency and Mode

Iris Sample Data Set Basic Visualization Techniques: Charts, Graphs and Maps CS598 Information Visualization Spring 2010 Many of the exploratory data techniques are illustrated with the Iris Plant data

### Data Mining Methods: Applications for Institutional Research

Data Mining Methods: Applications for Institutional Research Nora Galambos, PhD Office of Institutional Research, Planning & Effectiveness Stony Brook University NEAIR Annual Conference Philadelphia 2014

### CLASSIFICATION AND CLUSTERING. Anveshi Charuvaka

CLASSIFICATION AND CLUSTERING Anveshi Charuvaka Learning from Data Classification Regression Clustering Anomaly Detection Contrast Set Mining Classification: Definition Given a collection of records (training

### A Gentle Introduction to Machine Learning

A Gentle Introduction to Machine Learning Second Lecture Part I Olov Andersson, AIICS Linköpings Universitet Outline of Machine Learning Lectures First we will talk about Supervised Learning Definition

### Start-up Companies Predictive Models Analysis. Boyan Yankov, Kaloyan Haralampiev, Petko Ruskov

Start-up Companies Predictive Models Analysis Boyan Yankov, Kaloyan Haralampiev, Petko Ruskov Abstract: A quantitative research is performed to derive a model for predicting the success of Bulgarian start-up

### Classification: Basic Concepts, Decision Trees, and Model Evaluation. General Approach for Building Classification Model

10 10 Classification: Basic Concepts, Decision Trees, and Model Evaluation Dr. Hui Xiong Rutgers University Introduction to Data Mining 1//009 1 General Approach for Building Classification Model Tid Attrib1

### Analysis Tools and Libraries for BigData

+ Analysis Tools and Libraries for BigData Lecture 02 Abhijit Bendale + Office Hours 2 n Terry Boult (Waiting to Confirm) n Abhijit Bendale (Tue 2:45 to 4:45 pm). Best if you email me in advance, but I

### Data Mining Practical Machine Learning Tools and Techniques

Ensemble learning Data Mining Practical Machine Learning Tools and Techniques Slides for Chapter 8 of Data Mining by I. H. Witten, E. Frank and M. A. Hall Combining multiple models Bagging The basic idea

### Near Optimal Solutions

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

### Clustering. Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016

Clustering Danilo Croce Web Mining & Retrieval a.a. 2015/201 16/03/2016 1 Supervised learning vs. unsupervised learning Supervised learning: discover patterns in the data that relate data attributes with

### Lecture 2: Types of Variables

2typesofvariables.pdf Michael Hallstone, Ph.D. hallston@hawaii.edu Lecture 2: Types of Variables Recap what we talked about last time Recall how we study social world using populations and samples. Recall

### Graphical and Tabular. Summarization of Data OPRE 6301

Graphical and Tabular Summarization of Data OPRE 6301 Introduction and Re-cap... Descriptive statistics involves arranging, summarizing, and presenting a set of data in such a way that useful information

### Data Mining on Streams

Data Mining on Streams Using Decision Trees CS 536: Machine Learning Instructor: Michael Littman TA: Yihua Wu Outline Introduction to data streams Overview of traditional DT learning ALG DT learning ALGs

### Infinite Algebra 1 supports the teaching of the Common Core State Standards listed below.

Infinite Algebra 1 Kuta Software LLC Common Core Alignment Software version 2.05 Last revised July 2015 Infinite Algebra 1 supports the teaching of the Common Core State Standards listed below. High School

### Lecture 20: Clustering

Lecture 20: Clustering Wrap-up of neural nets (from last lecture Introduction to unsupervised learning K-means clustering COMP-424, Lecture 20 - April 3, 2013 1 Unsupervised learning In supervised learning,

### IBM SPSS Decision Trees 21

IBM SPSS Decision Trees 21 Note: Before using this information and the product it supports, read the general information under Notices on p. 104. This edition applies to IBM SPSS Statistics 21 and to all

### Prentice Hall Mathematics: Course 1 2008 Correlated to: Arizona Academic Standards for Mathematics (Grades 6)

PO 1. Express fractions as ratios, comparing two whole numbers (e.g., ¾ is equivalent to 3:4 and 3 to 4). Strand 1: Number Sense and Operations Every student should understand and use all concepts and

### Statistical Foundations: Measurement Scales. Psychology 790 Lecture #1 8/24/2006

Statistical Foundations: Measurement Scales Psychology 790 Lecture #1 8/24/2006 Today s Lecture Measurement What is always assumed. What we can say when we assign numbers to phenomena. Implications for

### The Data Mining Process

Sequence for Determining Necessary Data. Wrong: Catalog everything you have, and decide what data is important. Right: Work backward from the solution, define the problem explicitly, and map out the data

### DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS

DATA MINING CLUSTER ANALYSIS: BASIC CONCEPTS 1 AND ALGORITHMS Chiara Renso KDD-LAB ISTI- CNR, Pisa, Italy WHAT IS CLUSTER ANALYSIS? Finding groups of objects such that the objects in a group will be similar

### Numerical Analysis Lecture Notes

Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number

### Elementary Statistics

Elementary Statistics Chapter 1 Dr. Ghamsary Page 1 Elementary Statistics M. Ghamsary, Ph.D. Chap 01 1 Elementary Statistics Chapter 1 Dr. Ghamsary Page 2 Statistics: Statistics is the science of collecting,