Classification/Decision Trees (II)


 Patrick Atkins
 2 years ago
 Views:
Transcription
1 Classification/Decision Trees (II) Department of Statistics The Pennsylvania State University
2 Right Sized Trees Let the expected misclassification rate of a tree T be R (T ). Recall the resubstitution estimate for R (T ) is R(T ) = r(t)p(t) = R(t). t T t T R(T ) is biased downward. R(t) R(t L ) + R(t R ).
3 Digit recognition example No. Terminal Nodes R(T ) R ts (T ) The estimate R(T ) becomes increasingly less accurate as the trees grow larger. The estimate R ts decreases first when the tree becomes larger, hits minimum at the tree with 10 terminal nodes, and begins to increase when the tree further grows.
4 Preliminaries for Pruning Grow a very large tree T max. 1. Until all terminal nodes are pure (contain only one class) or contain only identical measurement vectors. 2. When the number of data in each terminal node is no greater than a certain threshold, say 5, or even As long as the tree is sufficiently large, the size of the initial tree is not critical.
5 1. Descendant: a node t is a descendant of node t if there is a connected path down the tree leading from t to t. 2. Ancestor: t is an ancestor of t if t is its descendant. 3. A branch T t of T with root node t T consists of the node t and all descendants of t in T. 4. Pruning a branch T t from a tree T consists of deleting from T all descendants of t, that is, cutting off all of T t except its root node. The tree pruned this way will be denoted by T T t. 5. If T is gotten from T by successively pruning off branches, then T is called a pruned subtree of T and denoted by T T.
6 Subtrees Even for a moderate sized T max, there is an enormously large number of subtrees and an even larger number ways to prune the initial tree to them. A selective pruning procedure is needed. The pruning is optimal in a certain sense. The search for different ways of pruning should be of manageable computational load.
7 Minimal CostComplexity Pruning Definition for the costcomplexity measure: For any subtree T T max, define its complexity as T, the number of terminal nodes in T. Let α 0 be a real number called the complexity parameter and define the costcomplexity measure R α (T ) as R α (T ) = R(T ) + α T.
8 For each value of α, find the subtree T (α) that minimizes R α (T ), i.e., R α (T (α)) = min R α (T ). T T max If α is small, the penalty for having a large number of terminal nodes is small and T (α) tends to be large. For α sufficiently large, the minimizing subtree T (α) will consist of the root node only.
9 Since there are at most a finite number of subtrees of T max, R α (T (α)) yields different values for only finitely many α s. T (α) continues to be the minimizing tree when α increases until a jump point is reached. Two questions: Is there a unique subtree T Tmax which minimizes R α (T )? In the minimizing sequence of trees T 1, T 2,..., is each subtree obtained by pruning upward from the previous subtree, i.e., does the nesting T 1 T 2 {t 1 } hold?
10 Definition: The smallest minimizing subtree T (α) for complexity parameter α is defined by the conditions: 1. R α (T (α)) = min T Tmax R α (T ) 2. If R α (T ) = R α (T (α)), then T (α) T. If subtree T (α) exists, it must be unique. It can be proved that for every value of α, there exists a smallest minimizing subtree.
11 The starting point for the pruning is not T max, but rather T 1 = T (0), which is the smallest subtree of T max satisfying R(T 1 ) = R(T max ). Let t L and t R be any two terminal nodes in T max descended from the same parent node t. If R(t) = R(t L ) + R(t R ), prune off t L and t R. Continue the process until no more pruning is possible. The resulting tree is T 1.
12 For T t any branch of T 1, define R(T t ) by R(T t ) = R(t ), t T t where T t is the set of terminal nodes of T t. For t any nonterminal node of T 1, R(t) > R(T t ).
13 WeakestLink Cutting For any node t T 1, set R α ({t}) = R(t) + α. For any branch T t, define R α (T t ) = R(T t ) + α T t. When α = 0, R 0 (T t ) < R 0 ({t}). The inequality holds for sufficiently small α. But at some critical value of α, the two costcomplexities become equal. For α exceeding this threshold, the inequality is reversed. Solve the inequality R α (T t ) < R α ({t}) and get α < R(t) R(T t) T t 1. The right hand side is always positive.
14 Define a function g 1 (t), t T 1 by { R(t) R(Tt), t / T g 1 (t) = T 1 t 1 +, t T 1 Define the weakest link t 1 in T 1 as the node such that and put α 2 = g 1 ( t 1 ). g 1 ( t 1 ) = min t T 1 g 1 (t).
15 When α increases, t 1 is the first node that becomes more preferable than the branch T t 1 descended from it. α 2 is the first value after α 1 = 0 that yields a strict subtree of T 1 with a smaller costcomplexity at this complexity parameter. That is, for all α 1 α < α 2, the tree with smallest costcomplexity is T 1. Let T 2 = T 1 T t 1.
16 Repeat the previous steps. Use T 2 instead of T 1, find the weakest link in T 2 and prune off at the weakest link node. g 2 (t) = { R(t) R(T2t ) T, t T 2t 1 2, t / T 2 +, t T 2 g 2 ( t 2 ) = min t T 2 g 2 (t) α 3 = g 2 ( t 2 ) T 3 = T 2 T t 2 If at any stage, there are multiple weakest links, for instance, if g k ( t k ) = g k ( t k ), then define T k+1 = T k T t k T t k. Two branches are either nested or share no node.
17 A decreasing sequence of nested subtrees are obtained: T 1 T 2 T 3 {t 1 }. Theorem: The {α k } are an increasing sequence, that is, α k < α k+1, k 1, where α 1 = 0. For k 1, α k α < α k+1, T (α) = T (α k ) = T k.
18 At the initial steps of pruning, the algorithm tends to cut off large subbranches with many leaf nodes. With the tree becoming smaller, it tends to cut off fewer. Digit recognition example: Tree T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 T 11 T 12 T 13 T k
19 Best Pruned Subtree Two approaches to choose the best pruned subtree: Use a test sample set. Crossvalidation Use a test set to compute the classification error rate of each minimum costcomplexity subtree. Choose the subtree with the minimum test error rate. Cross validation: tree structures are not stable. When the training data set changes slightly, there may be large structural change in the tree. It is difficult to correspond a subtree trained from the entire data set to a subtree trained from a majority part of it. Focus on choosing the right complexity parameter α.
20 Pruning by CrossValidation Consider V fold crossvalidation. The original learning sample L is divided by random selection into V subsets, L v, v = 1,..., V. Let the training sample set in each fold be L (v) = L L v. The tree grown on the original set is T max. V accessory trees T (v) max are grown on L (v).
21 For each value of the complexity parameter α, let T (α), T (v) (α), v = 1,..., V, be the corresponding minimal costcomplexity subtrees of T max, T max. (v) For each maximum tree, we obtain a sequence of jump points of α: α 1 < α 2 < α 3 < α k <. To find the corresponding minimal costcomplexity subtree at α, find α k from the list such that α k α < α k+1. Then the subtree corresponding to α k is the subtree for α.
22 The crossvalidation error rate of T (α) is computed by R CV (T (α)) = 1 V V v=1 N (v) miss N (v), where N (v) is the number of samples in the test set L v in fold v; and N (v) miss is the number of misclassified samples in L v using T (v) (α), a pruned tree of T (v) max trained from L (v). Although α is continuous, there are only finite minimum costcomplexity trees grown on L.
23 Let T k = T (α k ). To compute the crossvalidation error rate of T k, let α k = α k α k+1. Let R CV (T k ) = R CV (T (α k )). For the root node tree {t 1 }, R CV ({t 1 }) is set to the resubstitution cost R({t 1 }). Choose the subtree T k with minimum crossvalidation error rate R CV (T k ).
24 Computation Involved 1. Grow V + 1 maximum trees. 2. For each of the V + 1 trees, find the sequence of subtrees with minimum costcomplexity. 3. Suppose the maximum tree grown on the original data set T max has K subtrees. 4. For each of the (K 1) α k, compute the misclassification rate of each of the V test sample set, average the error rates and set the mean to the crossvalidation error rate. 5. Find the subtree of T max with minimum R CV (T k ).
Analysis of Algorithms I: Binary Search Trees
Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary
More information6 Classification and Regression Trees, 7 Bagging, and Boosting
hs24 v.2004/01/03 Prn:23/02/2005; 14:41 F:hs24011.tex; VTEX/ES p. 1 1 Handbook of Statistics, Vol. 24 ISSN: 01697161 2005 Elsevier B.V. All rights reserved. DOI 10.1016/S01697161(04)240111 1 6 Classification
More information!"!!"#$$%&'()*+$(,%!"#$%$&'()*""%(+,'*&./#$&'(&(0*".$#$1"(2&."3$'45"
!"!!"#$$%&'()*+$(,%!"#$%$&'()*""%(+,'*&./#$&'(&(0*".$#$1"(2&."3$'45"!"#"$%&#'()*+',$$.&#',/"0%.12'32./4'5,5'6/%&)$).2&'7./&)8'5,5'9/2%.%3%&8':")08';:
More informationBalanced Binary Search Tree
AVL Trees / Slide 1 Balanced Binary Search Tree Worst case height of binary search tree: N1 Insertion, deletion can be O(N) in the worst case We want a tree with small height Height of a binary tree with
More informationroot node level: internal node edge leaf node CS@VT Data Structures & Algorithms 20002009 McQuain
inary Trees 1 A binary tree is either empty, or it consists of a node called the root together with two binary trees called the left subtree and the right subtree of the root, which are disjoint from each
More informationData Mining Algorithms for Classification
Data Mining Algorithms for Classification BSc Thesis Artificial Intelligence Author: Patrick Ozer Radboud University Nijmegen January 2008 Supervisor: Dr. I.G. SprinkhuizenKuyper Radboud University Nijmegen
More informationLecture 7: Approximation via Randomized Rounding
Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining
More informationAn Overview and Evaluation of Decision Tree Methodology
An Overview and Evaluation of Decision Tree Methodology ASA Quality and Productivity Conference Terri Moore Motorola Austin, TX terri.moore@motorola.com Carole Jesse Cargill, Inc. Wayzata, MN carole_jesse@cargill.com
More informationClassification and regression trees
Classification and regression trees December 9 Introduction We ve seen that local methods and splines both operate by partitioning the sample space of the regression variable(s), and then fitting separate/piecewise
More informationLecture 10: Regression Trees
Lecture 10: Regression Trees 36350: Data Mining October 11, 2006 Reading: Textbook, sections 5.2 and 10.5. The next three lectures are going to be about a particular kind of nonlinear predictive model,
More informationFine Particulate Matter Concentration Level Prediction by using Treebased Ensemble Classification Algorithms
Fine Particulate Matter Concentration Level Prediction by using Treebased Ensemble Classification Algorithms Yin Zhao School of Mathematical Sciences Universiti Sains Malaysia (USM) Penang, Malaysia Yahya
More informationFrom Last Time: Remove (Delete) Operation
CSE 32 Lecture : More on Search Trees Today s Topics: Lazy Operations Run Time Analysis of Binary Search Tree Operations Balanced Search Trees AVL Trees and Rotations Covered in Chapter of the text From
More informationData Mining on Streams
Data Mining on Streams Using Decision Trees CS 536: Machine Learning Instructor: Michael Littman TA: Yihua Wu Outline Introduction to data streams Overview of traditional DT learning ALG DT learning ALGs
More informationGerry Hobbs, Department of Statistics, West Virginia University
Decision Trees as a Predictive Modeling Method Gerry Hobbs, Department of Statistics, West Virginia University Abstract Predictive modeling has become an important area of interest in tasks such as credit
More informationA binary search tree is a binary tree with a special property called the BSTproperty, which is given as follows:
Chapter 12: Binary Search Trees A binary search tree is a binary tree with a special property called the BSTproperty, which is given as follows: For all nodes x and y, if y belongs to the left subtree
More informationSequential lmove Games. Using Backward Induction (Rollback) to Find Equilibrium
Sequential lmove Games Using Backward Induction (Rollback) to Find Equilibrium Sequential Move Class Game: Century Mark Played by fixed pairs of players taking turns. At each turn, each player chooses
More informationGRAPH THEORY LECTURE 4: TREES
GRAPH THEORY LECTURE 4: TREES Abstract. 3.1 presents some standard characterizations and properties of trees. 3.2 presents several different types of trees. 3.7 develops a counting method based on a bijection
More informationFull and Complete Binary Trees
Full and Complete Binary Trees Binary Tree Theorems 1 Here are two important types of binary trees. Note that the definitions, while similar, are logically independent. Definition: a binary tree T is full
More informationINDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS
INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem
More informationApproximation Algorithms
Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NPCompleteness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms
More informationThe Goldberg Rao Algorithm for the Maximum Flow Problem
The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }
More information2 Decision tree + Crossvalidation with R (package rpart)
1 Subject Using crossvalidation for the performance evaluation of decision trees with R, KNIME and RAPIDMINER. This paper takes one of our old study on the implementation of crossvalidation for assessing
More informationAlgorithms Chapter 12 Binary Search Trees
Algorithms Chapter 1 Binary Search Trees Outline Assistant Professor: Ching Chi Lin 林 清 池 助 理 教 授 chingchi.lin@gmail.com Department of Computer Science and Engineering National Taiwan Ocean University
More informationLecture 13: Validation
Lecture 3: Validation g Motivation g The Holdout g Resampling techniques g Threeway data splits Motivation g Validation techniques are motivated by two fundamental problems in pattern recognition: model
More informationStatistical Data Mining. Practical Assignment 3 Discriminant Analysis and Decision Trees
Statistical Data Mining Practical Assignment 3 Discriminant Analysis and Decision Trees In this practical we discuss linear and quadratic discriminant analysis and treebased classification techniques.
More informationPractical Guide to the Simplex Method of Linear Programming
Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear
More informationOverview. Evaluation Connectionist and Statistical Language Processing. Test and Validation Set. Training and Test Set
Overview Evaluation Connectionist and Statistical Language Processing Frank Keller keller@coli.unisb.de Computerlinguistik Universität des Saarlandes training set, validation set, test set holdout, stratification
More informationBinary Search Trees. Data in each node. Larger than the data in its left child Smaller than the data in its right child
Binary Search Trees Data in each node Larger than the data in its left child Smaller than the data in its right child FIGURE 116 Arbitrary binary tree FIGURE 117 Binary search tree Data Structures Using
More informationBinary Heaps * * * * * * * / / \ / \ / \ / \ / \ * * * * * * * * * * * / / \ / \ / / \ / \ * * * * * * * * * *
Binary Heaps A binary heap is another data structure. It implements a priority queue. Priority Queue has the following operations: isempty add (with priority) remove (highest priority) peek (at highest
More informationOperations: search;; min;; max;; predecessor;; successor. Time O(h) with h height of the tree (more on later).
Binary search tree Operations: search;; min;; max;; predecessor;; successor. Time O(h) with h height of the tree (more on later). Data strutcure fields usually include for a given node x, the following
More informationLecture 4: BK inequality 27th August and 6th September, 2007
CSL866: Percolation and Random Graphs IIT Delhi Amitabha Bagchi Scribe: Arindam Pal Lecture 4: BK inequality 27th August and 6th September, 2007 4. Preliminaries The FKG inequality allows us to lower bound
More informationMultiple Linear Regression in Data Mining
Multiple Linear Regression in Data Mining Contents 2.1. A Review of Multiple Linear Regression 2.2. Illustration of the Regression Process 2.3. Subset Selection in Linear Regression 1 2 Chap. 2 Multiple
More informationData Mining Classification: Decision Trees
Data Mining Classification: Decision Trees Classification Decision Trees: what they are and how they work Hunt s (TDIDT) algorithm How to select the best split How to handle Inconsistent data Continuous
More informationPrevious Lectures. BTrees. External storage. Two types of memory. Btrees. Main principles
BTrees Algorithms and data structures for external memory as opposed to the main memory BTrees Previous Lectures Height balanced binary search trees: AVL trees, redblack trees. Multiway search trees:
More informationA single minimal complement for the c.e. degrees
A single minimal complement for the c.e. degrees Andrew Lewis Leeds University, April 2002 Abstract We show that there exists a single minimal (Turing) degree b < 0 s.t. for all c.e. degrees 0 < a < 0,
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 7 Binary heap, binomial heap, and Fibonacci heap 1 Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 The slides were
More informationKnowledge Discovery and Data Mining. Bootstrap review. Bagging Important Concepts. Notes. Lecture 19  Bagging. Tom Kelsey. Notes
Knowledge Discovery and Data Mining Lecture 19  Bagging Tom Kelsey School of Computer Science University of St Andrews http://tom.host.cs.standrews.ac.uk twk@standrews.ac.uk Tom Kelsey ID505919B &
More informationA NonLinear Schema Theorem for Genetic Algorithms
A NonLinear Schema Theorem for Genetic Algorithms William A Greene Computer Science Department University of New Orleans New Orleans, LA 70148 bill@csunoedu 5042806755 Abstract We generalize Holland
More informationIE 680 Special Topics in Production Systems: Networks, Routing and Logistics*
IE 680 Special Topics in Production Systems: Networks, Routing and Logistics* Rakesh Nagi Department of Industrial Engineering University at Buffalo (SUNY) *Lecture notes from Network Flows by Ahuja, Magnanti
More informationTransforming XML trees for efficient classification and clustering
Transforming XML trees for efficient classification and clustering Laurent Candillier 1,2, Isabelle Tellier 1, Fabien Torre 1 1 GRAppA  Charles de Gaulle University  Lille 3 http://www.grappa.univlille3.fr
More informationAlgorithms and Data Structures
Algorithms and Data Structures Part 2: Data Structures PD Dr. rer. nat. habil. RalfPeter Mundani Computation in Engineering (CiE) Summer Term 2016 Overview general linked lists stacks queues trees 2 2
More informationTHE SEARCH FOR NATURAL DEFINABILITY IN THE TURING DEGREES
THE SEARCH FOR NATURAL DEFINABILITY IN THE TURING DEGREES ANDREW E.M. LEWIS 1. Introduction This will be a course on the Turing degrees. We shall assume very little background knowledge: familiarity with
More informationRotation Operation for Binary Search Trees Idea:
Rotation Operation for Binary Search Trees Idea: Change a few pointers at a particular place in the tree so that one subtree becomes less deep in exchange for another one becoming deeper. A sequence of
More informationLecture 10 UnionFind The unionnd data structure is motivated by Kruskal's minimum spanning tree algorithm (Algorithm 2.6), in which we needed two operations on disjoint sets of vertices: determine whether
More informationHomework Assignment 7
Homework Assignment 7 36350, Data Mining Solutions 1. Base rates (10 points) (a) What fraction of the emails are actually spam? Answer: 39%. > sum(spam$spam=="spam") [1] 1813 > 1813/nrow(spam) [1] 0.3940448
More information1 Representation of Games. Kerschbamer: Commitment and Information in Games
1 epresentation of Games Kerschbamer: Commitment and Information in Games GameTheoretic Description of Interactive Decision Situations This lecture deals with the process of translating an informal description
More informationarxiv:1112.0829v1 [math.pr] 5 Dec 2011
How Not to Win a Million Dollars: A Counterexample to a Conjecture of L. Breiman Thomas P. Hayes arxiv:1112.0829v1 [math.pr] 5 Dec 2011 Abstract Consider a gambling game in which we are allowed to repeatedly
More informationCI6227: Data Mining. Lesson 11b: Ensemble Learning. Data Analytics Department, Institute for Infocomm Research, A*STAR, Singapore.
CI6227: Data Mining Lesson 11b: Ensemble Learning Sinno Jialin PAN Data Analytics Department, Institute for Infocomm Research, A*STAR, Singapore Acknowledgements: slides are adapted from the lecture notes
More informationGenerating Gaussian Mixture Models by Model Selection For Speech Recognition
Generating Gaussian Mixture Models by Model Selection For Speech Recognition Kai Yu F06 10701 Final Project Report kaiy@andrew.cmu.edu Abstract While all modern speech recognition systems use Gaussian
More informationHow Asymmetry Helps Load Balancing
How Asymmetry Helps oad Balancing Berthold Vöcking nternational Computer Science nstitute Berkeley, CA 947041198 voecking@icsiberkeleyedu Abstract This paper deals with balls and bins processes related
More informationDecision Tree Learning on Very Large Data Sets
Decision Tree Learning on Very Large Data Sets Lawrence O. Hall Nitesh Chawla and Kevin W. Bowyer Department of Computer Science and Engineering ENB 8 University of South Florida 4202 E. Fowler Ave. Tampa
More informationSMALL SKEW FIELDS CÉDRIC MILLIET
SMALL SKEW FIELDS CÉDRIC MILLIET Abstract A division ring of positive characteristic with countably many pure types is a field Wedderburn showed in 1905 that finite fields are commutative As for infinite
More informationCourse: Model, Learning, and Inference: Lecture 5
Course: Model, Learning, and Inference: Lecture 5 Alan Yuille Department of Statistics, UCLA Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract Probability distributions on structured representation.
More informationLecture 1: Course overview, circuits, and formulas
Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik
More informationPerformance Evaluation and Optimization of MathSimilarity Search
Performance Evaluation and Optimization of MathSimilarity Search Qun Zhang and Abdou Youssef Department of Computer Science The George Washington University Washington DC, 20052, USA Abstract Similarity
More informationAdaptive Online Gradient Descent
Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650
More informationProfessor Anita Wasilewska. Classification Lecture Notes
Professor Anita Wasilewska Classification Lecture Notes Classification (Data Mining Book Chapters 5 and 7) PART ONE: Supervised learning and Classification Data format: training and test data Concept,
More informationOrdered Lists and Binary Trees
Data Structures and Algorithms Ordered Lists and Binary Trees Chris Brooks Department of Computer Science University of San Francisco Department of Computer Science University of San Francisco p.1/62 60:
More informationB+ Tree Properties B+ Tree Searching B+ Tree Insertion B+ Tree Deletion Static Hashing Extendable Hashing Questions in pass papers
B+ Tree and Hashing B+ Tree Properties B+ Tree Searching B+ Tree Insertion B+ Tree Deletion Static Hashing Extendable Hashing Questions in pass papers B+ Tree Properties Balanced Tree Same height for paths
More informationBinary Heaps. CSE 373 Data Structures
Binary Heaps CSE Data Structures Readings Chapter Section. Binary Heaps BST implementation of a Priority Queue Worst case (degenerate tree) FindMin, DeleteMin and Insert (k) are all O(n) Best case (completely
More informationClassifying Large Data Sets Using SVMs with Hierarchical Clusters. Presented by :Limou Wang
Classifying Large Data Sets Using SVMs with Hierarchical Clusters Presented by :Limou Wang Overview SVM Overview Motivation Hierarchical microclustering algorithm ClusteringBased SVM (CBSVM) Experimental
More informationLecture 4 Online and streaming algorithms for clustering
CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 Online kclustering To the extent that clustering takes place in the brain, it happens in an online
More information6 March 2007 1. Array Implementation of Binary Trees
Heaps CSE 0 Winter 00 March 00 1 Array Implementation of Binary Trees Each node v is stored at index i defined as follows: If v is the root, i = 1 The left child of v is in position i The right child of
More informationPhysical Data Organization
Physical Data Organization Database design using logical model of the database  appropriate level for users to focus on  user independence from implementation details Performance  other major factor
More information8.1 Min Degree Spanning Tree
CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree
More informationScheduling. Open shop, job shop, flow shop scheduling. Related problems. Open shop, job shop, flow shop scheduling
Scheduling Basic scheduling problems: open shop, job shop, flow job The disjunctive graph representation Algorithms for solving the job shop problem Computational complexity of the job shop problem Open
More informationBinary Search Trees. A Generic Tree. Binary Trees. Nodes in a binary search tree ( BST) are of the form. P parent. Key. Satellite data L R
Binary Search Trees A Generic Tree Nodes in a binary search tree ( BST) are of the form P parent Key A Satellite data L R B C D E F G H I J The BST has a root node which is the only node whose parent
More informationTrimming a Tree and the TwoSided Skorohod Reflection
ALEA, Lat. Am. J. Probab. Math. Stat. 12 (2), 811 834 (2015) Trimming a Tree and the TwoSided Skorohod Reflection Emmanuel Schertzer UPMC Univ. Paris 06, Laboratoire de Probabilités et Modèles Aléatoires,
More information1. (First passage/hitting times/gambler s ruin problem:) Suppose that X has a discrete state space and let i be a fixed state. Let
Copyright c 2009 by Karl Sigman 1 Stopping Times 1.1 Stopping Times: Definition Given a stochastic process X = {X n : n 0}, a random time τ is a discrete random variable on the same probability space as
More informationCART: Classification and Regression Trees
Chapter 10 CART: Classification and Regression Trees Dan Steinberg Contents 10.1 Antecedents... 180 10.2 Overview... 181 10.3 A Running Example... 181 10.4 The Algorithm Briefly Stated... 183 10.5 Splitting
More informationData Management in Hierarchical Bus Networks
Data Management in Hierarchical Bus Networks F. Meyer auf der Heide Department of Mathematics and Computer Science and Heinz Nixdorf Institute, Paderborn University, Germany fmadh@upb.de H. Räcke Department
More informationAnalysis of Algorithms I: Optimal Binary Search Trees
Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search
More informationSimple Network Performance Tomography
Simple Network Performance Tomography Nick Duffield AT&T Labs Research 8 Park Avenue Florham Park, NJ 7932, USA duffield@research.att.com ABSTRACT In network performance tomography, characteristics of
More information6.3 Conditional Probability and Independence
222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted
More informationL13: crossvalidation
Resampling methods Cross validation Bootstrap L13: crossvalidation Bias and variance estimation with the Bootstrap Threeway data partitioning CSCE 666 Pattern Analysis Ricardo GutierrezOsuna CSE@TAMU
More informationNetwork Tomography of Binary Network Performance Characteristics
Network Tomography of Binary Network Performance Characteristics Nick Duffield AT&T Labs Research 80 Park Avenue, Florham Park, NJ 07932, USA Email: duffield@research.att.com Abstract In network performance
More informationData Mining  Evaluation of Classifiers
Data Mining  Evaluation of Classifiers Lecturer: JERZY STEFANOWSKI Institute of Computing Sciences Poznan University of Technology Poznan, Poland Lecture 4 SE Master Course 2008/2009 revised for 2010
More informationComplete Redundancy Detection in Firewalls
Complete Redundancy Detection in Firewalls Alex X. Liu and Mohamed G. Gouda Department of Computer Sciences, The University of Texas at Austin, Austin, Texas 787120233, USA {alex, gouda}@cs.utexas.edu
More informationBTrees. Algorithms and data structures for external memory as opposed to the main memory BTrees. B trees
BTrees Algorithms and data structures for external memory as opposed to the main memory BTrees Previous Lectures Height balanced binary search trees: AVL trees, redblack trees. Multiway search trees:
More informationData Mining Techniques Chapter 6: Decision Trees
Data Mining Techniques Chapter 6: Decision Trees What is a classification decision tree?.......................................... 2 Visualizing decision trees...................................................
More informationClassification: Basic Concepts, Decision Trees, and Model Evaluation. General Approach for Building Classification Model
10 10 Classification: Basic Concepts, Decision Trees, and Model Evaluation Dr. Hui Xiong Rutgers University Introduction to Data Mining 1//009 1 General Approach for Building Classification Model Tid Attrib1
More informationOutline BST Operations Worst case Average case Balancing AVL Redblack Btrees. Binary Search Trees. Lecturer: Georgy Gimel farb
Binary Search Trees Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 27 1 Properties of Binary Search Trees 2 Basic BST operations The worstcase time complexity of BST operations
More informationA Knowledge Model Sharing Based Approach to PrivacyPreserving Data Mining
433 467 A Knowledge Model Sharing Based Approach to PrivacyPreserving Data Mining Hongwei Tian, Weining Zhang, Shouhuai Xu and Patrick Sharkey Department of Computer Science, University of Texas at San
More informationMax Flow, Min Cut, and Matchings (Solution)
Max Flow, Min Cut, and Matchings (Solution) 1. The figure below shows a flow network on which an st flow is shown. The capacity of each edge appears as a label next to the edge, and the numbers in boxes
More informationHeap. Binary Search Tree. Heaps VS BSTs. < el el. Difference between a heap and a BST:
Heaps VS BSTs Difference between a heap and a BST: Heap el Binary Search Tree el el el < el el Perfectly balanced at all times Immediate access to maximal element Easy to code Does not provide efficient
More informationLecture 6 Online and streaming algorithms for clustering
CSE 291: Unsupervised learning Spring 2008 Lecture 6 Online and streaming algorithms for clustering 6.1 Online kclustering To the extent that clustering takes place in the brain, it happens in an online
More informationConverting a Number from Decimal to Binary
Converting a Number from Decimal to Binary Convert nonnegative integer in decimal format (base 10) into equivalent binary number (base 2) Rightmost bit of x Remainder of x after division by two Recursive
More information6.231 Dynamic Programming and Stochastic Control Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.231 Dynamic Programming and Stochastic Control Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 6.231
More informationDiscuss the size of the instance for the minimum spanning tree problem.
3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can
More informationUnit 4 DECISION ANALYSIS. Lesson 37. Decision Theory and Decision Trees. Learning objectives:
Unit 4 DECISION ANALYSIS Lesson 37 Learning objectives: To learn how to use decision trees. To structure complex decision making problems. To analyze the above problems. To find out limitations & advantages
More informationData Structure with C
Subject: Data Structure with C Topic : Tree Tree A tree is a set of nodes that either:is empty or has a designated node, called the root, from which hierarchically descend zero or more subtrees, which
More informationDiscernibility Thresholds and Approximate Dependency in Analysis of Decision Tables
Discernibility Thresholds and Approximate Dependency in Analysis of Decision Tables YuRu Syau¹, EnBing Lin²*, Lixing Jia³ ¹Department of Information Management, National Formosa University, Yunlin, 63201,
More informationOpen and Closed Sets
Open and Closed Sets Definition: A subset S of a metric space (X, d) is open if it contains an open ball about each of its points i.e., if x S : ɛ > 0 : B(x, ɛ) S. (1) Theorem: (O1) and X are open sets.
More informationA Constraint Programming based Column Generation Approach to Nurse Rostering Problems
Abstract A Constraint Programming based Column Generation Approach to Nurse Rostering Problems Fang He and Rong Qu The Automated Scheduling, Optimisation and Planning (ASAP) Group School of Computer Science,
More informationClassification and Prediction
Classification and Prediction Slides for Data Mining: Concepts and Techniques Chapter 7 Jiawei Han and Micheline Kamber Intelligent Database Systems Research Lab School of Computing Science Simon Fraser
More informationOnline Convex Optimization
E0 370 Statistical Learning heory Lecture 19 Oct 22, 2013 Online Convex Optimization Lecturer: Shivani Agarwal Scribe: Aadirupa 1 Introduction In this lecture we shall look at a fairly general setting
More informationClassification and Regression Trees (CART) Theory and Applications
Classification and Regression Trees (CART) Theory and Applications A Master Thesis Presented by Roman Timofeev (188778) to Prof. Dr. Wolfgang Härdle CASE  Center of Applied Statistics and Economics Humboldt
More informationData Mining Methods: Applications for Institutional Research
Data Mining Methods: Applications for Institutional Research Nora Galambos, PhD Office of Institutional Research, Planning & Effectiveness Stony Brook University NEAIR Annual Conference Philadelphia 2014
More informationXML Data Integration
XML Data Integration Lucja Kot Cornell University 11 November 2010 Lucja Kot (Cornell University) XML Data Integration 11 November 2010 1 / 42 Introduction Data Integration and Query Answering A data integration
More informationData Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation. Lecture Notes for Chapter 4. Introduction to Data Mining
Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data
More information