MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos


 Octavia Warner
 2 years ago
 Views:
Transcription
1 Machine Learning for Computer Vision 1 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Department of Applied Mathematics Ecole Centrale Paris Galen Group INRIASaclay
2 Administrative details 2 Make sure you have all received my from last week
3 Lecture outline 3 Recap & problems of linear regression Logistic Regression Introduction to Multiple Instance Learning
4 4 Learning problem Recover inputoutput mapping Output: y Input: x Method: f Parameters: w Aspects of the learning problem Identify methods that fit the problem setting Determine parameters that properly classify the training set Measure and control the complexity of these functions
5 Linear Classifiers Find linear expression (hyperplane) to separate positive and negative examples 5 Feature coordinate j x x i i positive : negative : x x i i w + b w + b 0 < 0 Each data point has a class label: +1 ( ) y t = 1 ( ) Feature coordinate i
6 Learning problem formulation 6 Given: Training set of featurelabel pairs Wanted: simple that works well for simple: penalizing function complexity, to guarantee generalization works well: quantified by loss criterion
7 Linear regression 7 Linear Loss function: quantify appropriateness of for Introduce vector notation
8 Inappropriateness of quadratic loss We chose the quadratic cost function for convenience Single, global minimum & closed form expression 8 But does it indicate classification performance? Computed Decision Boundary Linear Fit Desired decision boundary
9 Inappropriateness of quadratic loss Consider transformation: 9 Quadratic loss is not robust to outliers and penalizes outputs that are `too good
10 Lecture outline 10 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning
11 Probabilistic interpretation of least squares 11 Outputs: continuous random variables y = hx, wi +, N(0, ) linear function of inputs + zero mean Gaussian r.v. Optimal w: maximizes likelihood of observed outputs But the labels are discrete!
12 A probabilistic criterion for training a classifier 12 Training set: y: discrete observations: model as samples from Bernoulli distribution Find w that maximizes the likelihood of labels in the training set
13 Sigmoidal function & logistic regression 13 sigmoidal Training criterion from previous slide: How do we optimize it with respect to w? What does it mean?
14 Lecture outline 14 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning
15 Maximization by gradient ascent 15 Gradient ascent:
16 Maximization with NewtonRaphson 16 Newton method for finding roots of 1D function : Newton method for finding maxima of 1D function : condition NewtonRaphson method for finding maxima of ND function :
17 NewtonRaphson for Logistic Regression 17 Gradient: Hessian: In statistics: Iteratively Reweighted Least Squares (IRLS)
18 Lecture outline 18 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Support Vector Machines
19 A compact expression for the loss 19 Using, :
20 Log loss: 20
21 Log loss vs. quadratic loss 21
22 Logistic vs Linear Regression 22 Logistic regression is more robust Linear Regression Logistic Regression
23 Lecture outline 23 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning
24 Boundary detection problem Object/Surface Boundaries 24
25 25 Signallevel challenges Poor contrast Shadows Texture
26 Machine Learning for Computer Vision Lecture 2 Fundamental challenges: can humans do it? 26
27 Learningbased approaches 27 Boundary or nonboundary? Use humanannotated segmentations Use any visual cue as input to the decision function. Use decision trees/logisitic regression/boosting/ and learn to combine the individual inputs. S. Konishi, A.Yuille, J. Coughlan, S.C. Zhu, Statistical Edge Detection: Learning and Evaluating Edge Cues, IEEE PAMI, 2003 D. Martin, C. Fowlkes, J. Malik. "Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues", IEEE PAMI, 2004
28 Evaluation protocol 28 Threshold detector s output at certain level Match outputs to human groundtruth Count true/false positives (t/f)positives, misses (m) From precision, p and recall, r Quantify performance in terms of Fmeasure
29 29 Contours can be defined by any of a number of cues (P. Cavanagh) Slide credit: J. Malik, M. Maire
30 Cuelocalization Gray level photographs 30 Objects from motion Objects from luminance Objects from disparity Line drawings Objects from texture Slide credit: J. Malik, M. Maire GrillSpector et al., Neuron 1998
31 31 Local boundary cues Separate features per candidate orientation In specific: r (x,y) θ 1976 CIE L*a*b* colorspace Brightness Gradient BG(x,y,r,θ) Difference of L* distributions Color Gradient CG(x,y,r,θ) Difference of a*b* distributions Texture Gradient TG(x,y,r,θ) Difference of distributions of V1like filter responses Slide credit: J. Malik, M. Maire
32 Boundary cues (horizontal) 32 Brightness Color (a, b channels) Texture
33 Boundary π/4 33 Brightness Color (a, b channels) Texture
34 Brightness π/2 34 Brightness Color (a, b channels) Texture
35 Brightness 3π/4 35 Brightness Color (a, b channels) Texture
36 Cue combinations with logistic regression 36 Slide credit: J. Malik, M. Maire
37 Exploiting global constraints: image Segmentation as Graph Partitioning Build a weighted graph G=(V,E) from image V: image pixels 37 E: connections between pairs of nearby pixels Partition graph so that similarity within group is large and similarity between groups is small  Normalized Cuts [Shi & Malik 97] Slide credit: J. Malik, M. Maire
38 38 Wij small when intervening contour strong, strong when weak.. Slide credit: J. Malik, M. Maire
39 Normalized Cuts as a SpringMass system 39 Each pixel is a point mass; each connection is a spring: ( D W ) y = λdy Fundamental modes are generalized eigenvectors of (D  W) x = λdx Slide credit: J. Malik, M. Maire
40 Contour information from eigenvectors 40 Slide credit: J. Malik, M. Maire
41 Classifier 41 Logistic regression
42 Progress during the last 40 years 42 Humans Berkeley gpb, 08 Berkeley PB, 04 Canny+ Hysteresis ( 85) Prewitt, good feature= 5 years of work
43 Lecture outline 43 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning
44 Learning boundary detection 44 Problem I: inconsistent orientation information Problem II: inconsistent location information
45 Sneaking into the fun room 45 mom s keychain grandma s keychain dad s keychain We know that dad cannot enter the fun room, either Which key should we try? Slide Credit: B. Babenko/T. Dietterich
46 Multiple Instance Learning 46 Typical Learning Multiple Instance Learning Positive bag: at least one instance should be positive Negative bag: no instance should be positive Slide Credit: K. Grauman
47 Noisyor classifier combination N classifier responses, {p 1,...,p N } p i : probability of positive label (classifier i) 1 p i : probability of negative (classifier i) Negative bag: all instance classifiers are negative NY (1 p i ) p =1 i=1 Probability of bag being positive: NY (1 p i ) i=1 47
48 Progress during the last decade Precision : Humans 0.750: Fused (ours + others) 0.741: MIL detector (ours) 0.739: Sparse Code Gradients 0.728: Sketch Tokens 0.714: GlobalPb Recall PrecisionRecall Curves on the Berkeley Benchmark
49 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 49
50 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 50
51 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 51
52 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 52
53 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 53
54 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 54
55 Machine Learning for Computer Vision Lecture 2 Learning symmetry detection S. Tsogkas & I.K., Learningbased symmetry detection in natural images, ECCV
56 Multiscale and multiorientation features 56 S. Tsogkas & I.K., Learningbased symmetry detection in natural images, ECCV 2012
57 Algorithm pipeline 57 Input image P(x,y) Features Brightness Texture Color Spectra l P(x,y) Training Multiple Instance Learning (MIL) nonmaximum suppression softmax θ,s (P(x,y,θ,s)) 14x1 weight vector S. Tsogkas & I.K., Learningbased symmetry detection in natural images, ECCV 2012
58 58 Feature extraction Color: CIE Lab color space. Texture: texton map. Hard binning (32 bins for 3 color channels, 64 textons). Rectangle filters extract features at multiple scales and orientations. Differences of histograms ( gradients ) of color and texture content à symmetry indication. χ 2 distance à dissimilarity of feature content between adjacent rectangles. Integral images for fast extraction. S. Tsogkas & I.K., Learningbased symmetry detection in natural images, ECCV 2012
59 MIL Training 59 Input image θ = 22.5 θ = 45 Varying angle θ = 90 θ = 135 Varying scale θ = MIL training s = 4 s = 8 w w w =! w s = 12 s = 24 s = 16 Probability map S. Tsogkas & I.K., Learningbased symmetry detection in natural images, ECCV 2012
60 60 Machine Learning for Computer Vision Lecture 2 Results Lindeberg Levinshtein This work Stateoftheart performance Large boost from color, texture and spectral cues. Long contours (region proposals) S. Tsogkas & I.K., Learningbased symmetry detection in natural images, ECCV 2012 Groundtruth
61 61 Machine Learning for Computer Vision Lecture 2 Some more results Lindeberg Levinshtein This work Groundtruth
62 Lecture recap 62 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL
63 Logistic regression training cost 63 Log loss Quadratic loss
64 Lecture recap 64 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL
65 Maximization with NewtonRaphson 65 NewtonRaphson method for finding maxima of ND function: Condition for maximum: Jacobian: Hessian:
66 Lecture recap 66 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL
67 Progress during the last 40 years 67 Humans Berkeley gpb, 08 Berkeley PB, 04 Canny+ Hysteresis ( 85) Prewitt, good feature= 5 years of work
68 Lecture outline 68 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning
69 Appendix 69 Newton Raphson = Iteratively Reweighted Least Squares Masking problem in multiclass linear regression & softmax
70 Newton Raphson = Iteratively Reweighted Least Squares 70 Newton Raphson: Hessian: Rewrite Update: Transform: Weighted Least Squares Fit:
71 Multiple classes & linear regression 71 K classes: oneofk coding 4 classes, ith sample is in 3 rd class: Matrix notation: Loss function: Least squares fit:
72 Multiple classes & linear regression 72 One linear discriminant per class: Problem: ambiguous regions Solution: assign to discriminant with largest score
73 Masking Problem in linear regression Class 1 Class 2 Class 3 73 Nothing ever gets assigned to class 2! 2D version:
74 Multiple classes & logistic regression 74 Soft maximum of competing classes: Discriminants (inputs) Softmax (outputs) Label probability: Similar steps for parameter estimation (Bishop s book)
75 Logistic vs Linear Regression, n>2 classes 75 Linear regression Logistic regression Logistic regression does not exhibit the masking problem
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical
More informationConvolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/
Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters
More informationCS 688 Pattern Recognition Lecture 4. Linear Models for Classification
CS 688 Pattern Recognition Lecture 4 Linear Models for Classification Probabilistic generative models Probabilistic discriminative models 1 Generative Approach ( x ) p C k p( C k ) Ck p ( ) ( x Ck ) p(
More informationLinear Threshold Units
Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear
More informationSegmentation & Clustering
EECS 442 Computer vision Segmentation & Clustering Segmentation in human vision Kmean clustering Meanshift Graphcut Reading: Chapters 14 [FP] Some slides of this lectures are courtesy of prof F. Li,
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct
More informationPa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classiﬁca6on
Pa8ern Recogni6on and Machine Learning Chapter 4: Linear Models for Classiﬁca6on Represen'ng the target values for classifica'on If there are only two classes, we typically use a single real valued output
More informationMultiScale Improves Boundary Detection in Natural Images
MultiScale Improves Boundary Detection in Natural Images Xiaofeng Ren Toyota Technological Institute at Chicago 1427 E. 60 Street, Chicago, IL 60637, USA xren@ttic.org Abstract. In this work we empirically
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationLecture 8 February 4
ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt
More informationModelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu Problematic
More informationLogistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression
Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max
More informationThese slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More informationLocal features and matching. Image classification & object localization
Overview Instance level search Local features and matching Efficient visual recognition Image classification & object localization Category recognition Image classification: assigning a class label to
More informationPractical Tour of Visual tracking. David Fleet and Allan Jepson January, 2006
Practical Tour of Visual tracking David Fleet and Allan Jepson January, 2006 Designing a Visual Tracker: What is the state? pose and motion (position, velocity, acceleration, ) shape (size, deformation,
More informationAutomatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 269 Class Project Report
Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 69 Class Project Report Junhua Mao and Lunbo Xu University of California, Los Angeles mjhustc@ucla.edu and lunbo
More informationProbabilistic Linear Classification: Logistic Regression. Piyush Rai IIT Kanpur
Probabilistic Linear Classification: Logistic Regression Piyush Rai IIT Kanpur Probabilistic Machine Learning (CS772A) Jan 18, 2016 Probabilistic Machine Learning (CS772A) Probabilistic Linear Classification:
More informationLogistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld.
Logistic Regression Vibhav Gogate The University of Texas at Dallas Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Generative vs. Discriminative Classifiers Want to Learn: h:x Y X features
More informationCanny Edge Detection
Canny Edge Detection 09gr820 March 23, 2009 1 Introduction The purpose of edge detection in general is to significantly reduce the amount of data in an image, while preserving the structural properties
More informationLogistic Regression (1/24/13)
STA63/CBB540: Statistical methods in computational biology Logistic Regression (/24/3) Lecturer: Barbara Engelhardt Scribe: Dinesh Manandhar Introduction Logistic regression is model for regression used
More informationSegmentation. Lecture 12. Many slides from: S. Lazebnik, K. Grauman and P. Kumar
Segmentation Lecture 12 Many slides from: S. Lazebnik, K. Grauman and P. Kumar Image Segmentation Image segmentation The goals of segmentation Group together similarlooking pixels for efficiency of further
More informationComputer Vision  part II
Computer Vision  part II Review of main parts of Section B of the course School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture Name Course Name 1 1 2
More informationIntroduction to Machine Learning
Introduction to Machine Learning Prof. Alexander Ihler Prof. Max Welling icamp Tutorial July 22 What is machine learning? The ability of a machine to improve its performance based on previous results:
More informationCSCI567 Machine Learning (Fall 2014)
CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 22, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 22, 2014 1 /
More informationImage Segmentation and Registration
Image Segmentation and Registration Dr. Christine Tanner (tanner@vision.ee.ethz.ch) Computer Vision Laboratory, ETH Zürich Dr. Verena Kaynig, Machine Learning Laboratory, ETH Zürich Outline Segmentation
More informationMachine Learning and Pattern Recognition Logistic Regression
Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,
More informationEdge detection. (Trucco, Chapt 4 AND Jain et al., Chapt 5) Edges are significant local changes of intensity in an image.
Edge detection (Trucco, Chapt 4 AND Jain et al., Chapt 5) Definition of edges Edges are significant local changes of intensity in an image. Edges typically occur on the boundary between two different
More informationCCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York
BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal  the stuff biology is not
More informationTracking Algorithms. Lecture17: Stochastic Tracking. Joint Probability and Graphical Model. Probabilistic Tracking
Tracking Algorithms (2015S) Lecture17: Stochastic Tracking Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Deterministic methods Given input video and current state, tracking result is always same. Local
More informationExample: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation:  Feature vector X,  qualitative response Y, taking values in C
More informationPredict Influencers in the Social Network
Predict Influencers in the Social Network Ruishan Liu, Yang Zhao and Liuyu Zhou Email: rliu2, yzhao2, lyzhou@stanford.edu Department of Electrical Engineering, Stanford University Abstract Given two persons
More informationChristfried Webers. Canberra February June 2015
c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic
More informationThe Artificial Prediction Market
The Artificial Prediction Market Adrian Barbu Department of Statistics Florida State University Joint work with Nathan Lay, Siemens Corporate Research 1 Overview Main Contributions A mathematical theory
More informationAssessment. Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall
Automatic Photo Quality Assessment Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall Estimating i the photorealism of images: Distinguishing i i paintings from photographs h Florin
More informationDATA ANALYTICS USING R
DATA ANALYTICS USING R Duration: 90 Hours Intended audience and scope: The course is targeted at fresh engineers, practicing engineers and scientists who are interested in learning and understanding data
More informationNeural Networks. CAP5610 Machine Learning Instructor: GuoJun Qi
Neural Networks CAP5610 Machine Learning Instructor: GuoJun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector
More informationEnvironmental Remote Sensing GEOG 2021
Environmental Remote Sensing GEOG 2021 Lecture 4 Image classification 2 Purpose categorising data data abstraction / simplification data interpretation mapping for land cover mapping use land cover class
More informationMachine Learning and Data Mining. Regression Problem. (adapted from) Prof. Alexander Ihler
Machine Learning and Data Mining Regression Problem (adapted from) Prof. Alexander Ihler Overview Regression Problem Definition and define parameters ϴ. Prediction using ϴ as parameters Measure the error
More informationLecture 6: Classification & Localization. boris. ginzburg@intel.com
Lecture 6: Classification & Localization boris. ginzburg@intel.com 1 Agenda ILSVRC 2014 Overfeat: integrated classification, localization, and detection Classification with Localization Detection. 2 ILSVRC2014
More informationIntroduction to Segmentation
Lecture 2: Introduction to Segmentation Jonathan Krause 1 Goal Goal: Identify groups of pixels that go together image credit: Steve Seitz, Kristen Grauman 2 Types of Segmentation Semantic Segmentation:
More informationRobert Collins CSE598G. More on Meanshift. R.Collins, CSE, PSU CSE598G Spring 2006
More on Meanshift R.Collins, CSE, PSU Spring 2006 Recall: Kernel Density Estimation Given a set of data samples x i ; i=1...n Convolve with a kernel function H to generate a smooth function f(x) Equivalent
More informationPattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University
Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision
More informationLinear Models for Classification
Linear Models for Classification Sumeet Agarwal, EEL709 (Most figures from Bishop, PRML) Approaches to classification Discriminant function: Directly assigns each data point x to a particular class Ci
More informationTHE development of methods for automatic detection
Learning to Detect Objects in Images via a Sparse, PartBased Representation Shivani Agarwal, Aatif Awan and Dan Roth, Member, IEEE Computer Society 1 Abstract We study the problem of detecting objects
More informationSupervised Learning (Big Data Analytics)
Supervised Learning (Big Data Analytics) Vibhav Gogate Department of Computer Science The University of Texas at Dallas Practical advice Goal of Big Data Analytics Uncover patterns in Data. Can be used
More informationLinear Classification. Volker Tresp Summer 2015
Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong
More informationDetermining optimal window size for texture feature extraction methods
IX Spanish Symposium on Pattern Recognition and Image Analysis, Castellon, Spain, May 2001, vol.2, 237242, ISBN: 8480213515. Determining optimal window size for texture feature extraction methods Domènec
More information3F3: Signal and Pattern Processing
3F3: Signal and Pattern Processing Lecture 3: Classification Zoubin Ghahramani zoubin@eng.cam.ac.uk Department of Engineering University of Cambridge Lent Term Classification We will represent data by
More informationImage Gradients. Given a discrete image Á Òµ, consider the smoothed continuous image Üµ defined by
Image Gradients Given a discrete image Á Òµ, consider the smoothed continuous image Üµ defined by Üµ Ü ¾ Ö µ Á Òµ Ü ¾ Ö µá µ (1) where Ü ¾ Ö Ô µ Ü ¾ Ý ¾. ½ ¾ ¾ Ö ¾ Ü ¾ ¾ Ö. Here Ü is the 2norm for the
More informationClassifying Chess Positions
Classifying Chess Positions Christopher De Sa December 14, 2012 Chess was one of the first problems studied by the AI community. While currently, chessplaying programs perform very well using primarily
More informationMachine Learning Logistic Regression
Machine Learning Logistic Regression Jeff Howbert Introduction to Machine Learning Winter 2012 1 Logistic regression Name is somewhat misleading. Really a technique for classification, not regression.
More informationA New Robust Algorithm for Video Text Extraction
A New Robust Algorithm for Video Text Extraction Pattern Recognition, vol. 36, no. 6, June 2003 Edward K. Wong and Minya Chen School of Electrical Engineering and Computer Science Kyungpook National Univ.
More informationEnsemble Methods. Knowledge Discovery and Data Mining 2 (VU) (707.004) Roman Kern. KTI, TU Graz 20150305
Ensemble Methods Knowledge Discovery and Data Mining 2 (VU) (707004) Roman Kern KTI, TU Graz 20150305 Roman Kern (KTI, TU Graz) Ensemble Methods 20150305 1 / 38 Outline 1 Introduction 2 Classification
More informationIntroduction to Logistic Regression
OpenStaxCNX module: m42090 1 Introduction to Logistic Regression Dan Calderon This work is produced by OpenStaxCNX and licensed under the Creative Commons Attribution License 3.0 Abstract Gives introduction
More informationIntroduction to Machine Learning Using Python. Vikram Kamath
Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression
More informationCSC 321 H1S Study Guide (Last update: April 3, 2016) Winter 2016
1. Suppose our training set and test set are the same. Why would this be a problem? 2. Why is it necessary to have both a test set and a validation set? 3. Images are generally represented as n m 3 arrays,
More informationImproving Webbased Image Search via Content Based Clustering
Improving Webbased Image Search via Content Based Clustering Nadav BenHaim, Boris Babenko and Serge Belongie Department of Computer Science and Engineering University of California, San Diego {nbenhaim,bbabenko,sjb}@ucsd.edu
More informationDecision Trees from large Databases: SLIQ
Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values
More informationAzure Machine Learning, SQL Data Mining and R
Azure Machine Learning, SQL Data Mining and R Daybyday Agenda Prerequisites No formal prerequisites. Basic knowledge of SQL Server Data Tools, Excel and any analytical experience helps. Best of all:
More informationColour Image Segmentation Technique for Screen Printing
60 R.U. Hewage and D.U.J. Sonnadara Department of Physics, University of Colombo, Sri Lanka ABSTRACT Screenprinting is an industry with a large number of applications ranging from printing mobile phone
More informationMACHINE LEARNING IN HIGH ENERGY PHYSICS
MACHINE LEARNING IN HIGH ENERGY PHYSICS LECTURE #1 Alex Rogozhnikov, 2015 INTRO NOTES 4 days two lectures, two practice seminars every day this is introductory track to machine learning kaggle competition!
More informationConcepts in Machine Learning, Unsupervised Learning & Astronomy Applications
Data Mining In Modern Astronomy Sky Surveys: Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications ChingWa Yip cwyip@pha.jhu.edu; Bloomberg 518 Human are Great Pattern Recognizers
More informationGLM, insurance pricing & big data: paying attention to convergence issues.
GLM, insurance pricing & big data: paying attention to convergence issues. Michaël NOACK  michael.noack@addactis.com Senior consultant & Manager of ADDACTIS Pricing Copyright 2014 ADDACTIS Worldwide.
More information521466S Machine Vision Assignment #7 Hough transform
521466S Machine Vision Assignment #7 Hough transform Spring 2014 In this assignment we use the hough transform to extract lines from images. We use the standard (r, θ) parametrization of lines, lter the
More informationAnalecta Vol. 8, No. 2 ISSN 20647964
EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,
More informationA Simple Introduction to Support Vector Machines
A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Largemargin linear
More informationLecture 2: The SVM classifier
Lecture 2: The SVM classifier C19 Machine Learning Hilary 2015 A. Zisserman Review of linear classifiers Linear separability Perceptron Support Vector Machine (SVM) classifier Wide margin Cost function
More informationClass #6: Nonlinear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris
Class #6: Nonlinear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Nonlinear classification Linear Support Vector Machines
More informationProbabilistic Latent Semantic Analysis (plsa)
Probabilistic Latent Semantic Analysis (plsa) SS 2008 Bayesian Networks Multimedia Computing, Universität Augsburg Rainer.Lienhart@informatik.uniaugsburg.de www.multimediacomputing.{de,org} References
More informationPractical Data Science with Azure Machine Learning, SQL Data Mining, and R
Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Overview This 4day class is the first of the two data science courses taught by Rafal Lukawiecki. Some of the topics will be
More informationLeastSquares Intersection of Lines
LeastSquares Intersection of Lines Johannes Traa  UIUC 2013 This writeup derives the leastsquares solution for the intersection of lines. In the general case, a set of lines will not intersect at a
More informationIntroduction to Deep Learning Variational Inference, Mean Field Theory
Introduction to Deep Learning Variational Inference, Mean Field Theory 1 Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Center for Visual Computing Ecole Centrale Paris Galen Group INRIASaclay Lecture 3: recap
More informationMeanShift Tracking with Random Sampling
1 MeanShift Tracking with Random Sampling Alex Po Leung, Shaogang Gong Department of Computer Science Queen Mary, University of London, London, E1 4NS Abstract In this work, boosting the efficiency of
More informationLecture 6: Logistic Regression
Lecture 6: CS 19410, Fall 2011 Laurent El Ghaoui EECS Department UC Berkeley September 13, 2011 Outline Outline Classification task Data : X = [x 1,..., x m]: a n m matrix of data points in R n. y { 1,
More informationSimple and efficient online algorithms for real world applications
Simple and efficient online algorithms for real world applications Università degli Studi di Milano Milano, Italy Talk @ Centro de Visión por Computador Something about me PhD in Robotics at LIRALab,
More informationQUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING
QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING MRS. A H. TIRMARE 1, MS.R.N.KULKARNI 2, MR. A R. BHOSALE 3 MR. C.S. MORE 4 MR.A.G.NIMBALKAR 5 1, 2 Assistant professor Bharati Vidyapeeth s college
More informationOnline Learning for Fast Segmentation of Moving Objects
Online Learning for Fast Segmentation of Moving Objects Liam Ellis, Vasileios Zografos {liam.ellis,vasileios.zografos}@liu.se CVL, Linköping University, Linköping, Sweden Abstract. This work addresses
More informationLecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection
CSED703R: Deep Learning for Visual Recognition (206S) Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection Bohyung Han Computer Vision Lab. bhhan@postech.ac.kr 2 3 Object detection
More informationData Mining. Nonlinear Classification
Data Mining Unit # 6 Sajjad Haider Fall 2014 1 Nonlinear Classification Classes may not be separable by a linear boundary Suppose we randomly generate a data set as follows: X has range between 0 to 15
More informationRecognizing Cats and Dogs with Shape and Appearance based Models. Group Member: Chu Wang, Landu Jiang
Recognizing Cats and Dogs with Shape and Appearance based Models Group Member: Chu Wang, Landu Jiang Abstract Recognizing cats and dogs from images is a challenging competition raised by Kaggle platform
More informationAn Iterative Image Registration Technique with an Application to Stereo Vision
An Iterative Image Registration Technique with an Application to Stereo Vision Bruce D. Lucas Takeo Kanade Computer Science Department CarnegieMellon University Pittsburgh, Pennsylvania 15213 Abstract
More informationIntroduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk
Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems
More informationAutomatic parameter regulation for a tracking system with an autocritical function
Automatic parameter regulation for a tracking system with an autocritical function Daniela Hall INRIA RhôneAlpes, St. Ismier, France Email: Daniela.Hall@inrialpes.fr Abstract In this article we propose
More informationDYNAMIC RANGE IMPROVEMENT THROUGH MULTIPLE EXPOSURES. Mark A. Robertson, Sean Borman, and Robert L. Stevenson
c 1999 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or
More informationClassification of Bad Accounts in Credit Card Industry
Classification of Bad Accounts in Credit Card Industry Chengwei Yuan December 12, 2014 Introduction Risk management is critical for a credit card company to survive in such competing industry. In addition
More informationCSC 411: Lecture 07: Multiclass Classification
CSC 411: Lecture 07: Multiclass Classification Class based on Raquel Urtasun & Rich Zemel s lectures Sanja Fidler University of Toronto Feb 1, 2016 Urtasun, Zemel, Fidler (UofT) CSC 411: 07Multiclass
More informationNatural Language Processing. Today. Logistic Regression Models. Lecture 13 10/6/2015. Jim Martin. Multinomial Logistic Regression
Natural Language Processing Lecture 13 10/6/2015 Jim Martin Today Multinomial Logistic Regression Aka loglinear models or maximum entropy (maxent) Components of the model Learning the parameters 10/1/15
More informationThe Role of Size Normalization on the Recognition Rate of Handwritten Numerals
The Role of Size Normalization on the Recognition Rate of Handwritten Numerals Chun Lei He, Ping Zhang, Jianxiong Dong, Ching Y. Suen, Tien D. Bui Centre for Pattern Recognition and Machine Intelligence,
More informationCOMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS
COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS B.K. Mohan and S. N. Ladha Centre for Studies in Resources Engineering IIT
More informationSocial Media Mining. Data Mining Essentials
Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers
More informationA Study on SURF Algorithm and RealTime Tracking Objects Using Optical Flow
, pp.233237 http://dx.doi.org/10.14257/astl.2014.51.53 A Study on SURF Algorithm and RealTime Tracking Objects Using Optical Flow Giwoo Kim 1, HyeYoun Lim 1 and DaeSeong Kang 1, 1 Department of electronices
More informationFeature Tracking and Optical Flow
02/09/12 Feature Tracking and Optical Flow Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve
More informationInteractive Offline Tracking for Color Objects
Interactive Offline Tracking for Color Objects Yichen Wei Jian Sun Xiaoou Tang HeungYeung Shum Microsoft Research Asia, Beijing, China {yichenw,jiansun,xitang,hshum}@microsoft.com Abstract In this paper,
More informationSemantic Recognition: Object Detection and Scene Segmentation
Semantic Recognition: Object Detection and Scene Segmentation Xuming He xuming.he@nicta.com.au Computer Vision Research Group NICTA Robotic Vision Summer School 2015 Acknowledgement: Slides from FeiFei
More informationA Simple Feature Extraction Technique of a Pattern By Hopfield Network
A Simple Feature Extraction Technique of a Pattern By Hopfield Network A.Nag!, S. Biswas *, D. Sarkar *, P.P. Sarkar *, B. Gupta **! Academy of Technology, Hoogly  722 *USIC, University of Kalyani, Kalyani
More informationPalmprint Recognition. By Sree Rama Murthy kora Praveen Verma Yashwant Kashyap
Palmprint Recognition By Sree Rama Murthy kora Praveen Verma Yashwant Kashyap Palm print Palm Patterns are utilized in many applications: 1. To correlate palm patterns with medical disorders, e.g. genetic
More informationAN IMPROVED DOUBLE CODING LOCAL BINARY PATTERN ALGORITHM FOR FACE RECOGNITION
AN IMPROVED DOUBLE CODING LOCAL BINARY PATTERN ALGORITHM FOR FACE RECOGNITION Saurabh Asija 1, Rakesh Singh 2 1 Research Scholar (Computer Engineering Department), Punjabi University, Patiala. 2 Asst.
More informationNonlinear Iterative Partial Least Squares Method
Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., RichardPlouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments for
More informationLIBSVX and Video Segmentation Evaluation
CVPR 14 Tutorial! 1! LIBSVX and Video Segmentation Evaluation Chenliang Xu and Jason J. Corso!! Computer Science and Engineering! SUNY at Buffalo!! Electrical Engineering and Computer Science! University
More information