MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos

Size: px
Start display at page:

Download "MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr"

Transcription

1 Machine Learning for Computer Vision 1 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Department of Applied Mathematics Ecole Centrale Paris Galen Group INRIA-Saclay

2 Administrative details 2 Make sure you have all received my from last week

3 Lecture outline 3 Recap & problems of linear regression Logistic Regression Introduction to Multiple Instance Learning

4 4 Learning problem Recover input-output mapping Output: y Input: x Method: f Parameters: w Aspects of the learning problem Identify methods that fit the problem setting Determine parameters that properly classify the training set Measure and control the complexity of these functions

5 Linear Classifiers Find linear expression (hyperplane) to separate positive and negative examples 5 Feature coordinate j x x i i positive : negative : x x i i w + b w + b 0 < 0 Each data point has a class label: +1 ( ) y t = -1 ( ) Feature coordinate i

6 Learning problem formulation 6 Given: Training set of feature-label pairs Wanted: simple that works well for simple: penalizing function complexity, to guarantee generalization works well: quantified by loss criterion

7 Linear regression 7 Linear Loss function: quantify appropriateness of for Introduce vector notation

8 Inappropriateness of quadratic loss We chose the quadratic cost function for convenience Single, global minimum & closed form expression 8 But does it indicate classification performance? Computed Decision Boundary Linear Fit Desired decision boundary

9 Inappropriateness of quadratic loss Consider transformation: 9 Quadratic loss is not robust to outliers and penalizes outputs that are `too good

10 Lecture outline 10 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

11 Probabilistic interpretation of least squares 11 Outputs: continuous random variables y = hx, wi +, N(0, ) linear function of inputs + zero mean Gaussian r.v. Optimal w: maximizes likelihood of observed outputs But the labels are discrete!

12 A probabilistic criterion for training a classifier 12 Training set: y: discrete observations: model as samples from Bernoulli distribution Find w that maximizes the likelihood of labels in the training set

13 Sigmoidal function & logistic regression 13 sigmoidal Training criterion from previous slide: How do we optimize it with respect to w? What does it mean?

14 Lecture outline 14 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

15 Maximization by gradient ascent 15 Gradient ascent:

16 Maximization with Newton-Raphson 16 Newton method for finding roots of 1D function : Newton method for finding maxima of 1D function : condition Newton-Raphson method for finding maxima of N-D function :

17 Newton-Raphson for Logistic Regression 17 Gradient: Hessian: In statistics: Iteratively Reweighted Least Squares (IRLS)

18 Lecture outline 18 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Support Vector Machines

19 A compact expression for the loss 19 Using, :

20 Log loss: 20

21 Log loss vs. quadratic loss 21

22 Logistic vs Linear Regression 22 Logistic regression is more robust Linear Regression Logistic Regression

23 Lecture outline 23 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

24 Boundary detection problem Object/Surface Boundaries 24

25 25 Signal-level challenges Poor contrast Shadows Texture

26 Machine Learning for Computer Vision Lecture 2 Fundamental challenges: can humans do it? 26

27 Learning-based approaches 27 Boundary or non-boundary? Use human-annotated segmentations Use any visual cue as input to the decision function. Use decision trees/logisitic regression/boosting/ and learn to combine the individual inputs. S. Konishi, A.Yuille, J. Coughlan, S.C. Zhu, Statistical Edge Detection: Learning and Evaluating Edge Cues, IEEE PAMI, 2003 D. Martin, C. Fowlkes, J. Malik. "Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues", IEEE PAMI, 2004

28 Evaluation protocol 28 Threshold detector s output at certain level Match outputs to human ground-truth Count true/false positives (t/f)positives, misses (m) From precision, p and recall, r Quantify performance in terms of F-measure

29 29 Contours can be defined by any of a number of cues (P. Cavanagh) Slide credit: J. Malik, M. Maire

30 Cue-localization Gray level photographs 30 Objects from motion Objects from luminance Objects from disparity Line drawings Objects from texture Slide credit: J. Malik, M. Maire Grill-Spector et al., Neuron 1998

31 31 Local boundary cues Separate features per candidate orientation In specific: r (x,y) θ 1976 CIE L*a*b* colorspace Brightness Gradient BG(x,y,r,θ) Difference of L* distributions Color Gradient CG(x,y,r,θ) Difference of a*b* distributions Texture Gradient TG(x,y,r,θ) Difference of distributions of V1-like filter responses Slide credit: J. Malik, M. Maire

32 Boundary cues (horizontal) 32 Brightness Color (a, b channels) Texture

33 Boundary π/4 33 Brightness Color (a, b channels) Texture

34 Brightness π/2 34 Brightness Color (a, b channels) Texture

35 Brightness 3π/4 35 Brightness Color (a, b channels) Texture

36 Cue combinations with logistic regression 36 Slide credit: J. Malik, M. Maire

37 Exploiting global constraints: image Segmentation as Graph Partitioning Build a weighted graph G=(V,E) from image V: image pixels 37 E: connections between pairs of nearby pixels Partition graph so that similarity within group is large and similarity between groups is small -- Normalized Cuts [Shi & Malik 97] Slide credit: J. Malik, M. Maire

38 38 Wij small when intervening contour strong, strong when weak.. Slide credit: J. Malik, M. Maire

39 Normalized Cuts as a Spring-Mass system 39 Each pixel is a point mass; each connection is a spring: ( D W ) y = λdy Fundamental modes are generalized eigenvectors of (D - W) x = λdx Slide credit: J. Malik, M. Maire

40 Contour information from eigenvectors 40 Slide credit: J. Malik, M. Maire

41 Classifier 41 Logistic regression

42 Progress during the last 40 years 42 Humans Berkeley gpb, 08 Berkeley PB, 04 Canny+ Hysteresis ( 85) Prewitt, good feature= 5 years of work

43 Lecture outline 43 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

44 Learning boundary detection 44 Problem I: inconsistent orientation information Problem II: inconsistent location information

45 Sneaking into the fun room 45 mom s keychain grandma s keychain dad s keychain We know that dad cannot enter the fun room, either Which key should we try? Slide Credit: B. Babenko/T. Dietterich

46 Multiple Instance Learning 46 Typical Learning Multiple Instance Learning Positive bag: at least one instance should be positive Negative bag: no instance should be positive Slide Credit: K. Grauman

47 Noisy-or classifier combination N classifier responses, {p 1,...,p N } p i : probability of positive label (classifier i) 1 p i : probability of negative (classifier i) Negative bag: all instance classifiers are negative NY (1 p i ) p =1 i=1 Probability of bag being positive: NY (1 p i ) i=1 47

48 Progress during the last decade Precision : Humans 0.750: Fused (ours + others) 0.741: MIL detector (ours) 0.739: Sparse Code Gradients 0.728: Sketch Tokens 0.714: GlobalPb Recall Precision-Recall Curves on the Berkeley Benchmark

49 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 49

50 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 50

51 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 51

52 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 52

53 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 53

54 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 54

55 Machine Learning for Computer Vision Lecture 2 Learning symmetry detection S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV

56 Multi-scale and multi-orientation features 56 S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

57 Algorithm pipeline 57 Input image P(x,y) Features Brightness Texture Color Spectra l P(x,y) Training Multiple Instance Learning (MIL) non-maximum suppression softmax θ,s (P(x,y,θ,s)) 14x1 weight vector S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

58 58 Feature extraction Color: CIE Lab color space. Texture: texton map. Hard binning (32 bins for 3 color channels, 64 textons). Rectangle filters extract features at multiple scales and orientations. Differences of histograms ( gradients ) of color and texture content à symmetry indication. χ 2 distance à dissimilarity of feature content between adjacent rectangles. Integral images for fast extraction. S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

59 MIL Training 59 Input image θ = 22.5 θ = 45 Varying angle θ = 90 θ = 135 Varying scale θ = MIL training s = 4 s = 8 w w w =! w s = 12 s = 24 s = 16 Probability map S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

60 60 Machine Learning for Computer Vision Lecture 2 Results Lindeberg Levinshtein This work State-of-the-art performance Large boost from color, texture and spectral cues. Long contours (region proposals) S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012 Ground-truth

61 61 Machine Learning for Computer Vision Lecture 2 Some more results Lindeberg Levinshtein This work Ground-truth

62 Lecture recap 62 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL

63 Logistic regression training cost 63 Log loss Quadratic loss

64 Lecture recap 64 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL

65 Maximization with Newton-Raphson 65 Newton-Raphson method for finding maxima of N-D function: Condition for maximum: Jacobian: Hessian:

66 Lecture recap 66 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL

67 Progress during the last 40 years 67 Humans Berkeley gpb, 08 Berkeley PB, 04 Canny+ Hysteresis ( 85) Prewitt, good feature= 5 years of work

68 Lecture outline 68 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

69 Appendix 69 Newton Raphson = Iteratively Reweighted Least Squares Masking problem in multi-class linear regression & softmax

70 Newton Raphson = Iteratively Reweighted Least Squares 70 Newton Raphson: Hessian: Rewrite Update: Transform: Weighted Least Squares Fit:

71 Multiple classes & linear regression 71 K classes: one-of-k coding 4 classes, i-th sample is in 3 rd class: Matrix notation: Loss function: Least squares fit:

72 Multiple classes & linear regression 72 One linear discriminant per class: Problem: ambiguous regions Solution: assign to discriminant with largest score

73 Masking Problem in linear regression Class 1 Class 2 Class 3 73 Nothing ever gets assigned to class 2! 2D version:

74 Multiple classes & logistic regression 74 Soft maximum of competing classes: Discriminants (inputs) Softmax (outputs) Label probability: Similar steps for parameter estimation (Bishop s book)

75 Logistic vs Linear Regression, n>2 classes 75 Linear regression Logistic regression Logistic regression does not exhibit the masking problem

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

More information

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/ Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters

More information

CS 688 Pattern Recognition Lecture 4. Linear Models for Classification

CS 688 Pattern Recognition Lecture 4. Linear Models for Classification CS 688 Pattern Recognition Lecture 4 Linear Models for Classification Probabilistic generative models Probabilistic discriminative models 1 Generative Approach ( x ) p C k p( C k ) Ck p ( ) ( x Ck ) p(

More information

Linear Threshold Units

Linear Threshold Units Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

More information

Segmentation & Clustering

Segmentation & Clustering EECS 442 Computer vision Segmentation & Clustering Segmentation in human vision K-mean clustering Mean-shift Graph-cut Reading: Chapters 14 [FP] Some slides of this lectures are courtesy of prof F. Li,

More information

STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

More information

Pa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classifica6on

Pa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classifica6on Pa8ern Recogni6on and Machine Learning Chapter 4: Linear Models for Classifica6on Represen'ng the target values for classifica'on If there are only two classes, we typically use a single real valued output

More information

Multi-Scale Improves Boundary Detection in Natural Images

Multi-Scale Improves Boundary Detection in Natural Images Multi-Scale Improves Boundary Detection in Natural Images Xiaofeng Ren Toyota Technological Institute at Chicago 1427 E. 60 Street, Chicago, IL 60637, USA xren@tti-c.org Abstract. In this work we empirically

More information

Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

Lecture 8 February 4

Lecture 8 February 4 ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt

More information

Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches

Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu Problematic

More information

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

More information

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher

More information

Statistical Machine Learning

Statistical Machine Learning Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

More information

Local features and matching. Image classification & object localization

Local features and matching. Image classification & object localization Overview Instance level search Local features and matching Efficient visual recognition Image classification & object localization Category recognition Image classification: assigning a class label to

More information

Practical Tour of Visual tracking. David Fleet and Allan Jepson January, 2006

Practical Tour of Visual tracking. David Fleet and Allan Jepson January, 2006 Practical Tour of Visual tracking David Fleet and Allan Jepson January, 2006 Designing a Visual Tracker: What is the state? pose and motion (position, velocity, acceleration, ) shape (size, deformation,

More information

Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 269 Class Project Report

Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 269 Class Project Report Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 69 Class Project Report Junhua Mao and Lunbo Xu University of California, Los Angeles mjhustc@ucla.edu and lunbo

More information

Probabilistic Linear Classification: Logistic Regression. Piyush Rai IIT Kanpur

Probabilistic Linear Classification: Logistic Regression. Piyush Rai IIT Kanpur Probabilistic Linear Classification: Logistic Regression Piyush Rai IIT Kanpur Probabilistic Machine Learning (CS772A) Jan 18, 2016 Probabilistic Machine Learning (CS772A) Probabilistic Linear Classification:

More information

Logistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld.

Logistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Logistic Regression Vibhav Gogate The University of Texas at Dallas Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Generative vs. Discriminative Classifiers Want to Learn: h:x Y X features

More information

Canny Edge Detection

Canny Edge Detection Canny Edge Detection 09gr820 March 23, 2009 1 Introduction The purpose of edge detection in general is to significantly reduce the amount of data in an image, while preserving the structural properties

More information

Logistic Regression (1/24/13)

Logistic Regression (1/24/13) STA63/CBB540: Statistical methods in computational biology Logistic Regression (/24/3) Lecturer: Barbara Engelhardt Scribe: Dinesh Manandhar Introduction Logistic regression is model for regression used

More information

Segmentation. Lecture 12. Many slides from: S. Lazebnik, K. Grauman and P. Kumar

Segmentation. Lecture 12. Many slides from: S. Lazebnik, K. Grauman and P. Kumar Segmentation Lecture 12 Many slides from: S. Lazebnik, K. Grauman and P. Kumar Image Segmentation Image segmentation The goals of segmentation Group together similar-looking pixels for efficiency of further

More information

Computer Vision - part II

Computer Vision - part II Computer Vision - part II Review of main parts of Section B of the course School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture Name Course Name 1 1 2

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Prof. Alexander Ihler Prof. Max Welling icamp Tutorial July 22 What is machine learning? The ability of a machine to improve its performance based on previous results:

More information

CSCI567 Machine Learning (Fall 2014)

CSCI567 Machine Learning (Fall 2014) CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 22, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 22, 2014 1 /

More information

Image Segmentation and Registration

Image Segmentation and Registration Image Segmentation and Registration Dr. Christine Tanner (tanner@vision.ee.ethz.ch) Computer Vision Laboratory, ETH Zürich Dr. Verena Kaynig, Machine Learning Laboratory, ETH Zürich Outline Segmentation

More information

Machine Learning and Pattern Recognition Logistic Regression

Machine Learning and Pattern Recognition Logistic Regression Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,

More information

Edge detection. (Trucco, Chapt 4 AND Jain et al., Chapt 5) -Edges are significant local changes of intensity in an image.

Edge detection. (Trucco, Chapt 4 AND Jain et al., Chapt 5) -Edges are significant local changes of intensity in an image. Edge detection (Trucco, Chapt 4 AND Jain et al., Chapt 5) Definition of edges -Edges are significant local changes of intensity in an image. -Edges typically occur on the boundary between two different

More information

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

More information

Tracking Algorithms. Lecture17: Stochastic Tracking. Joint Probability and Graphical Model. Probabilistic Tracking

Tracking Algorithms. Lecture17: Stochastic Tracking. Joint Probability and Graphical Model. Probabilistic Tracking Tracking Algorithms (2015S) Lecture17: Stochastic Tracking Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Deterministic methods Given input video and current state, tracking result is always same. Local

More information

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not. Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

More information

Predict Influencers in the Social Network

Predict Influencers in the Social Network Predict Influencers in the Social Network Ruishan Liu, Yang Zhao and Liuyu Zhou Email: rliu2, yzhao2, lyzhou@stanford.edu Department of Electrical Engineering, Stanford University Abstract Given two persons

More information

Christfried Webers. Canberra February June 2015

Christfried Webers. Canberra February June 2015 c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic

More information

The Artificial Prediction Market

The Artificial Prediction Market The Artificial Prediction Market Adrian Barbu Department of Statistics Florida State University Joint work with Nathan Lay, Siemens Corporate Research 1 Overview Main Contributions A mathematical theory

More information

Assessment. Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall

Assessment. Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall Automatic Photo Quality Assessment Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall Estimating i the photorealism of images: Distinguishing i i paintings from photographs h Florin

More information

DATA ANALYTICS USING R

DATA ANALYTICS USING R DATA ANALYTICS USING R Duration: 90 Hours Intended audience and scope: The course is targeted at fresh engineers, practicing engineers and scientists who are interested in learning and understanding data

More information

Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi

Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector

More information

Environmental Remote Sensing GEOG 2021

Environmental Remote Sensing GEOG 2021 Environmental Remote Sensing GEOG 2021 Lecture 4 Image classification 2 Purpose categorising data data abstraction / simplification data interpretation mapping for land cover mapping use land cover class

More information

Machine Learning and Data Mining. Regression Problem. (adapted from) Prof. Alexander Ihler

Machine Learning and Data Mining. Regression Problem. (adapted from) Prof. Alexander Ihler Machine Learning and Data Mining Regression Problem (adapted from) Prof. Alexander Ihler Overview Regression Problem Definition and define parameters ϴ. Prediction using ϴ as parameters Measure the error

More information

Lecture 6: Classification & Localization. boris. ginzburg@intel.com

Lecture 6: Classification & Localization. boris. ginzburg@intel.com Lecture 6: Classification & Localization boris. ginzburg@intel.com 1 Agenda ILSVRC 2014 Overfeat: integrated classification, localization, and detection Classification with Localization Detection. 2 ILSVRC-2014

More information

Introduction to Segmentation

Introduction to Segmentation Lecture 2: Introduction to Segmentation Jonathan Krause 1 Goal Goal: Identify groups of pixels that go together image credit: Steve Seitz, Kristen Grauman 2 Types of Segmentation Semantic Segmentation:

More information

Robert Collins CSE598G. More on Mean-shift. R.Collins, CSE, PSU CSE598G Spring 2006

Robert Collins CSE598G. More on Mean-shift. R.Collins, CSE, PSU CSE598G Spring 2006 More on Mean-shift R.Collins, CSE, PSU Spring 2006 Recall: Kernel Density Estimation Given a set of data samples x i ; i=1...n Convolve with a kernel function H to generate a smooth function f(x) Equivalent

More information

Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University

Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision

More information

Linear Models for Classification

Linear Models for Classification Linear Models for Classification Sumeet Agarwal, EEL709 (Most figures from Bishop, PRML) Approaches to classification Discriminant function: Directly assigns each data point x to a particular class Ci

More information

THE development of methods for automatic detection

THE development of methods for automatic detection Learning to Detect Objects in Images via a Sparse, Part-Based Representation Shivani Agarwal, Aatif Awan and Dan Roth, Member, IEEE Computer Society 1 Abstract We study the problem of detecting objects

More information

Supervised Learning (Big Data Analytics)

Supervised Learning (Big Data Analytics) Supervised Learning (Big Data Analytics) Vibhav Gogate Department of Computer Science The University of Texas at Dallas Practical advice Goal of Big Data Analytics Uncover patterns in Data. Can be used

More information

Linear Classification. Volker Tresp Summer 2015

Linear Classification. Volker Tresp Summer 2015 Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

More information

Determining optimal window size for texture feature extraction methods

Determining optimal window size for texture feature extraction methods IX Spanish Symposium on Pattern Recognition and Image Analysis, Castellon, Spain, May 2001, vol.2, 237-242, ISBN: 84-8021-351-5. Determining optimal window size for texture feature extraction methods Domènec

More information

3F3: Signal and Pattern Processing

3F3: Signal and Pattern Processing 3F3: Signal and Pattern Processing Lecture 3: Classification Zoubin Ghahramani zoubin@eng.cam.ac.uk Department of Engineering University of Cambridge Lent Term Classification We will represent data by

More information

Image Gradients. Given a discrete image Á Òµ, consider the smoothed continuous image ܵ defined by

Image Gradients. Given a discrete image Á Òµ, consider the smoothed continuous image ܵ defined by Image Gradients Given a discrete image Á Òµ, consider the smoothed continuous image ܵ defined by ܵ Ü ¾ Ö µ Á Òµ Ü ¾ Ö µá µ (1) where Ü ¾ Ö Ô µ Ü ¾ Ý ¾. ½ ¾ ¾ Ö ¾ Ü ¾ ¾ Ö. Here Ü is the 2-norm for the

More information

Classifying Chess Positions

Classifying Chess Positions Classifying Chess Positions Christopher De Sa December 14, 2012 Chess was one of the first problems studied by the AI community. While currently, chessplaying programs perform very well using primarily

More information

Machine Learning Logistic Regression

Machine Learning Logistic Regression Machine Learning Logistic Regression Jeff Howbert Introduction to Machine Learning Winter 2012 1 Logistic regression Name is somewhat misleading. Really a technique for classification, not regression.

More information

A New Robust Algorithm for Video Text Extraction

A New Robust Algorithm for Video Text Extraction A New Robust Algorithm for Video Text Extraction Pattern Recognition, vol. 36, no. 6, June 2003 Edward K. Wong and Minya Chen School of Electrical Engineering and Computer Science Kyungpook National Univ.

More information

Ensemble Methods. Knowledge Discovery and Data Mining 2 (VU) (707.004) Roman Kern. KTI, TU Graz 2015-03-05

Ensemble Methods. Knowledge Discovery and Data Mining 2 (VU) (707.004) Roman Kern. KTI, TU Graz 2015-03-05 Ensemble Methods Knowledge Discovery and Data Mining 2 (VU) (707004) Roman Kern KTI, TU Graz 2015-03-05 Roman Kern (KTI, TU Graz) Ensemble Methods 2015-03-05 1 / 38 Outline 1 Introduction 2 Classification

More information

Introduction to Logistic Regression

Introduction to Logistic Regression OpenStax-CNX module: m42090 1 Introduction to Logistic Regression Dan Calderon This work is produced by OpenStax-CNX and licensed under the Creative Commons Attribution License 3.0 Abstract Gives introduction

More information

Introduction to Machine Learning Using Python. Vikram Kamath

Introduction to Machine Learning Using Python. Vikram Kamath Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression

More information

CSC 321 H1S Study Guide (Last update: April 3, 2016) Winter 2016

CSC 321 H1S Study Guide (Last update: April 3, 2016) Winter 2016 1. Suppose our training set and test set are the same. Why would this be a problem? 2. Why is it necessary to have both a test set and a validation set? 3. Images are generally represented as n m 3 arrays,

More information

Improving Web-based Image Search via Content Based Clustering

Improving Web-based Image Search via Content Based Clustering Improving Web-based Image Search via Content Based Clustering Nadav Ben-Haim, Boris Babenko and Serge Belongie Department of Computer Science and Engineering University of California, San Diego {nbenhaim,bbabenko,sjb}@ucsd.edu

More information

Decision Trees from large Databases: SLIQ

Decision Trees from large Databases: SLIQ Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values

More information

Azure Machine Learning, SQL Data Mining and R

Azure Machine Learning, SQL Data Mining and R Azure Machine Learning, SQL Data Mining and R Day-by-day Agenda Prerequisites No formal prerequisites. Basic knowledge of SQL Server Data Tools, Excel and any analytical experience helps. Best of all:

More information

Colour Image Segmentation Technique for Screen Printing

Colour Image Segmentation Technique for Screen Printing 60 R.U. Hewage and D.U.J. Sonnadara Department of Physics, University of Colombo, Sri Lanka ABSTRACT Screen-printing is an industry with a large number of applications ranging from printing mobile phone

More information

MACHINE LEARNING IN HIGH ENERGY PHYSICS

MACHINE LEARNING IN HIGH ENERGY PHYSICS MACHINE LEARNING IN HIGH ENERGY PHYSICS LECTURE #1 Alex Rogozhnikov, 2015 INTRO NOTES 4 days two lectures, two practice seminars every day this is introductory track to machine learning kaggle competition!

More information

Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications

Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications Data Mining In Modern Astronomy Sky Surveys: Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications Ching-Wa Yip cwyip@pha.jhu.edu; Bloomberg 518 Human are Great Pattern Recognizers

More information

GLM, insurance pricing & big data: paying attention to convergence issues.

GLM, insurance pricing & big data: paying attention to convergence issues. GLM, insurance pricing & big data: paying attention to convergence issues. Michaël NOACK - michael.noack@addactis.com Senior consultant & Manager of ADDACTIS Pricing Copyright 2014 ADDACTIS Worldwide.

More information

521466S Machine Vision Assignment #7 Hough transform

521466S Machine Vision Assignment #7 Hough transform 521466S Machine Vision Assignment #7 Hough transform Spring 2014 In this assignment we use the hough transform to extract lines from images. We use the standard (r, θ) parametrization of lines, lter the

More information

Analecta Vol. 8, No. 2 ISSN 2064-7964

Analecta Vol. 8, No. 2 ISSN 2064-7964 EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,

More information

A Simple Introduction to Support Vector Machines

A Simple Introduction to Support Vector Machines A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Large-margin linear

More information

Lecture 2: The SVM classifier

Lecture 2: The SVM classifier Lecture 2: The SVM classifier C19 Machine Learning Hilary 2015 A. Zisserman Review of linear classifiers Linear separability Perceptron Support Vector Machine (SVM) classifier Wide margin Cost function

More information

Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

More information

Probabilistic Latent Semantic Analysis (plsa)

Probabilistic Latent Semantic Analysis (plsa) Probabilistic Latent Semantic Analysis (plsa) SS 2008 Bayesian Networks Multimedia Computing, Universität Augsburg Rainer.Lienhart@informatik.uni-augsburg.de www.multimedia-computing.{de,org} References

More information

Practical Data Science with Azure Machine Learning, SQL Data Mining, and R

Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Overview This 4-day class is the first of the two data science courses taught by Rafal Lukawiecki. Some of the topics will be

More information

Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

More information

Introduction to Deep Learning Variational Inference, Mean Field Theory

Introduction to Deep Learning Variational Inference, Mean Field Theory Introduction to Deep Learning Variational Inference, Mean Field Theory 1 Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Center for Visual Computing Ecole Centrale Paris Galen Group INRIA-Saclay Lecture 3: recap

More information

Mean-Shift Tracking with Random Sampling

Mean-Shift Tracking with Random Sampling 1 Mean-Shift Tracking with Random Sampling Alex Po Leung, Shaogang Gong Department of Computer Science Queen Mary, University of London, London, E1 4NS Abstract In this work, boosting the efficiency of

More information

Lecture 6: Logistic Regression

Lecture 6: Logistic Regression Lecture 6: CS 194-10, Fall 2011 Laurent El Ghaoui EECS Department UC Berkeley September 13, 2011 Outline Outline Classification task Data : X = [x 1,..., x m]: a n m matrix of data points in R n. y { 1,

More information

Simple and efficient online algorithms for real world applications

Simple and efficient online algorithms for real world applications Simple and efficient online algorithms for real world applications Università degli Studi di Milano Milano, Italy Talk @ Centro de Visión por Computador Something about me PhD in Robotics at LIRA-Lab,

More information

QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING

QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING MRS. A H. TIRMARE 1, MS.R.N.KULKARNI 2, MR. A R. BHOSALE 3 MR. C.S. MORE 4 MR.A.G.NIMBALKAR 5 1, 2 Assistant professor Bharati Vidyapeeth s college

More information

Online Learning for Fast Segmentation of Moving Objects

Online Learning for Fast Segmentation of Moving Objects Online Learning for Fast Segmentation of Moving Objects Liam Ellis, Vasileios Zografos {liam.ellis,vasileios.zografos}@liu.se CVL, Linköping University, Linköping, Sweden Abstract. This work addresses

More information

Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection

Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection CSED703R: Deep Learning for Visual Recognition (206S) Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection Bohyung Han Computer Vision Lab. bhhan@postech.ac.kr 2 3 Object detection

More information

Data Mining. Nonlinear Classification

Data Mining. Nonlinear Classification Data Mining Unit # 6 Sajjad Haider Fall 2014 1 Nonlinear Classification Classes may not be separable by a linear boundary Suppose we randomly generate a data set as follows: X has range between 0 to 15

More information

Recognizing Cats and Dogs with Shape and Appearance based Models. Group Member: Chu Wang, Landu Jiang

Recognizing Cats and Dogs with Shape and Appearance based Models. Group Member: Chu Wang, Landu Jiang Recognizing Cats and Dogs with Shape and Appearance based Models Group Member: Chu Wang, Landu Jiang Abstract Recognizing cats and dogs from images is a challenging competition raised by Kaggle platform

More information

An Iterative Image Registration Technique with an Application to Stereo Vision

An Iterative Image Registration Technique with an Application to Stereo Vision An Iterative Image Registration Technique with an Application to Stereo Vision Bruce D. Lucas Takeo Kanade Computer Science Department Carnegie-Mellon University Pittsburgh, Pennsylvania 15213 Abstract

More information

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

More information

Automatic parameter regulation for a tracking system with an auto-critical function

Automatic parameter regulation for a tracking system with an auto-critical function Automatic parameter regulation for a tracking system with an auto-critical function Daniela Hall INRIA Rhône-Alpes, St. Ismier, France Email: Daniela.Hall@inrialpes.fr Abstract In this article we propose

More information

DYNAMIC RANGE IMPROVEMENT THROUGH MULTIPLE EXPOSURES. Mark A. Robertson, Sean Borman, and Robert L. Stevenson

DYNAMIC RANGE IMPROVEMENT THROUGH MULTIPLE EXPOSURES. Mark A. Robertson, Sean Borman, and Robert L. Stevenson c 1999 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or

More information

Classification of Bad Accounts in Credit Card Industry

Classification of Bad Accounts in Credit Card Industry Classification of Bad Accounts in Credit Card Industry Chengwei Yuan December 12, 2014 Introduction Risk management is critical for a credit card company to survive in such competing industry. In addition

More information

CSC 411: Lecture 07: Multiclass Classification

CSC 411: Lecture 07: Multiclass Classification CSC 411: Lecture 07: Multiclass Classification Class based on Raquel Urtasun & Rich Zemel s lectures Sanja Fidler University of Toronto Feb 1, 2016 Urtasun, Zemel, Fidler (UofT) CSC 411: 07-Multiclass

More information

Natural Language Processing. Today. Logistic Regression Models. Lecture 13 10/6/2015. Jim Martin. Multinomial Logistic Regression

Natural Language Processing. Today. Logistic Regression Models. Lecture 13 10/6/2015. Jim Martin. Multinomial Logistic Regression Natural Language Processing Lecture 13 10/6/2015 Jim Martin Today Multinomial Logistic Regression Aka log-linear models or maximum entropy (maxent) Components of the model Learning the parameters 10/1/15

More information

The Role of Size Normalization on the Recognition Rate of Handwritten Numerals

The Role of Size Normalization on the Recognition Rate of Handwritten Numerals The Role of Size Normalization on the Recognition Rate of Handwritten Numerals Chun Lei He, Ping Zhang, Jianxiong Dong, Ching Y. Suen, Tien D. Bui Centre for Pattern Recognition and Machine Intelligence,

More information

COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS

COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS B.K. Mohan and S. N. Ladha Centre for Studies in Resources Engineering IIT

More information

Social Media Mining. Data Mining Essentials

Social Media Mining. Data Mining Essentials Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

More information

A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow

A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow , pp.233-237 http://dx.doi.org/10.14257/astl.2014.51.53 A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow Giwoo Kim 1, Hye-Youn Lim 1 and Dae-Seong Kang 1, 1 Department of electronices

More information

Feature Tracking and Optical Flow

Feature Tracking and Optical Flow 02/09/12 Feature Tracking and Optical Flow Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve

More information

Interactive Offline Tracking for Color Objects

Interactive Offline Tracking for Color Objects Interactive Offline Tracking for Color Objects Yichen Wei Jian Sun Xiaoou Tang Heung-Yeung Shum Microsoft Research Asia, Beijing, China {yichenw,jiansun,xitang,hshum}@microsoft.com Abstract In this paper,

More information

Semantic Recognition: Object Detection and Scene Segmentation

Semantic Recognition: Object Detection and Scene Segmentation Semantic Recognition: Object Detection and Scene Segmentation Xuming He xuming.he@nicta.com.au Computer Vision Research Group NICTA Robotic Vision Summer School 2015 Acknowledgement: Slides from Fei-Fei

More information

A Simple Feature Extraction Technique of a Pattern By Hopfield Network

A Simple Feature Extraction Technique of a Pattern By Hopfield Network A Simple Feature Extraction Technique of a Pattern By Hopfield Network A.Nag!, S. Biswas *, D. Sarkar *, P.P. Sarkar *, B. Gupta **! Academy of Technology, Hoogly - 722 *USIC, University of Kalyani, Kalyani

More information

Palmprint Recognition. By Sree Rama Murthy kora Praveen Verma Yashwant Kashyap

Palmprint Recognition. By Sree Rama Murthy kora Praveen Verma Yashwant Kashyap Palmprint Recognition By Sree Rama Murthy kora Praveen Verma Yashwant Kashyap Palm print Palm Patterns are utilized in many applications: 1. To correlate palm patterns with medical disorders, e.g. genetic

More information

AN IMPROVED DOUBLE CODING LOCAL BINARY PATTERN ALGORITHM FOR FACE RECOGNITION

AN IMPROVED DOUBLE CODING LOCAL BINARY PATTERN ALGORITHM FOR FACE RECOGNITION AN IMPROVED DOUBLE CODING LOCAL BINARY PATTERN ALGORITHM FOR FACE RECOGNITION Saurabh Asija 1, Rakesh Singh 2 1 Research Scholar (Computer Engineering Department), Punjabi University, Patiala. 2 Asst.

More information

Nonlinear Iterative Partial Least Squares Method

Nonlinear Iterative Partial Least Squares Method Numerical Methods for Determining Principal Component Analysis Abstract Factors Béchu, S., Richard-Plouet, M., Fernandez, V., Walton, J., and Fairley, N. (2016) Developments in numerical treatments for

More information

LIBSVX and Video Segmentation Evaluation

LIBSVX and Video Segmentation Evaluation CVPR 14 Tutorial! 1! LIBSVX and Video Segmentation Evaluation Chenliang Xu and Jason J. Corso!! Computer Science and Engineering! SUNY at Buffalo!! Electrical Engineering and Computer Science! University

More information