# MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos

Save this PDF as:

Size: px
Start display at page:

Download "MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr"

## Transcription

1 Machine Learning for Computer Vision 1 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Department of Applied Mathematics Ecole Centrale Paris Galen Group INRIA-Saclay

2 Administrative details 2 Make sure you have all received my from last week

3 Lecture outline 3 Recap & problems of linear regression Logistic Regression Introduction to Multiple Instance Learning

4 4 Learning problem Recover input-output mapping Output: y Input: x Method: f Parameters: w Aspects of the learning problem Identify methods that fit the problem setting Determine parameters that properly classify the training set Measure and control the complexity of these functions

5 Linear Classifiers Find linear expression (hyperplane) to separate positive and negative examples 5 Feature coordinate j x x i i positive : negative : x x i i w + b w + b 0 < 0 Each data point has a class label: +1 ( ) y t = -1 ( ) Feature coordinate i

6 Learning problem formulation 6 Given: Training set of feature-label pairs Wanted: simple that works well for simple: penalizing function complexity, to guarantee generalization works well: quantified by loss criterion

7 Linear regression 7 Linear Loss function: quantify appropriateness of for Introduce vector notation

8 Inappropriateness of quadratic loss We chose the quadratic cost function for convenience Single, global minimum & closed form expression 8 But does it indicate classification performance? Computed Decision Boundary Linear Fit Desired decision boundary

9 Inappropriateness of quadratic loss Consider transformation: 9 Quadratic loss is not robust to outliers and penalizes outputs that are `too good

10 Lecture outline 10 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

11 Probabilistic interpretation of least squares 11 Outputs: continuous random variables y = hx, wi +, N(0, ) linear function of inputs + zero mean Gaussian r.v. Optimal w: maximizes likelihood of observed outputs But the labels are discrete!

12 A probabilistic criterion for training a classifier 12 Training set: y: discrete observations: model as samples from Bernoulli distribution Find w that maximizes the likelihood of labels in the training set

13 Sigmoidal function & logistic regression 13 sigmoidal Training criterion from previous slide: How do we optimize it with respect to w? What does it mean?

14 Lecture outline 14 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

16 Maximization with Newton-Raphson 16 Newton method for finding roots of 1D function : Newton method for finding maxima of 1D function : condition Newton-Raphson method for finding maxima of N-D function :

17 Newton-Raphson for Logistic Regression 17 Gradient: Hessian: In statistics: Iteratively Reweighted Least Squares (IRLS)

18 Lecture outline 18 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Support Vector Machines

19 A compact expression for the loss 19 Using, :

20 Log loss: 20

21 Log loss vs. quadratic loss 21

22 Logistic vs Linear Regression 22 Logistic regression is more robust Linear Regression Logistic Regression

23 Lecture outline 23 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

24 Boundary detection problem Object/Surface Boundaries 24

25 25 Signal-level challenges Poor contrast Shadows Texture

26 Machine Learning for Computer Vision Lecture 2 Fundamental challenges: can humans do it? 26

27 Learning-based approaches 27 Boundary or non-boundary? Use human-annotated segmentations Use any visual cue as input to the decision function. Use decision trees/logisitic regression/boosting/ and learn to combine the individual inputs. S. Konishi, A.Yuille, J. Coughlan, S.C. Zhu, Statistical Edge Detection: Learning and Evaluating Edge Cues, IEEE PAMI, 2003 D. Martin, C. Fowlkes, J. Malik. "Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues", IEEE PAMI, 2004

28 Evaluation protocol 28 Threshold detector s output at certain level Match outputs to human ground-truth Count true/false positives (t/f)positives, misses (m) From precision, p and recall, r Quantify performance in terms of F-measure

29 29 Contours can be defined by any of a number of cues (P. Cavanagh) Slide credit: J. Malik, M. Maire

30 Cue-localization Gray level photographs 30 Objects from motion Objects from luminance Objects from disparity Line drawings Objects from texture Slide credit: J. Malik, M. Maire Grill-Spector et al., Neuron 1998

31 31 Local boundary cues Separate features per candidate orientation In specific: r (x,y) θ 1976 CIE L*a*b* colorspace Brightness Gradient BG(x,y,r,θ) Difference of L* distributions Color Gradient CG(x,y,r,θ) Difference of a*b* distributions Texture Gradient TG(x,y,r,θ) Difference of distributions of V1-like filter responses Slide credit: J. Malik, M. Maire

32 Boundary cues (horizontal) 32 Brightness Color (a, b channels) Texture

33 Boundary π/4 33 Brightness Color (a, b channels) Texture

34 Brightness π/2 34 Brightness Color (a, b channels) Texture

35 Brightness 3π/4 35 Brightness Color (a, b channels) Texture

36 Cue combinations with logistic regression 36 Slide credit: J. Malik, M. Maire

37 Exploiting global constraints: image Segmentation as Graph Partitioning Build a weighted graph G=(V,E) from image V: image pixels 37 E: connections between pairs of nearby pixels Partition graph so that similarity within group is large and similarity between groups is small -- Normalized Cuts [Shi & Malik 97] Slide credit: J. Malik, M. Maire

38 38 Wij small when intervening contour strong, strong when weak.. Slide credit: J. Malik, M. Maire

39 Normalized Cuts as a Spring-Mass system 39 Each pixel is a point mass; each connection is a spring: ( D W ) y = λdy Fundamental modes are generalized eigenvectors of (D - W) x = λdx Slide credit: J. Malik, M. Maire

40 Contour information from eigenvectors 40 Slide credit: J. Malik, M. Maire

41 Classifier 41 Logistic regression

42 Progress during the last 40 years 42 Humans Berkeley gpb, 08 Berkeley PB, 04 Canny+ Hysteresis ( 85) Prewitt, good feature= 5 years of work

43 Lecture outline 43 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

44 Learning boundary detection 44 Problem I: inconsistent orientation information Problem II: inconsistent location information

45 Sneaking into the fun room 45 mom s keychain grandma s keychain dad s keychain We know that dad cannot enter the fun room, either Which key should we try? Slide Credit: B. Babenko/T. Dietterich

46 Multiple Instance Learning 46 Typical Learning Multiple Instance Learning Positive bag: at least one instance should be positive Negative bag: no instance should be positive Slide Credit: K. Grauman

47 Noisy-or classifier combination N classifier responses, {p 1,...,p N } p i : probability of positive label (classifier i) 1 p i : probability of negative (classifier i) Negative bag: all instance classifiers are negative NY (1 p i ) p =1 i=1 Probability of bag being positive: NY (1 p i ) i=1 47

48 Progress during the last decade Precision : Humans 0.750: Fused (ours + others) 0.741: MIL detector (ours) 0.739: Sparse Code Gradients 0.728: Sketch Tokens 0.714: GlobalPb Recall Precision-Recall Curves on the Berkeley Benchmark

49 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 49

50 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 50

51 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 51

52 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 52

53 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold gpb 53

54 Machine Learning for Computer Vision Lecture 2 Comparisons with optimal threshold Our results 54

55 Machine Learning for Computer Vision Lecture 2 Learning symmetry detection S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV

56 Multi-scale and multi-orientation features 56 S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

57 Algorithm pipeline 57 Input image P(x,y) Features Brightness Texture Color Spectra l P(x,y) Training Multiple Instance Learning (MIL) non-maximum suppression softmax θ,s (P(x,y,θ,s)) 14x1 weight vector S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

58 58 Feature extraction Color: CIE Lab color space. Texture: texton map. Hard binning (32 bins for 3 color channels, 64 textons). Rectangle filters extract features at multiple scales and orientations. Differences of histograms ( gradients ) of color and texture content à symmetry indication. χ 2 distance à dissimilarity of feature content between adjacent rectangles. Integral images for fast extraction. S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

59 MIL Training 59 Input image θ = 22.5 θ = 45 Varying angle θ = 90 θ = 135 Varying scale θ = MIL training s = 4 s = 8 w w w =! w s = 12 s = 24 s = 16 Probability map S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012

60 60 Machine Learning for Computer Vision Lecture 2 Results Lindeberg Levinshtein This work State-of-the-art performance Large boost from color, texture and spectral cues. Long contours (region proposals) S. Tsogkas & I.K., Learning-based symmetry detection in natural images, ECCV 2012 Ground-truth

61 61 Machine Learning for Computer Vision Lecture 2 Some more results Lindeberg Levinshtein This work Ground-truth

62 Lecture recap 62 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL

63 Logistic regression training cost 63 Log loss Quadratic loss

64 Lecture recap 64 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL

65 Maximization with Newton-Raphson 65 Newton-Raphson method for finding maxima of N-D function: Condition for maximum: Jacobian: Hessian:

66 Lecture recap 66 Logistic Regression Training criterion Optimization Application to boundary detection Introduction to MIL

67 Progress during the last 40 years 67 Humans Berkeley gpb, 08 Berkeley PB, 04 Canny+ Hysteresis ( 85) Prewitt, good feature= 5 years of work

68 Lecture outline 68 Recap & problems of linear regression Logistic Regression Training criterion formulation Optimization Interpretation Application to boundary detection Introduction to Multiple Instance Learning

69 Appendix 69 Newton Raphson = Iteratively Reweighted Least Squares Masking problem in multi-class linear regression & softmax

70 Newton Raphson = Iteratively Reweighted Least Squares 70 Newton Raphson: Hessian: Rewrite Update: Transform: Weighted Least Squares Fit:

71 Multiple classes & linear regression 71 K classes: one-of-k coding 4 classes, i-th sample is in 3 rd class: Matrix notation: Loss function: Least squares fit:

72 Multiple classes & linear regression 72 One linear discriminant per class: Problem: ambiguous regions Solution: assign to discriminant with largest score

73 Masking Problem in linear regression Class 1 Class 2 Class 3 73 Nothing ever gets assigned to class 2! 2D version:

74 Multiple classes & logistic regression 74 Soft maximum of competing classes: Discriminants (inputs) Softmax (outputs) Label probability: Similar steps for parameter estimation (Bishop s book)

75 Logistic vs Linear Regression, n>2 classes 75 Linear regression Logistic regression Logistic regression does not exhibit the masking problem

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

### Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/

Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters

### CS 688 Pattern Recognition Lecture 4. Linear Models for Classification

CS 688 Pattern Recognition Lecture 4 Linear Models for Classification Probabilistic generative models Probabilistic discriminative models 1 Generative Approach ( x ) p C k p( C k ) Ck p ( ) ( x Ck ) p(

### Linear Threshold Units

Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

### Segmentation & Clustering

EECS 442 Computer vision Segmentation & Clustering Segmentation in human vision K-mean clustering Mean-shift Graph-cut Reading: Chapters 14 [FP] Some slides of this lectures are courtesy of prof F. Li,

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

### Pa8ern Recogni6on. and Machine Learning. Chapter 4: Linear Models for Classiﬁca6on

Pa8ern Recogni6on and Machine Learning Chapter 4: Linear Models for Classiﬁca6on Represen'ng the target values for classifica'on If there are only two classes, we typically use a single real valued output

### Multi-Scale Improves Boundary Detection in Natural Images

Multi-Scale Improves Boundary Detection in Natural Images Xiaofeng Ren Toyota Technological Institute at Chicago 1427 E. 60 Street, Chicago, IL 60637, USA xren@tti-c.org Abstract. In this work we empirically

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Lecture 8 February 4

ICS273A: Machine Learning Winter 2008 Lecture 8 February 4 Scribe: Carlos Agell (Student) Lecturer: Deva Ramanan 8.1 Neural Nets 8.1.1 Logistic Regression Recall the logistic function: g(x) = 1 1 + e θt

### Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches

Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu Problematic

### Logistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression

Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max

### These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop

Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Local features and matching. Image classification & object localization

Overview Instance level search Local features and matching Efficient visual recognition Image classification & object localization Category recognition Image classification: assigning a class label to

### Practical Tour of Visual tracking. David Fleet and Allan Jepson January, 2006

Practical Tour of Visual tracking David Fleet and Allan Jepson January, 2006 Designing a Visual Tracker: What is the state? pose and motion (position, velocity, acceleration, ) shape (size, deformation,

### Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 269 Class Project Report

Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 69 Class Project Report Junhua Mao and Lunbo Xu University of California, Los Angeles mjhustc@ucla.edu and lunbo

### Probabilistic Linear Classification: Logistic Regression. Piyush Rai IIT Kanpur

Probabilistic Linear Classification: Logistic Regression Piyush Rai IIT Kanpur Probabilistic Machine Learning (CS772A) Jan 18, 2016 Probabilistic Machine Learning (CS772A) Probabilistic Linear Classification:

### Logistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld.

Logistic Regression Vibhav Gogate The University of Texas at Dallas Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Generative vs. Discriminative Classifiers Want to Learn: h:x Y X features

### Canny Edge Detection

Canny Edge Detection 09gr820 March 23, 2009 1 Introduction The purpose of edge detection in general is to significantly reduce the amount of data in an image, while preserving the structural properties

### Logistic Regression (1/24/13)

STA63/CBB540: Statistical methods in computational biology Logistic Regression (/24/3) Lecturer: Barbara Engelhardt Scribe: Dinesh Manandhar Introduction Logistic regression is model for regression used

### Segmentation. Lecture 12. Many slides from: S. Lazebnik, K. Grauman and P. Kumar

Segmentation Lecture 12 Many slides from: S. Lazebnik, K. Grauman and P. Kumar Image Segmentation Image segmentation The goals of segmentation Group together similar-looking pixels for efficiency of further

### Computer Vision - part II

Computer Vision - part II Review of main parts of Section B of the course School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture Name Course Name 1 1 2

### Introduction to Machine Learning

Introduction to Machine Learning Prof. Alexander Ihler Prof. Max Welling icamp Tutorial July 22 What is machine learning? The ability of a machine to improve its performance based on previous results:

### CSCI567 Machine Learning (Fall 2014)

CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 22, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 22, 2014 1 /

### Image Segmentation and Registration

Image Segmentation and Registration Dr. Christine Tanner (tanner@vision.ee.ethz.ch) Computer Vision Laboratory, ETH Zürich Dr. Verena Kaynig, Machine Learning Laboratory, ETH Zürich Outline Segmentation

### Machine Learning and Pattern Recognition Logistic Regression

Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,

### Edge detection. (Trucco, Chapt 4 AND Jain et al., Chapt 5) -Edges are significant local changes of intensity in an image.

Edge detection (Trucco, Chapt 4 AND Jain et al., Chapt 5) Definition of edges -Edges are significant local changes of intensity in an image. -Edges typically occur on the boundary between two different

### CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

### Tracking Algorithms. Lecture17: Stochastic Tracking. Joint Probability and Graphical Model. Probabilistic Tracking

Tracking Algorithms (2015S) Lecture17: Stochastic Tracking Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Deterministic methods Given input video and current state, tracking result is always same. Local

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### Predict Influencers in the Social Network

Predict Influencers in the Social Network Ruishan Liu, Yang Zhao and Liuyu Zhou Email: rliu2, yzhao2, lyzhou@stanford.edu Department of Electrical Engineering, Stanford University Abstract Given two persons

### Christfried Webers. Canberra February June 2015

c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic

### The Artificial Prediction Market

The Artificial Prediction Market Adrian Barbu Department of Statistics Florida State University Joint work with Nathan Lay, Siemens Corporate Research 1 Overview Main Contributions A mathematical theory

### Assessment. Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall

Automatic Photo Quality Assessment Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall Estimating i the photorealism of images: Distinguishing i i paintings from photographs h Florin

### DATA ANALYTICS USING R

DATA ANALYTICS USING R Duration: 90 Hours Intended audience and scope: The course is targeted at fresh engineers, practicing engineers and scientists who are interested in learning and understanding data

### Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi

Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector

### Environmental Remote Sensing GEOG 2021

Environmental Remote Sensing GEOG 2021 Lecture 4 Image classification 2 Purpose categorising data data abstraction / simplification data interpretation mapping for land cover mapping use land cover class

### Machine Learning and Data Mining. Regression Problem. (adapted from) Prof. Alexander Ihler

Machine Learning and Data Mining Regression Problem (adapted from) Prof. Alexander Ihler Overview Regression Problem Definition and define parameters ϴ. Prediction using ϴ as parameters Measure the error

### Lecture 6: Classification & Localization. boris. ginzburg@intel.com

Lecture 6: Classification & Localization boris. ginzburg@intel.com 1 Agenda ILSVRC 2014 Overfeat: integrated classification, localization, and detection Classification with Localization Detection. 2 ILSVRC-2014

### Introduction to Segmentation

Lecture 2: Introduction to Segmentation Jonathan Krause 1 Goal Goal: Identify groups of pixels that go together image credit: Steve Seitz, Kristen Grauman 2 Types of Segmentation Semantic Segmentation:

### Robert Collins CSE598G. More on Mean-shift. R.Collins, CSE, PSU CSE598G Spring 2006

More on Mean-shift R.Collins, CSE, PSU Spring 2006 Recall: Kernel Density Estimation Given a set of data samples x i ; i=1...n Convolve with a kernel function H to generate a smooth function f(x) Equivalent

### Pattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University

Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision

### Linear Models for Classification

Linear Models for Classification Sumeet Agarwal, EEL709 (Most figures from Bishop, PRML) Approaches to classification Discriminant function: Directly assigns each data point x to a particular class Ci

### THE development of methods for automatic detection

Learning to Detect Objects in Images via a Sparse, Part-Based Representation Shivani Agarwal, Aatif Awan and Dan Roth, Member, IEEE Computer Society 1 Abstract We study the problem of detecting objects

### Supervised Learning (Big Data Analytics)

Supervised Learning (Big Data Analytics) Vibhav Gogate Department of Computer Science The University of Texas at Dallas Practical advice Goal of Big Data Analytics Uncover patterns in Data. Can be used

### Linear Classification. Volker Tresp Summer 2015

Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong

### Determining optimal window size for texture feature extraction methods

IX Spanish Symposium on Pattern Recognition and Image Analysis, Castellon, Spain, May 2001, vol.2, 237-242, ISBN: 84-8021-351-5. Determining optimal window size for texture feature extraction methods Domènec

### 3F3: Signal and Pattern Processing

3F3: Signal and Pattern Processing Lecture 3: Classification Zoubin Ghahramani zoubin@eng.cam.ac.uk Department of Engineering University of Cambridge Lent Term Classification We will represent data by

### Image Gradients. Given a discrete image Á Òµ, consider the smoothed continuous image Üµ defined by

Image Gradients Given a discrete image Á Òµ, consider the smoothed continuous image Üµ defined by Üµ Ü ¾ Ö µ Á Òµ Ü ¾ Ö µá µ (1) where Ü ¾ Ö Ô µ Ü ¾ Ý ¾. ½ ¾ ¾ Ö ¾ Ü ¾ ¾ Ö. Here Ü is the 2-norm for the

### Classifying Chess Positions

Classifying Chess Positions Christopher De Sa December 14, 2012 Chess was one of the first problems studied by the AI community. While currently, chessplaying programs perform very well using primarily

### Machine Learning Logistic Regression

Machine Learning Logistic Regression Jeff Howbert Introduction to Machine Learning Winter 2012 1 Logistic regression Name is somewhat misleading. Really a technique for classification, not regression.

### A New Robust Algorithm for Video Text Extraction

A New Robust Algorithm for Video Text Extraction Pattern Recognition, vol. 36, no. 6, June 2003 Edward K. Wong and Minya Chen School of Electrical Engineering and Computer Science Kyungpook National Univ.

### Ensemble Methods. Knowledge Discovery and Data Mining 2 (VU) (707.004) Roman Kern. KTI, TU Graz 2015-03-05

Ensemble Methods Knowledge Discovery and Data Mining 2 (VU) (707004) Roman Kern KTI, TU Graz 2015-03-05 Roman Kern (KTI, TU Graz) Ensemble Methods 2015-03-05 1 / 38 Outline 1 Introduction 2 Classification

### Introduction to Logistic Regression

OpenStax-CNX module: m42090 1 Introduction to Logistic Regression Dan Calderon This work is produced by OpenStax-CNX and licensed under the Creative Commons Attribution License 3.0 Abstract Gives introduction

### Introduction to Machine Learning Using Python. Vikram Kamath

Introduction to Machine Learning Using Python Vikram Kamath Contents: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Introduction/Definition Where and Why ML is used Types of Learning Supervised Learning Linear Regression

### CSC 321 H1S Study Guide (Last update: April 3, 2016) Winter 2016

1. Suppose our training set and test set are the same. Why would this be a problem? 2. Why is it necessary to have both a test set and a validation set? 3. Images are generally represented as n m 3 arrays,

### Improving Web-based Image Search via Content Based Clustering

Improving Web-based Image Search via Content Based Clustering Nadav Ben-Haim, Boris Babenko and Serge Belongie Department of Computer Science and Engineering University of California, San Diego {nbenhaim,bbabenko,sjb}@ucsd.edu

### Decision Trees from large Databases: SLIQ

Decision Trees from large Databases: SLIQ C4.5 often iterates over the training set How often? If the training set does not fit into main memory, swapping makes C4.5 unpractical! SLIQ: Sort the values

### Azure Machine Learning, SQL Data Mining and R

Azure Machine Learning, SQL Data Mining and R Day-by-day Agenda Prerequisites No formal prerequisites. Basic knowledge of SQL Server Data Tools, Excel and any analytical experience helps. Best of all:

### Colour Image Segmentation Technique for Screen Printing

60 R.U. Hewage and D.U.J. Sonnadara Department of Physics, University of Colombo, Sri Lanka ABSTRACT Screen-printing is an industry with a large number of applications ranging from printing mobile phone

### MACHINE LEARNING IN HIGH ENERGY PHYSICS

MACHINE LEARNING IN HIGH ENERGY PHYSICS LECTURE #1 Alex Rogozhnikov, 2015 INTRO NOTES 4 days two lectures, two practice seminars every day this is introductory track to machine learning kaggle competition!

### Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications

Data Mining In Modern Astronomy Sky Surveys: Concepts in Machine Learning, Unsupervised Learning & Astronomy Applications Ching-Wa Yip cwyip@pha.jhu.edu; Bloomberg 518 Human are Great Pattern Recognizers

### GLM, insurance pricing & big data: paying attention to convergence issues.

GLM, insurance pricing & big data: paying attention to convergence issues. Michaël NOACK - michael.noack@addactis.com Senior consultant & Manager of ADDACTIS Pricing Copyright 2014 ADDACTIS Worldwide.

### 521466S Machine Vision Assignment #7 Hough transform

521466S Machine Vision Assignment #7 Hough transform Spring 2014 In this assignment we use the hough transform to extract lines from images. We use the standard (r, θ) parametrization of lines, lter the

### Analecta Vol. 8, No. 2 ISSN 2064-7964

EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,

### A Simple Introduction to Support Vector Machines

A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University Outline A brief history of SVM Large-margin linear

### Lecture 2: The SVM classifier

Lecture 2: The SVM classifier C19 Machine Learning Hilary 2015 A. Zisserman Review of linear classifiers Linear separability Perceptron Support Vector Machine (SVM) classifier Wide margin Cost function

### Class #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris

Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines

### Probabilistic Latent Semantic Analysis (plsa)

Probabilistic Latent Semantic Analysis (plsa) SS 2008 Bayesian Networks Multimedia Computing, Universität Augsburg Rainer.Lienhart@informatik.uni-augsburg.de www.multimedia-computing.{de,org} References

### Practical Data Science with Azure Machine Learning, SQL Data Mining, and R

Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Overview This 4-day class is the first of the two data science courses taught by Rafal Lukawiecki. Some of the topics will be

### Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

### Introduction to Deep Learning Variational Inference, Mean Field Theory

Introduction to Deep Learning Variational Inference, Mean Field Theory 1 Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Center for Visual Computing Ecole Centrale Paris Galen Group INRIA-Saclay Lecture 3: recap

### Mean-Shift Tracking with Random Sampling

1 Mean-Shift Tracking with Random Sampling Alex Po Leung, Shaogang Gong Department of Computer Science Queen Mary, University of London, London, E1 4NS Abstract In this work, boosting the efficiency of

### Lecture 6: Logistic Regression

Lecture 6: CS 194-10, Fall 2011 Laurent El Ghaoui EECS Department UC Berkeley September 13, 2011 Outline Outline Classification task Data : X = [x 1,..., x m]: a n m matrix of data points in R n. y { 1,

### Simple and efficient online algorithms for real world applications

Simple and efficient online algorithms for real world applications Università degli Studi di Milano Milano, Italy Talk @ Centro de Visión por Computador Something about me PhD in Robotics at LIRA-Lab,

### QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING

QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING MRS. A H. TIRMARE 1, MS.R.N.KULKARNI 2, MR. A R. BHOSALE 3 MR. C.S. MORE 4 MR.A.G.NIMBALKAR 5 1, 2 Assistant professor Bharati Vidyapeeth s college

### Online Learning for Fast Segmentation of Moving Objects

Online Learning for Fast Segmentation of Moving Objects Liam Ellis, Vasileios Zografos {liam.ellis,vasileios.zografos}@liu.se CVL, Linköping University, Linköping, Sweden Abstract. This work addresses

### Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection

CSED703R: Deep Learning for Visual Recognition (206S) Lecture 6: CNNs for Detection, Tracking, and Segmentation Object Detection Bohyung Han Computer Vision Lab. bhhan@postech.ac.kr 2 3 Object detection

### Data Mining. Nonlinear Classification

Data Mining Unit # 6 Sajjad Haider Fall 2014 1 Nonlinear Classification Classes may not be separable by a linear boundary Suppose we randomly generate a data set as follows: X has range between 0 to 15

### Recognizing Cats and Dogs with Shape and Appearance based Models. Group Member: Chu Wang, Landu Jiang

Recognizing Cats and Dogs with Shape and Appearance based Models Group Member: Chu Wang, Landu Jiang Abstract Recognizing cats and dogs from images is a challenging competition raised by Kaggle platform

### An Iterative Image Registration Technique with an Application to Stereo Vision

An Iterative Image Registration Technique with an Application to Stereo Vision Bruce D. Lucas Takeo Kanade Computer Science Department Carnegie-Mellon University Pittsburgh, Pennsylvania 15213 Abstract

### Introduction to Machine Learning and Data Mining. Prof. Dr. Igor Trajkovski trajkovski@nyus.edu.mk

Introduction to Machine Learning and Data Mining Prof. Dr. Igor Trakovski trakovski@nyus.edu.mk Neural Networks 2 Neural Networks Analogy to biological neural systems, the most robust learning systems

### Automatic parameter regulation for a tracking system with an auto-critical function

Automatic parameter regulation for a tracking system with an auto-critical function Daniela Hall INRIA Rhône-Alpes, St. Ismier, France Email: Daniela.Hall@inrialpes.fr Abstract In this article we propose

### DYNAMIC RANGE IMPROVEMENT THROUGH MULTIPLE EXPOSURES. Mark A. Robertson, Sean Borman, and Robert L. Stevenson

c 1999 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or

### Classification of Bad Accounts in Credit Card Industry

Classification of Bad Accounts in Credit Card Industry Chengwei Yuan December 12, 2014 Introduction Risk management is critical for a credit card company to survive in such competing industry. In addition

### CSC 411: Lecture 07: Multiclass Classification

CSC 411: Lecture 07: Multiclass Classification Class based on Raquel Urtasun & Rich Zemel s lectures Sanja Fidler University of Toronto Feb 1, 2016 Urtasun, Zemel, Fidler (UofT) CSC 411: 07-Multiclass

### Natural Language Processing. Today. Logistic Regression Models. Lecture 13 10/6/2015. Jim Martin. Multinomial Logistic Regression

Natural Language Processing Lecture 13 10/6/2015 Jim Martin Today Multinomial Logistic Regression Aka log-linear models or maximum entropy (maxent) Components of the model Learning the parameters 10/1/15

### The Role of Size Normalization on the Recognition Rate of Handwritten Numerals

The Role of Size Normalization on the Recognition Rate of Handwritten Numerals Chun Lei He, Ping Zhang, Jianxiong Dong, Ching Y. Suen, Tien D. Bui Centre for Pattern Recognition and Machine Intelligence,

### COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS

COMPARISON OF OBJECT BASED AND PIXEL BASED CLASSIFICATION OF HIGH RESOLUTION SATELLITE IMAGES USING ARTIFICIAL NEURAL NETWORKS B.K. Mohan and S. N. Ladha Centre for Studies in Resources Engineering IIT

### Social Media Mining. Data Mining Essentials

Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

### A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow

, pp.233-237 http://dx.doi.org/10.14257/astl.2014.51.53 A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow Giwoo Kim 1, Hye-Youn Lim 1 and Dae-Seong Kang 1, 1 Department of electronices

### Feature Tracking and Optical Flow

02/09/12 Feature Tracking and Optical Flow Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve

### Interactive Offline Tracking for Color Objects

Interactive Offline Tracking for Color Objects Yichen Wei Jian Sun Xiaoou Tang Heung-Yeung Shum Microsoft Research Asia, Beijing, China {yichenw,jiansun,xitang,hshum}@microsoft.com Abstract In this paper,

### Semantic Recognition: Object Detection and Scene Segmentation

Semantic Recognition: Object Detection and Scene Segmentation Xuming He xuming.he@nicta.com.au Computer Vision Research Group NICTA Robotic Vision Summer School 2015 Acknowledgement: Slides from Fei-Fei

### A Simple Feature Extraction Technique of a Pattern By Hopfield Network

A Simple Feature Extraction Technique of a Pattern By Hopfield Network A.Nag!, S. Biswas *, D. Sarkar *, P.P. Sarkar *, B. Gupta **! Academy of Technology, Hoogly - 722 *USIC, University of Kalyani, Kalyani

### Palmprint Recognition. By Sree Rama Murthy kora Praveen Verma Yashwant Kashyap

Palmprint Recognition By Sree Rama Murthy kora Praveen Verma Yashwant Kashyap Palm print Palm Patterns are utilized in many applications: 1. To correlate palm patterns with medical disorders, e.g. genetic

### AN IMPROVED DOUBLE CODING LOCAL BINARY PATTERN ALGORITHM FOR FACE RECOGNITION

AN IMPROVED DOUBLE CODING LOCAL BINARY PATTERN ALGORITHM FOR FACE RECOGNITION Saurabh Asija 1, Rakesh Singh 2 1 Research Scholar (Computer Engineering Department), Punjabi University, Patiala. 2 Asst.