Seminar: Feature extraction by André Aichert I Feature detection - What is a feature? - Image processing essentials - Edge detection (Sobel & Canny) - Hough transform - Some images II An Entropy-based approach - Saliency - Entropy - Scale & self similarity - Outline: Kadir & Brady Algorithm
Seminar: Feature extraction What is a feature? feature = point of interest for image description Properties of a good feature - consistent over several images of the same scene - invariant towards certain transformations - insensitive to noise - salient If we define features by saliency, what is saliency then? - a feature should pop out to the eye - should be a local definition (-> transformations) - should have high complexity (->scales)
Seminar: Feature extraction Image processing essentials Reminder:Linear Filters - Project one image of intensities to another - Linear for every pixel position: weighted sum of original intensities - Usually from the neighborhood of the pixel location - Then, described fully by the weights used Convolution kernel filter - Use a matrix of weights - Figure: 3x3 neighborhood 3x3 matrix of 9 weights - Move the red window
Seminar: Edge detection Example: Sobel 1 0-1 2 0-2 1 0-1 1 2 1 0 0 0-1 -2-1
Seminar: Edge detection What is an edge? An edge is a location - of strong intentensity contrast - expressed as a fast change in intensity -> gradient - thin, long shape
Seminar: Sobel edge detection Derivation: Derivative along X Derivative in x-direction - Difference of consecutive samples - As a kernel - Sensitive to noise - Sobel uses a slightly more inaccurate method
Seminar: Sobel edge detection Derivation: Gaussian blur Gaussian kernels - Blurs images - Discrete zero-mean 2d-Gaussian function where (i,j) is a cell of the kernel - t is the strength of the blur - example 5x5 kernel for t=1
Seminar: Sobel edge detection Derivation: Gaussian blur
Seminar: Sobel edge detection Derivation: noise reduction Altogether... - Sobel blurs only in directions perpendicular to the axis for which the gradient is computed - For X-Edges blur Y-direction using - Concatenation with E X can be written in one kernel
Seminar: Sobel edge detection Complete filter Sobel edge detection is not a linear filter - We search maxima in gradient magnitude - Compute the gradient in both directions applying both S X, S Y =S X T - Then gradient (edge) direction and magnitude are - Edges can now be extracted by a simple threshold
Seminar: Sobel edge detection Gradient direction Corner: Region with more than one dominant edge direction
Seminar: Canny edge detection Overview Canny designed an optimal edge detector - Sobel does not always extract thin edges - Sobel is sensitive to around a scale of 3 pixels - A single threshold can lead to dashed edges - Canny: two step algorithm 1. Extract gradient using a gaussian derivative 2. Suppress non-maxima using two thresholds For 1. methods similar to or Sobel itself can be used The 2. part makes the diffference
Seminar: Canny edge detection II: Edge tracking
Seminar: Canny edge detection II: Edge tracking Non maxima suppression - For every peak in gradient magnitude >T1 follow the edge using the gradient direction mark the pixel an edge and suppress neighbors stop tracking when grad. magnitude < T2 - Y-junctions impossible (can be fixed) - Canny's parameters: scale, T1, T2
Seminar: Hough Transform Idea Hough transformation - Transformation of the image to a parameter space - Parameters of any particular shape (eg. lines) - Peaks in Hough space are features of that shape
Seminar: Hough Transform Algorithm for Lines - Use parametric representation of lines - r, represents the line cos x sin y=r - Use the aforementioned image filters to compute gradient direction (eg. Sobel) extract edges (eg. Canny) - Set up boxes for results (counter and r, ) decides upon number of lines that differ too many boxes result in weak peaks too few degrade accuracy - Fit a line by edge pixels' direction - Choose a box and increase it's counter - Boxes with max. counter contain resulting lines
Seminar: Hough Transform Example images
Seminar: Entropy Connection to Saliency An entropy and scale based approach to saliency - Salient means descrptive, thus complex - Entropy is a measure for signal complexity - Use a descriptor D, such as intensities of 8bit image - Then d i are intensity values 0... 255 - R X a region around X (image patch) - P D, R X d i the probability that in a random pixel - Computed through a histogram of then R X D=d i
Seminar: Entropy Educated Guesses
Seminar: Entropy Connection to Scale R X is a matter of scale: - Usually a circular or rectangular patch - Size has a relevant influence on the entropy - There is not a unique scale for one image Solution: scale space - An array of differently scaled images as a basis - Additional parameter t representing scale - Compare: Canny -> Relation between scales is a gaussian blur
Seminar: Scale Example scales
Seminar: Scale Feature Selection What features are better? Small or big? - Different scales, different features - Assign a detected feature a scale - Problem: self similarity: Accuracy of detection, likeliness of mismatch - Example: hair, tree (, fractals) We want self-dissimilar features: images: absolute difference over many scales
Seminar: Kadir and Brady Algorithm Improvement: weighting Kadir and Brady feature detection - Scale and entropy based - Selects scales of peaked entropy and - Weights by a measure of self-similarity - Quality of a feature: product of entropy and weight - Weight: rate of change for varying scale
Seminar: Kadir and Brady Algorithm Essential Steps 1. For each pixel location x compute the local entropy of descriptor D(using the PDF from the histogram) 2. Select the scales S where entropy is peaked (eg. zero-crossings of derivative, can be empty) 3. Weight the entropy for the scales in S by the sum of absulute difference of the local descriptor around S Note that the local Descriptor is quite a general concept. We used image intensity. Other Descriptions yeild other etropy, thus other features.
Seminar: Kadir and Brady Algorithm Results (top: Gilles)
Seminar: Feature extraction by André Aichert Note that there is a paper along with this presentation. Please refer to the paper's sources.