Introduction to Image Analysis

Transcription

1 1 Introduction to Image Analysis Image Processing vs Analysis So far we have been studying low level tasks which are processing algorithms. Image analysis concerns higher level tasks which are intended to understanding a scene. Consider a robot with a camera, navigating a room Segmentation Images Feature Extraction Classification Recognition Objects Features Object types Commands 1

2 3 Basics of Segmentation Detection of discontinuities in Graylevel Texture Shape Etc. 4 Graylevel Discontinuities: Edge Detection Edge Detection is accomplished mainly through the application of derivative-type filters, combined with some type of thresholding. Example: Sobel Edge Detector: f x f y 1 f x f y Compute Magnitudes, Add, Threshold Edge Image

3 5 Image gradient The gradient of an image: The gradient points in the direction of most rapid change in intensity The gradient direction is given by: how does this relate to the direction of the edge? The edge strength is given by the gradient magnitude 6 Edges: What are they? 3

4 7 Edges From First and Second Order Derivatives 8 Edges in Noise 4

5 5 9 LoG Edge Detector Derivative computation, especially second order, is sensitive to noise. To mitigate this sensitivity, we can first prefilter using a Gaussian low-pass. Laplacian: y f x f f or ) * ( ) * ( ) * ( y h f x h f h f )/ ( ), ( y x e y x h 10 Application of LoG ) *( )* ( ) * ( h f h f h f Linear Operations Laplacian of Gaussian

6 11 Effect of LoG on a Step Edge Convolve with Gaussian Take First Derivative Take Second Derivative Zero-crossing locates the edge 1 Example Original Sobel Edge LoG Image LoG Threshold LoG 0-Cross 6

7 Designing an edge detector Criteria for an optimal edge detector: Good detection: the optimal detector must minimize the probability of false positives (detecting spurious edges caused by noise), as well as that of false negatives (missing real edges) Good localization: the edges detected must be as close as possible to the true edges Single response: the detector must return one point only for each true edge point; that is, minimize the number of local maxima around the true edge Source: L. Fei-Fei Canny edge detector This is probably the most widely used edge detector Theoretical model: step-edges corrupted by additive Gaussian noise Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization MATLAB: edge(image, canny ) J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8: , Source: L. Fei-Fei 7

8 Canny edge detector 1. Filter image with derivative of Gaussian. Find magnitude and orientation of gradient 3. Non-maximum suppression Thin multi-pixel wide ridges down to single pixel width 4. Linking of edge points Source: D. Lowe, L. Fei-Fei Example original image (Lena) 8

9 Example norm of the gradient Example thresholding 9

10 Example Non-maximum suppression Non-maximum suppression At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values. Source: D. Forsyth 10

11 1 Suppression of Non-maxima Suppression of Non-maxima 11

12 Edge linking Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s). Source: D. Forsyth 4 Edge Linking Edge detection operators identify pixels individually as belonging to an edge or not Need to connect points thus identified to recover a proper edge. Suppose two nearby points are identified as edge pixels. Compare gradients. ( x 0, y ) 0 Magnitude Match f x, y ) f ( x, y ) ( T m ( 1 x 1, y ) Angular Match f x, y ) f ( x, y ) ( T a 1

13 5 Edge Linking II Keep a running list of points so linked together. For book-keeping, assign a different (fictitious) gray level to each linked set of points to keep account of different edges. Canny edge detector 1. Filter image with derivative of Gaussian. Find magnitude and orientation of gradient 3. Non-maximum suppression Thin multi-pixel wide ridges down to single pixel width 4. Linking of edge points Hysteresis thresholding: use a higher threshold to start edge curves and a lower threshold to continue them Source: D. Lowe, L. Fei-Fei 13

14 Hysteresis thresholding Use a high threshold to start edge curves and a low threshold to continue them Reduces drop-outs Source: S. Seitz Hysteresis thresholding original image high threshold (strong edges) low threshold (weak edges) hysteresis threshold Source: L. Fei-Fei 14

15 Effect of (Gaussian kernel spread/size) original Canny with Canny with The choice of depends on desired behavior large detects large scale edges small detects fine features Source: S. Seitz Edge detection is just the beginning image human segmentation gradient magnitude Berkeley segmentation database: 15

16 Fitting Curves to Points: Parameter space representation A line in the image corresponds to a point in Hough space Image space Hough parameter space Source: S. Seitz 16

17 Parameter space representation What does a point (x 0, y 0 ) in the image space map to in the Hough space? Image space Hough parameter space Parameter space representation What does a point (x 0, y 0 ) in the image space map to in the Hough space? Answer: the solutions of b = x 0 m + y 0 This is a line in Hough space Image space Hough parameter space 17

18 Parameter space representation Where is the line that contains both (x 0, y 0 ) and (x 1, y 1 )? Image space Hough parameter space (x 1, y 1 ) (x 0, y 0 ) b = x 1 m + y 1 Parameter space representation Where is the line that contains both (x 0, y 0 ) and (x 1, y 1 )? It is the intersection of the lines b = x 0 m + y 0 and b = x 1 m + y 1 Image space Hough parameter space (x 0, y 0 ) (x 1, y 1 ) b = x 1 m + y 1 18

19 Parameter space representation Problems with the (m,b) space: Unbounded parameter domain Vertical lines require infinite m Alternative: polar representation xcos y sin Each point will add a sinusoid in the (,) parameter space 38 Hough Transform Simplest case: Lines Fit a straight line to a set of edge pixels. Hough Transform (196): Pattern Matching Parameterization of a line in the plane: P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation,

20 39 Hough Transform Algorithm Subdivide the parameter plane into bins Accumulate the total number sinusoids that cross each bin Threshold the value of the bins in the parameter plane to declare the presence of lines Note the effect of the length of lines on the accumulated values Note similarity to the Radon Transform 40 Hough Transform Examples 0

21 41 Hough Transform Examples 4 Hough Transform Examples 1

22 43 Hough Transform Examples Effect of noise votes Peak gets fuzzy and hard to locate

23 Random points features votes Uniform noise can lead to spurious peaks in the array Dealing with noise Choose a good grid / discretization Too coarse: large votes obtained when too many different lines correspond to a single bucket Too fine: miss lines because some points that are not exactly collinear cast votes for different buckets Increment neighboring bins (smoothing in accumulator array) Try to get rid of irrelevant features Take only edge points with significant gradient magnitude 3

24 Incorporating image gradients Recall: when we detect an edge point, we also know its gradient direction But this means that the line is uniquely determined! Modified Hough transform: For each edge point (x,y) θ = gradient orientation at (x,y) ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end Hough transform for circles How many dimensions will the parameter space have? Given an oriented edge point, what are all possible bins that it can vote for? 4

25 Hough transform for circles y image space Hough parameter space r ( x, y) ri ( x, y) (x,y) ( x, y) ri ( x, y) x x y Hough transform for circles Conceptually equivalent procedure: for each (x,y,r), draw the corresponding circle in the image and compute its support r y x Is this more or less efficient than voting with features? 5

26 51 Parametric equation for a circle of (fixed) radius r Identify circle centers. Generalized Hough Transform: Circles Count the number of edge pixels on these circles. Change r, repeat. x r cos x y r sin y Example of Circle Detection 6

27 53 Segmentation by Thresholding Global Thresholding Local Thresholding 54 Some Statistical Analysis Histograms are almost never so nicely separated. Foreground Background Suppose each pixel belongs to foreground with probability a, and to background with probability 1-a In classifying any pixel with threshold T, we can make two types of errors. T 1( T) p( z dz T) T e ) Probability of misclassifying a background pixel as foreground e ( p1( z) dz Probability of misclassifying a foreground pixel as background 7

28 55 Optimal Thresholding Pick threshold T to minimize the overall expected probability of error. e T) (1 a) e ( T) ae ( ) ( 1 T e( T) (1 a) T (1 a) p T T p( z) dz a T ( T) ap ( T) 0 1 T p ( z) dz 1 p T a 1 1( ) p ( T) a Solve for threshold T Example: Two Gaussians with same variance and different means. m m a T 1 m1 m ln 1 a Special case: a=1/ m 1 m T 56 Morphological Processing 8

29 57 Operations on Sets of Points 58 Other Set Operations 9

30 59 Dilation 60 Dilation Example 30

31 61 Another Example of Dilation 6 Erosion (Opposite of Dilation) 31

32 63 Another Erosion Example 64 Closing and Opening 3

33 65 Closing Example 66 Opening Example 33

34 67 Morphological Filtering Example: Closing, followed by Opening 68 Morphological Boundary Detection 34