Image Gradients Given a discrete image Á Òµ, consider the smoothed continuous image ܵ defined by ܵ Ü ¾ Ö µ Á Òµ Ü ¾ Ö µá µ (1) where Ü ¾ Ö Ô µ Ü ¾ Ý ¾. ½ ¾ ¾ Ö ¾ Ü ¾ ¾ Ö. Here Ü is the 2-norm for the vector Ü Ü Ýµ Ì. That is, Ü Note that ܵ does not satisfy the interpolation conditions ¼µ ½ and Òµ ¼ for integer valued Ò ¼. Therefore ܵ does not in general interpolate the original discrete image Á Òµ (i.e., generally Òµ Á Òµ for integer valued image coordinates Ò). Instead ܵ provides a smoothed approximation of the image Á Òµ at Ü Ò. The gradient of a smooth image ܵ is defined to be the vector of partial derivatives, Ö Üµ ܵ Ü Ý ÜµµÌ (2) By differentiating inside the sum in (1) we find Ü Üµ Ü Ü µá µ (3) Ý Üµ Ý Ü µá µ (4) Note the derivative of a 2D Gaussian is the separable product of a 1D Gaussian times the derivative of a 1D Gaussian, as in Ü Ü ¾ Ü Ö µ ¾ Ü ¾ Ö µ Ö Ü ¾ Ö Ü ¾ Ö µ Ý ¾ Ö µ where Ü ¾ Ö µ Ô ½ ¾Ö ܾ ¾Öµ ¾ is a one-dimensional Gaussian. Therefore separable convolution can be used to compute the image gradient according to equations (2), (3) and (4) above. Implementation Details. Typically the radius à of the discrete filter support is taken to be à Ö. This gives 1D filter kernels of length ¾Ã ½. Moreover, in order to avoid strong discretization artifacts in sampling the Gaussian, typically Ö ½ is used. The smallest gradient filters of this type are therefore ½ and ½, which are used for Ö ½(see cannytutorial.m). 320: Edge Detection Notes: 12
Properties of Gradients What does the image gradient Ö Üµ tell us about the local image brightness? To understand this, consider the directional derivative of the image at Ü in the direction Ù Ù ½ Ù ¾ µ Ì,definedby ٠ܵ Ø Ü Ùص ؼ Ü Ü Ùص Ø Ü Ù ½Øµ by the chain rule ¾Øµ Ý Ü Ùص Ø Ø¼ Ý Ù Ì Ö Üµ Ù (5) Therefore, given the gradient, we can easily compute the directional derivative in any direction Ù. Note that from equation (5) we see ٠ܵ ¼for directions Ù orthogonal to the gradient Ö Üµ. The image gradient is therefore orthogonal to curves of constant intensity, i.e. contours satisfying Ü Øµµ, for any constant. The steepest ascent direction Ù (at a particular value of Ü) is defined to be the unit vector which maximizes the directional derivative ٠ܵ. From equation (5) this steepest ascent direction is given by Ù Ö Üµ Ö Üµ (6) Ô where Û Û ¾ ½ Û¾ denotes the Euclidean norm (i.e., 2-norm). Thus the gradient points in the steepest ascent direction. If the gradient Ö Üµ ¼,thenÜ is said to be a stationary point of ܵ. Typically this is a local minimum, maximum or saddle point in ܵ. At a stationary point Ü, the directional derivative ٠ܵ ¼for any direction Ù, and therefore the steepest ascent direction is undefined at such an Ü (i.e., a divide by zero occurs in equation (6)). 320: Edge Detection Notes: 13
2D Edge Detection We extend our approach to 1D edge detection to 2D images by considering the variation of image brightness in particular directions Ù. That is, at a pixel Ü, we consider the variation along a 1D slice, Á Ü Ùص, in the neighbourhood of Ø ¼. The direction of this slice is chosen to be the steepest ascent direction at each pixel, as given by the direction of the image gradient Ê Üµ: ٠ܵ Ê Üµ Ê Üµ As described in the previous notes, this gradient can be estimated by differentiating a Gaussian blurred and interpolated approximation of the image, Ê Üµ Ö Ü ¾ Ö µ Á ܵ Image Edgel Detection: Recall that in 1D we detected edges by identifying local maxima in the absolute value of the response of a derivative of Gaussian filter applied to the signal. The analogous operation in 2D is to search for maxima in the directional image derivative taken in the gradient direction ٠ܵ. Since the gradient direction ٠ܵ is perpendicular to curves of constant brightness, we take any detected edgel to have normal Ò given by the gradient direction, that is, Ò Üµ ٠ܵ. 320: Edge Detection Page: 14
2D Edge Detection (cont.) Search for local maxima of gradient magnitude Ë Üµ Ê Üµ, in the direction normal to local edge, Ò Üµ, suppressing all responses except for local maxima (called non-maximum suppression). In practice, the search for local maxima of Ë Üµ takes place on the discrete sampling grid. Given Ü ¼, with normal Ò ¼, compare Ë Ü ¼ µ to nearby pixels closest to the direction of Ò ¼, e.g., pixels at Ü ¼ Õ ¼, ½ where Õ ¼ is Ò ¾ Ò µ ¼ with each of its coefficients rounded to the nearest integer. The red circle depicts points Ü ¼ ½ Ò ¾ Ò µ ¼. Normal directions between (blue) radial lines all map to the same neighbour of Ü ¼. 320: Edge Detection Page: 15
Canny Edge Detection Algorithm: 1. Convolve with gradient filters (possibly at multiple scales Ö ) Ê Üµ Ê ½ ܵ Ê ¾ ܵµ Ì Ö Ü ¾ Ö µ Á ܵ 2. Compute response magnitude, Ë Üµ Ô ¾ ʽ ܵ ʾ ¾ ܵ. 3. Compute local edge orientation (represented by unit normal): Ò Üµ Ê ½ ܵ Ê ¾ ܵµË ܵ if Ë Üµ ØÖ ÓÐ ¼ otherwise 4. Peak detection (non-maximum suppression along edge normal) Extensions: In order to select an appropriate scale Ö for an edgel, non-maximum suppression can also be done across neighbouring scales. Also the simple thresholding described above can be replaced by hysteresis thresholding along edges (see Canny (1986) for details). These are beyond the scope of this course. 320: Edge Detection Page: 16
Filtering with Derivatives of Gaussians Image three.pgm Gaussian Blur =1.0 Gradient in Ü Gradient in Ý 320: Edge Detection Page: 17
Canny Edgel Measurement Gradient Strength Gradient Orientations Canny Edgels Edgel Overlay Colour gives gradient direction (red ¼ Æ ; blue ¼ Æ ; green ¾¼ Æ ) 320: Edge Detection Page: 18
Gaussian Pyramid Filtering (Subsample 2) Blurred and Down-Sampled ( ¾) Gaussian Blur =1.0 Gradient Magnitude (dec ¾) Gradient Orientations 320: Edge Detection Page: 19
Gaussian Pyramid Filtering (Subsample 4) Blurred and Down-Sampled ( ) Gaussian Blur =1.0 Gradient Magnitude (dec ) Gradient Orientations 320: Edge Detection Page: 20
Multiscale Canny Edgels Image three.pgm Edgels (x 1) Edgels (x 2) Edgels (x 4) 320: Edge Detection Page: 21
Edge-Based Image Editing Existing edge detectors are sufficient for a wide variety of applications, such as image editing, tracking, and simple recognition. [from Elder and Goldberg (2001)] Approach: 1. Edgels represented by location, orientation, blur scale (min reliable scale for detection), and brightness on each side. 2. Edgels are grouped into curves (i.e., maximum likelihood curves joining two edge segments specified by a user.) 3. Curves are then manipulated (i.e., deleted, moved, clipped etc). 4. The image is reconstructed from edgel positions and the image brightnesses each side. 320: Edge Detection Page: 22
Further Readings Castleman, K.R., Digital Image Processing, Prentice Hall, 1995 John Canny, A computational approach to edge detection. IEEE Transactions on PAMI, 8(6):679 698, 1986. James Elder and Richard Goldberg, Image editing in the contour domain. IEEE Transactions on PAMI, 23(3):291 296, 2001. 320: Edge Detection Notes: 23