Image Filtering & Edge Detection

Transcription

1 What is imae filterin? Imae Filterin & Ede Detection Modify the pixels in an imae based on some function of a local neihborhood of the pixels. Some function Readin: Chapter 7 and 8, F&P Linear Functions Simplest: linear filterin. Replace each pixel by a linear combination of its neihbors. The prescription for the linear combination is called the convolution kernel. Convolution Let I be the imae and be the kernel. The output of convolvin I with is denoted I f[ m, n] = I = I[ m k, n l] [ k, l] k, l I Key properties Linearity: filter(i + I 2 ) = filter(i ) + filter(i 2 ) Shift invariance: same behavior reardless of pixel location filter(shift(i)) = shift(filter(i)) Theoretical result: Any linear shift-invariant operator can be represented as a convolution

2 Properties in more detail Commutative: a * b = b * a Conceptually no difference between filter and sinal Associative: a * (b * c) = (a * b) * c Often apply several filters one after another: (((a * b ) * b 2 ) * b 3 ) This is equivalent to applyin one filter: a * (b * b 2 * b 3 ) Distributes over addition: a * (b + c) = (a * b) + (a * c) Scalars factor out: ka * b = a * kb = k (a * b) Identity: unit impulse e = [,,,,,, ], a * e = a

3 Yucky details What is the size of the output? MATLAB: filter2(, I, shape) shape = full : output size is sum of sizes of I and shape = same : output size is same as I shape = valid :output size is of difference sizes for I & full same valid I I I Implementation details What about near the ede? the filter window falls off the ede of the imae need to extrapolate methods: clip filter (black) wrap around copy ede reflect across ede Implementation details What about near the ede? the filter window falls off the ede of the imae need to extrapolate methods (MATLAB): clip filter (black): imfilter(f,, ) wrap around: imfilter(f,, circular ) copy ede: imfilter(f,, replicate ) reflect across ede: imfilter(f,, symmetric ) Source: S. Marschner Source: S. Marschner

4 Linear filterin (warm-up slide) Linear filterin (warm-up slide).?. Filtered (no chane) Linear filterin Shift.?. shifted Linear filterin Blurrin.3.3? Box filter: Blurred (filter applied in both dimensions).

5 Blur examples Blur examples impulse impulse filtered filtered ede filtered Smoothin with box filter revisited Smoothin with an averae actually doesn t compare at all well with a defocused lens Most obvious difference is that a sinle point of liht viewed in a defocused lens looks like a fuzzy blob; but the averain process would ive a little square Smoothin with box filter revisited Smoothin with an averae actually doesn t compare at all well with a defocused lens Most obvious difference is that a sinle point of liht viewed in a defocused lens looks like a fuzzy blob; but the averain process would ive a little square Better idea: to eliminate ede effects, weiht contribution of neihborhood pixels accordin to their closeness to the center, like so: Source: D. Forsyth fuzzy blob Gaussian Kernel Choosin kernel width Gaussian filters have infinite support, but discrete filters use finite kernels x 5, σ = Constant factor at front makes volume sum to (can be inored, as we should re-normalize weihts to sum to in any case) Source: C. Rasmussen

6 Gaussian filterin A Gaussian kernel ives less weiht to pixels further from the center of the window Example: Smoothin with a Gaussian This kernel is an approximation of a Gaussian function: Mean vs. Gaussian filterin Separability of the Gaussian filter Source: D. Lowe Separability example 2D convolution (center location only) The filter factors into a product of D filters: Perform convolution alon rows: Followed by convolution alon the remainin column: * * = = Gaussian filters Remove hih-frequency components from the imae (low-pass filter) Convolution with self is another Gaussian So can smooth with small-width kernel, repeat, and et same result as larer-width kernel would have Convolvin two times with Gaussian kernel of width σ is same as convolvin once with kernel of width sqrt(2) σ Separable kernel Factors into product of two D Gaussians Useful: can convolve all rows, then all columns How does this chane the computational complexity? Linear vs. quadratic in mask size

7 Review: Linear filterin Noise What are the definin mathematical properties of a convolution? What is the difference between blurrin with a box filter and blurrin with a Gaussian? What happens when we convolve a Gaussian with another Gaussian? What is separability? How does separability affect computational complexity? Oriinal Impulse noise Salt and pepper noise Gaussian noise Salt and pepper noise: contains random occurrences of black and white pixels Impulse noise: contains random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution Source: S. Seitz Gaussian noise Mathematical model: sum of many independent factors Good for small standard deviations Assumption: independent, zero-mean noise Reducin Gaussian noise Smoothin with larer standard deviations suppresses noise, but also blurs the imae Reducin salt-and-pepper noise 3x3 5x5 7x7 Alternative idea: Median filterin A median filter operates over a window by selectin the median intensity in the window What s wron with the results? Is median filterin linear? No. Not a convolution

8 Median filter What advantae does median filterin have over Gaussian filterin? Robustness to outliers Median filter Salt-and-pepper noise Median filtered MATLAB: medfilt2(imae, [h w]) Median vs. Gaussian filterin 3x3 5x5 7x7 Linear filterin (warm-up slide) Gaussian 2..? Median Linear filterin (no chane) Linear filterin ? Filtered (no chane)

9 (Remember blurrin) Sharpenin Blurred (filter applied in both dimensions). Sharpened Sharpenin Sharpenin example Sharpened (differences are accentuated; constant areas are left untouched). Sharpenin Sharpenin 2 -? 2 - Oriinal (Note that filter sums to ) Oriinal Sharpenin filter - Accentuates differences with local averae Source: D. Lowe Source: D. Lowe

10 Sharpenin Unsharp mask filter I + α( I I ) = ( + α ) I α I = I (( + α ) e ) imae blurred imae unit impulse (identity) before after unit impulse Gaussian Laplacian of Gaussian Sharpenin Revisited What does blurrin take away? = smoothed (5x5) Let s add it back: detail Ede detection Goal: Identify sudden chanes (discontinuities) in an imae Intuitively, most semantic and shape information from the imae can be encoded in the edes More compact than pixels + α detail = sharpened Ideal: artist s line drawin (but artist is also usin object-level knowlede) Source: D. Lowe Oriin of Edes surface normal discontinuity Characterizin edes An ede is a place of rapid chane in the imae intensity function depth discontinuity imae intensity function (alon horizontal scanline) first derivative surface color discontinuity illumination discontinuity Edes are caused by a variety of factors edes correspond to extrema of derivative Source: Steve Seitz

11 Imae radient The radient of an imae: The radient points in the direction of most rapid chane in intensity Differentiation and convolution Recall, for 2D function, f(x,y): f x = lim f( x + ε, y) f( x, y) ε ε ε We could approximate this as f x f( x,y) f x, y n+ n x ( ) The radient direction is iven by: how does this relate to the direction of the ede? perpendicular The ede strenth is iven by the radient manitude This is linear and shift invariant, so must be the result of a convolution. (which is obviously a convolution) - Source: D. Forsyth, D. Lowe Finite differences: example Finite difference filters Other approximations of derivative filters exist: Which one is the radient in the x-direction (resp. y-direction)? Effects of noise Consider a sinle row or column of the imae Plottin intensity as a function of position ives a sinal How to compute a derivative? Effects of noise Finite difference filters respond stronly to noise Imae noise results in pixels that look very different from their neihbors Generally, the larer the noise the stroner the response What is to be done? Where is the ede? Source: D. Forsyth

12 Effects of noise Finite difference filters respond stronly to noise Imae noise results in pixels that look very different from their neihbors Generally, the larer the noise the stroner the response What is to be done? Smoothin the imae should help, by forcin pixels different to their neihbors (=noise pixels?) to look more like neihbors Solution: smooth first Source: D. Forsyth Where is the ede? Look for peaks in Derivative theorem of convolution Differentiation is convolution, and convolution is associative: d d ( f ) = f dx dx This saves us one operation: f Derivative of Gaussian filter [ -] = * d dx d f dx Source: S. Seitz Derivative of Gaussian filter x-direction y-direction Which one finds horizontal/vertical edes? Summary: Filter mask properties Filters act as templates Hihest response for reions that look the most like the filter Dot product as correlation Smoothin masks Values positive Sum to constant reions are unchaned Amount of smoothin proportional to mask size Derivative masks Opposite sins used to et hih response in reions of hih contrast Sum to no response in constant reions Hih absolute value at points of hih contrast

13 Tradeoff between smoothin and localization Implementation issues pixel 3 pixels 7 pixels Smoothed derivative removes noise, but blurs ede. Also finds edes at different scales. Source: D. Forsyth The radient manitude is lare alon a thick trail or ride, so how do we identify the actual ede points? How do we link the ede points to form curves? Source: D. Forsyth Laplacian of Gaussian 2D ede detection filters Consider Laplacian of Gaussian Laplacian of Gaussian operator Gaussian derivative of Gaussian is the Laplacian operator: Where is the ede? Zero-crossins of bottom raph MATLAB demo Ede findin = fspecial('aussian',5,2); imaesc() surfl() clown = conv2(clown,,'same'); imaesc(conv2(clown,[- ],'same')); imaesc(conv2(clown,[- ],'same')); dx = conv2(,[- ],'same'); imaesc(conv2(clown,dx,'same')); l = fspecial('lo',5,2); lclown = conv2(clown,l,'same'); imaesc(lclown) imaesc(clown +.2*lclown) We wish to mark points alon the curve where the manitude is biest. We can do this by lookin for a maximum alon a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two alorithmic issues: at which point is the maximum, and where is the next one? Source: D. Forsyth

14 Non-maximum suppression At q, we have a maximum if the value is larer than those at both p and at r. Interpolate to et these values. Predictin the next ede point Assume the marked point is an ede point. Then we construct the tanent to the ede curve (which is normal to the radient at that point) and use this to predict the next points (here either r or s). Source: D. Forsyth Source: D. Forsyth Desinin an ede detector Criteria for an optimal ede detector: Good detection: the optimal detector must minimize the probability of false positives (detectin spurious edes caused by noise), as well as that of false neatives (missin real edes) Good localization: the edes detected must be as close as possible to the true edes Sinle response: the detector must return one point only for each true ede point; that is, minimize the number of local maxima around the true ede Canny ede detector This is probably the most widely used ede detector in computer vision Theoretical model: step-edes corrupted by additive Gaussian noise Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of sinal-to-noise ratio and localization J. Canny, A Computational Approach To Ede Detection, IEEE Trans. Pattern Analysis and Machine Intellience, 8:679-74, 986. Source: L. Fei-Fei Source: L. Fei-Fei Canny ede detector. Filter imae with derivative of Gaussian 2. Find manitude and orientation of radient 3. Non-maximum suppression: Thin multi-pixel wide rides down to sinle pixel width 4. Linkin and thresholdin (hysteresis): Define two thresholds: low and hih Use the hih threshold to start ede curves and the low threshold to continue them The Canny ede detector MATLAB: ede(imae, canny ) imae (Lena) Source: D. Lowe, L. Fei-Fei

15 The Canny ede detector The Canny ede detector norm of the radient thresholdin The Canny ede detector Hysteresis thresholdin imae thinnin (non-maximum suppression) hih threshold (stron edes) low threshold (weak edes) hysteresis threshold Source: L. Fei-Fei Effect of σ (Gaussian kernel spread/size) Ede detection is just the beinnin imae human sementation radient manitude Canny with Canny with The choice of σ depends on desired behavior lare σ detects lare scale edes small σ detects fine features Source: S. Seitz Berkeley sementation database: nch/