Computer Vision: Filtering

Size: px
Start display at page:

Download "Computer Vision: Filtering"

Transcription

1 Computer Vision: Filtering Raquel Urtasun TTI Chicago Jan 10, 2013 Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

2 Today s lecture... Image formation Image Filtering Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

3 Readings Chapter 2 and 3 of Rich Szeliski s book Available online here Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

4 How is an image created? The image formation process that produced a particular image depends on lighting conditions scene geometry surface properties camera optics [Source: R. Szeliski] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

5 Image formation Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

6 [Source: A. Efros] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82 What is an image?!"#"$%&'(%)*+%' 0*1&&'23456'37'$-*6*'"7'$-"6'4&%66',-*'./*'

7 From photons to RGB values Sample the 2D space on a regular grid. Quantize each sample, i.e., the photons arriving at each active cell are integrated and then digitized. [Source: D. Hoiem] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

8 What is an image? A grid (matrix) of intensity values #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #% % #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ &$ &$ &$ #$$ #$$ #$$ #$$ #$$ #$$!" #$$ #$$ &$ '$ '$ &$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ '( )#& )*$ )&$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ )#& )*$ )&$ )&$ )&$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ )#& )*$ #%% #%% )&$ )&$ '$ #$$ #$$ #$$ #$$ #$$ )#& )*$ #%% #%% )&$ )&$ '$ *& #$$ #$$ #$$ #$$ )#& )*$ )*$ )&$ )#& )#& '$ *& #$$ #$$ #$$ #$$ &* )#& )#& )#& '$ '$ '$ *& #$$ #$$ #$$ #$$ #$$ &* &* &* &* &* &* #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ #$$ Common to use one byte per value: 0=black, 255=white) [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

9 What is an image? We can think of a (grayscale) image as a function f : R 2 R giving the intensity at position (x, y) f (x, y) x y A digital image is a discrete (sampled, quantized) version of this function [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

10 Image Transformations As with any function, we can apply operators to an image!g (x,y) = f (x,y) + 20!g (x,y) = f (-x,y) We ll talk about special kinds of operators, correlation and convolution (linear filtering) [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

11 Filtering Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

12 Question: Noise reduction Given a camera and a still scene, how can you reduce noise? Take lots of images and average them! Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

13 Question: Noise reduction Given a camera and a still scene, how can you reduce noise? Take lots of images and average them! Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

14 Question: Noise reduction Given a camera and a still scene, how can you reduce noise? Take lots of images and average them! What s the next best thing? [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

15 Question: Noise reduction Given a camera and a still scene, how can you reduce noise? Take lots of images and average them! What s the next best thing? [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

16 Image filtering Modify the pixels in an image based on some function of a local neighborhood of each pixel #'" $" #"!" &"!" #" #" %" 5).0"678*9)8" %" ()*+,"-.+/0"1+2+" 3)1-401"-.+/0"1+2+" [Source: L. Zhang] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

17 Applications of Filtering Enhance an image, e.g., denoise, resize. Extract information, e.g., texture, edges. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

18 Applications of Filtering Enhance an image, e.g., denoise, resize. Extract information, e.g., texture, edges. Detect patterns, e.g., template matching. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

19 Applications of Filtering Enhance an image, e.g., denoise, resize. Extract information, e.g., texture, edges. Detect patterns, e.g., template matching. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

20 Noise reduction Simplest thing: replace each pixel by the average of its neighbors. This assumes that neighboring pixels are similar, and the noise to be independent from pixel to pixel. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

21 Noise reduction Simplest thing: replace each pixel by the average of its neighbors. This assumes that neighboring pixels are similar, and the noise to be independent from pixel to pixel. [Source: S. Marschner] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

22 Noise reduction Simplest thing: replace each pixel by the average of its neighbors. This assumes that neighboring pixels are similar, and the noise to be independent from pixel to pixel. [Source: S. Marschner] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

23 Noise reduction Simplest thing: replace each pixel by the average of its neighbors This assumes that neighboring pixels are similar, and the noise to be independent from pixel to pixel. Moving average in 1D: [1, 1, 1, 1, 1]/5 [Source: S. Marschner] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

24 Noise reduction Simpler thing: replace each pixel by the average of its neighbors This assumes that neighboring pixels are similar, and the noise to be independent from pixel to pixel. Non-uniform weights [1, 4, 6, 4, 1] / 16 [Source: S. Marschner] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

25 Moving Average in 2D [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

26 Moving Average in 2D [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

27 Moving Average in 2D [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

28 Moving Average in 2D [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

29 Moving Average in 2D [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

30 Moving Average in 2D [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

31 Linear Filtering: Correlation Involves weighted combinations of pixels in small neighborhoods. The output pixels value is determined as a weighted sum of input pixel values g(i, j) = k,l f (i + k, j + l)h(k, l) Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

32 Linear Filtering: Correlation Involves weighted combinations of pixels in small neighborhoods. The output pixels value is determined as a weighted sum of input pixel values g(i, j) = k,l f (i + k, j + l)h(k, l) The entries of the weight kernel or mask h(k, l) are often called the filter coefficients. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

33 Linear Filtering: Correlation Involves weighted combinations of pixels in small neighborhoods. The output pixels value is determined as a weighted sum of input pixel values g(i, j) = k,l f (i + k, j + l)h(k, l) The entries of the weight kernel or mask h(k, l) are often called the filter coefficients. This operator is the correlation operator g = f h Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

34 Linear Filtering: Correlation Involves weighted combinations of pixels in small neighborhoods. The output pixels value is determined as a weighted sum of input pixel values g(i, j) = k,l f (i + k, j + l)h(k, l) The entries of the weight kernel or mask h(k, l) are often called the filter coefficients. This operator is the correlation operator g = f h Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

35 Smoothing by averaging What if the filter size was 5 x 5 instead of 3 x 3? [Source: K. Graumann] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

36 Gaussian filter What if we want nearest neighboring pixels to have the most influence on the output? Removes high-frequency components from the image (low-pass filter). [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

37 Smoothing with a Gaussian [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

38 Mean vs Gaussian Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

39 Gaussian filter: Parameters Size of kernel or mask: Gaussian function has infinite support, but discrete filters use finite kernels. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

40 Gaussian filter: Parameters Variance of the Gaussian: determines extent of smoothing. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

41 Gaussian filter: Parameters [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

42 Is this the most general Gaussian? No, the most general form for x R d N (x; µ, Σ) = ( 1 exp 1 ) (2π) d/2 Σ 1/2 2 (x µ)t Σ 1 (x µ) But the simplified version is typically use for filtering. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

43 Properties of the Smoothing All values are positive. They all sum to 1. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

44 Properties of the Smoothing All values are positive. They all sum to 1. Amount of smoothing proportional to mask size. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

45 Properties of the Smoothing All values are positive. They all sum to 1. Amount of smoothing proportional to mask size. Remove high-frequency components; low-pass filter. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

46 Properties of the Smoothing All values are positive. They all sum to 1. Amount of smoothing proportional to mask size. Remove high-frequency components; low-pass filter. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

47 Example of Correlation What is the result of filtering the impulse signal (image) F with the arbitrary kernel H? [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

48 Convolution Convolution operator g(i, j) = k,l f (i k, j l)h(k, l) = k,l f (k, l)h(i k, j l) = f h and h is then called the impulse response function. Equivalent to flip the filter in both dimensions (bottom to top, right to left) and apply cross-correlation. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

49 Convolution Convolution operator g(i, j) = k,l f (i k, j l)h(k, l) = k,l f (k, l)h(i k, j l) = f h and h is then called the impulse response function. Equivalent to flip the filter in both dimensions (bottom to top, right to left) and apply cross-correlation. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

50 Matrix form Correlation and convolution can both be written as a matrix-vector multiply, if we first convert the two-dimensional images f (i, j) and g(i, j) into raster-ordered vectors f and g with H a sparse matrix. g = Hf Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

51 Correlation vs Convolution Convolution g(i, j) = k,l f (i k, j l)h(k, l) G = H F Correlation g(i, j) = k,l f (i + k, j + l)h(k, l) G = H F For a Gaussian or box filter, how will the outputs differ? If the input is an impulse signal, how will the outputs differ? h δ?, and h δ? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

52 Example What s the result? [Source: D. Lowe] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

53 Example What s the result? [Source: D. Lowe] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

54 Example What s the result? [Source: D. Lowe] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

55 Example What s the result? [Source: D. Lowe] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

56 Example What s the result? * #" #" #" #" $" #" #" #" #"!"!"!" -!!"!"!" %"!"!"!" Original! [Source: D. Lowe] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

57 Example What s the result? * #" #" #" #" $" #" #" #" #" -!!"!"!"!"!"!"!"!"!" 0" Original!!"#$%&'(')*+,-&$* %&''()*+&*(,"(-.(,/* [Source: D. Lowe] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

58 Sharpening [Source: D. Lowe] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

59 Gaussian Filter Convolution with itself is another Gaussian *!" Convolving twice with Gaussian kernel of width σ is the same as convolving once with kernel of width σ 2 [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

60 Sharpening revisited What does blurring take away?!" #"!"#$#%&'( )*!!+,-.(/0102(.-+&#'( Let s add it back #"$"!" *$+,+'#-) (&.#+-)!"#$%&'&() [Source: S. Lazebnik] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

61 Sharpening!"#$%&'&() #$%&'&() [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

62 Optical Convolution Camera Shake!" * Figure: Fergus, et al., SIGGRAPH 2006 Blur in out-of-focus regions of an image. Figure: Bokeh: Click for more info [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

63 Correlation vs Convolution The convolution is both commutative and associative. The Fourier transform of two convolved images is the product of their individual Fourier transforms. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

64 Correlation vs Convolution The convolution is both commutative and associative. The Fourier transform of two convolved images is the product of their individual Fourier transforms. Both correlation and convolution are linear shift-invariant (LSI) operators, which obey both the superposition principle and the shift invariance principle h (f 0 + f 1 ) = h f o + h f 1 if g(i, j) = f (i + k, j + l) (h g)(i, j) = (h f )(i + k, j + l) which means that shifting a signal commutes with applying the operator. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

65 Correlation vs Convolution The convolution is both commutative and associative. The Fourier transform of two convolved images is the product of their individual Fourier transforms. Both correlation and convolution are linear shift-invariant (LSI) operators, which obey both the superposition principle and the shift invariance principle h (f 0 + f 1 ) = h f o + h f 1 if g(i, j) = f (i + k, j + l) (h g)(i, j) = (h f )(i + k, j + l) which means that shifting a signal commutes with applying the operator. Is the same as saying that the effect of the operator is the same everywhere. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

66 Correlation vs Convolution The convolution is both commutative and associative. The Fourier transform of two convolved images is the product of their individual Fourier transforms. Both correlation and convolution are linear shift-invariant (LSI) operators, which obey both the superposition principle and the shift invariance principle h (f 0 + f 1 ) = h f o + h f 1 if g(i, j) = f (i + k, j + l) (h g)(i, j) = (h f )(i + k, j + l) which means that shifting a signal commutes with applying the operator. Is the same as saying that the effect of the operator is the same everywhere. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

67 Boundary Effects The results of filtering the image in this form will lead to a darkening of the corner pixels. The original image is effectively being padded with 0 values wherever the convolution kernel extends beyond the original image boundaries. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

68 Boundary Effects The results of filtering the image in this form will lead to a darkening of the corner pixels. The original image is effectively being padded with 0 values wherever the convolution kernel extends beyond the original image boundaries. A number of alternative padding or extension modes have been developed. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

69 Boundary Effects The results of filtering the image in this form will lead to a darkening of the corner pixels. The original image is effectively being padded with 0 values wherever the convolution kernel extends beyond the original image boundaries. A number of alternative padding or extension modes have been developed. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

70 Boundary Effects The results of filtering the image in this form will lead to a darkening of the corner pixels. The original image is effectively being padded with 0 values wherever the convolution kernel extends beyond the original image boundaries. A number of alternative padding or extension modes have been developed. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

71 Separable Filters The process of performing a convolution requires K 2 operations per pixel, where K is the size (width or height) of the convolution kernel. In many cases, this operation can be speed up by first performing a 1D horizontal convolution followed by a 1D vertical convolution, requiring 2K operations. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

72 Separable Filters The process of performing a convolution requires K 2 operations per pixel, where K is the size (width or height) of the convolution kernel. In many cases, this operation can be speed up by first performing a 1D horizontal convolution followed by a 1D vertical convolution, requiring 2K operations. If his is possible, then the convolution kernel is called separable. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

73 Separable Filters The process of performing a convolution requires K 2 operations per pixel, where K is the size (width or height) of the convolution kernel. In many cases, this operation can be speed up by first performing a 1D horizontal convolution followed by a 1D vertical convolution, requiring 2K operations. If his is possible, then the convolution kernel is called separable. And it is the outer product of two kernels K = vh T Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

74 Separable Filters The process of performing a convolution requires K 2 operations per pixel, where K is the size (width or height) of the convolution kernel. In many cases, this operation can be speed up by first performing a 1D horizontal convolution followed by a 1D vertical convolution, requiring 2K operations. If his is possible, then the convolution kernel is called separable. And it is the outer product of two kernels K = vh T Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

75 Let s play a game... Is this separable? If yes, what s the separable version? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

76 Let s play a game... Is this separable? If yes, what s the separable version? What does this filter do? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

77 Let s play a game... Is this separable? If yes, what s the separable version? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

78 Let s play a game... Is this separable? If yes, what s the separable version? What does this filter do? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

79 Let s play a game... Is this separable? If yes, what s the separable version? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

80 Let s play a game... Is this separable? If yes, what s the separable version? What does this filter do? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

81 Let s play a game... Is this separable? If yes, what s the separable version? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

82 Let s play a game... Is this separable? If yes, what s the separable version? What does this filter do? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

83 Let s play a game... Is this separable? If yes, what s the separable version? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

84 Let s play a game... Is this separable? If yes, what s the separable version? What does this filter do? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

85 How can we tell if a given kernel K is indeed separable? Inspection... this is what we were doing. Looking at the analytic form of it. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

86 How can we tell if a given kernel K is indeed separable? Inspection... this is what we were doing. Looking at the analytic form of it. Look at the singular value decomposition (SVD), and if only one singular value is non-zero, then it is separable K = UΣV T = i σ i u i v T i with Σ = diag(σ i ). Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

87 How can we tell if a given kernel K is indeed separable? Inspection... this is what we were doing. Looking at the analytic form of it. Look at the singular value decomposition (SVD), and if only one singular value is non-zero, then it is separable K = UΣV T = i σ i u i v T i with Σ = diag(σ i ). σ1 u 1 and σ 1 v T 1 are the vertical and horizontal kernels. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

88 How can we tell if a given kernel K is indeed separable? Inspection... this is what we were doing. Looking at the analytic form of it. Look at the singular value decomposition (SVD), and if only one singular value is non-zero, then it is separable K = UΣV T = i σ i u i v T i with Σ = diag(σ i ). σ1 u 1 and σ 1 v T 1 are the vertical and horizontal kernels. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

89 Application of filtering: Template matching Filters as templates: filters look like the effects they are intended to find. Use normalized cross-correlation score to find a given pattern (template) in the image. Normalization needed to control for relative brightnesses. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

90 Template matching [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

91 More complex Scenes Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

92 Let s talk about Edge Detection Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

93 Filtering: Edge detection Map image from 2d array of pixels to a set of curves or line segments or contours. More compact than pixels. Look for strong gradients, post-process. Figure: [Shotton et al. PAMI, 07] [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

94 Origin of edges Edges are caused by a variety of factors surface normal discontinuity depth discontinuity surface color discontinuity illumination discontinuity [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

95 What causes an edge? [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

96 Looking more locally... [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

97 Images as functions Edges look like steep cliffs [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

98 Characterizing Edges An edge is a place of rapid change in the image intensity function. [Source: S. Lazebnik] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

99 How to Implement Derivatives with Convolution How can we differentiate a digital image F[x,y]? Option 1: reconstruct a continuous image f, then compute the partial derivative as f (x, y) x = lim ɛ 0 f (x + ɛ, y) f (x) ɛ Option 2: take discrete derivative (finite difference) f (x, y) x f [x + 1, y] f [x] 1 Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

100 How to Implement Derivatives with Convolution How can we differentiate a digital image F[x,y]? Option 1: reconstruct a continuous image f, then compute the partial derivative as f (x, y) x = lim ɛ 0 f (x + ɛ, y) f (x) ɛ Option 2: take discrete derivative (finite difference) f (x, y) x f [x + 1, y] f [x] 1 What would be the filter to implement this using convolution? Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

101 How to Implement Derivatives with Convolution How can we differentiate a digital image F[x,y]? Option 1: reconstruct a continuous image f, then compute the partial derivative as f (x, y) x = lim ɛ 0 f (x + ɛ, y) f (x) ɛ Option 2: take discrete derivative (finite difference) f (x, y) x f [x + 1, y] f [x] 1 What would be the filter to implement this using convolution?!"!" [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

102 How to Implement Derivatives with Convolution How can we differentiate a digital image F[x,y]? Option 1: reconstruct a continuous image f, then compute the partial derivative as f (x, y) x = lim ɛ 0 f (x + ɛ, y) f (x) ɛ Option 2: take discrete derivative (finite difference) f (x, y) x f [x + 1, y] f [x] 1 What would be the filter to implement this using convolution?!"!" [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

103 Partial derivatives of an image Figure: Using correlation filters [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

104 Finite Difference Filters [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

105 Image Gradient The gradient of an image f = [ ] f x, f y The gradient points in the direction of most rapid change in intensity Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

106 Image Gradient The gradient of an image f = [ ] f x, f y The gradient points in the direction of most rapid change in intensity The gradient direction (orientation of edge normal) is given by: ( f θ = tan 1 y / f ) x Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

107 Image Gradient The gradient of an image f = [ ] f x, f y The gradient points in the direction of most rapid change in intensity The gradient direction (orientation of edge normal) is given by: ( f θ = tan 1 y / f ) x The edge strength is given by the magnitude f = ( f x )2 + ( f y )2 [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

108 Image Gradient The gradient of an image f = [ ] f x, f y The gradient points in the direction of most rapid change in intensity The gradient direction (orientation of edge normal) is given by: ( f θ = tan 1 y / f ) x The edge strength is given by the magnitude f = ( f x )2 + ( f y )2 [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

109 Image Gradient [Source: S. Lazebnik] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

110 Effects of noise Consider a single row or column of the image. Plotting intensity as a function of position gives a signal.!"#$%&#'()*&#+,-.& [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

111 Effects of noise Smooth first, and look for picks in x (h f ). [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

112 Derivative theorem of convolution Differentiation property of convolution It saves one operation h (h f ) = ( x x ) f = h ( f x ) [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

113 2D Edge Detection Filters Gaussian h σ (x, y) = 1 2πσ exp u 2 2 +v 2 2σ 2 Derivative of Gaussian (x) x h σ(u, v) [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

114 Derivative of Gaussians [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

115 Laplacian of Gaussians Edge by detecting zero-crossings of bottom graph [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

116 2D Edge Filtering with 2 the Laplacian operator 2 f = 2 f x + 2 f 2 y 2 [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

117 Effect of σ on derivatives The detected structures differ depending on the Gaussian s scale parameter: Larger values: larger scale edges detected. Smaller values: finer features detected. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

118 Derivatives Use opposite signs to get response in regions of high contrast. They sum to 0 so that there is no response in constant regions. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

119 Derivatives Use opposite signs to get response in regions of high contrast. They sum to 0 so that there is no response in constant regions. High absolute value at points of high contrast. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

120 Derivatives Use opposite signs to get response in regions of high contrast. They sum to 0 so that there is no response in constant regions. High absolute value at points of high contrast. [Source: K. Grauman] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

121 Band-pass filters The Sobel and corner filters are band-pass and oriented filters. More sophisticated filters can be obtained by convolving with a Gaussian filter G(x, y, σ) = 1 ( 2πσ 2 exp x 2 + y 2 ) 2σ 2 and taking the first or second derivatives. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

122 Band-pass filters The Sobel and corner filters are band-pass and oriented filters. More sophisticated filters can be obtained by convolving with a Gaussian filter G(x, y, σ) = 1 ( 2πσ 2 exp x 2 + y 2 ) 2σ 2 and taking the first or second derivatives. These filters are band-pass filters: they filter low and high frequencies. Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

123 Band-pass filters The Sobel and corner filters are band-pass and oriented filters. More sophisticated filters can be obtained by convolving with a Gaussian filter G(x, y, σ) = 1 ( 2πσ 2 exp x 2 + y 2 ) 2σ 2 and taking the first or second derivatives. These filters are band-pass filters: they filter low and high frequencies. The second derivative of a two-dimensional image is the laplacian operator 2 f = 2 f x f y 2 Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

124 Band-pass filters The Sobel and corner filters are band-pass and oriented filters. More sophisticated filters can be obtained by convolving with a Gaussian filter G(x, y, σ) = 1 ( 2πσ 2 exp x 2 + y 2 ) 2σ 2 and taking the first or second derivatives. These filters are band-pass filters: they filter low and high frequencies. The second derivative of a two-dimensional image is the laplacian operator 2 f = 2 f x f y 2 Blurring an image with a Gaussian and then taking its Laplacian is equivalent to convolving directly with the Laplacian of Gaussian (LoG) filter, Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

125 Band-pass filters The Sobel and corner filters are band-pass and oriented filters. More sophisticated filters can be obtained by convolving with a Gaussian filter G(x, y, σ) = 1 ( 2πσ 2 exp x 2 + y 2 ) 2σ 2 and taking the first or second derivatives. These filters are band-pass filters: they filter low and high frequencies. The second derivative of a two-dimensional image is the laplacian operator 2 f = 2 f x f y 2 Blurring an image with a Gaussian and then taking its Laplacian is equivalent to convolving directly with the Laplacian of Gaussian (LoG) filter, Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

126 Band-pass filters The directional or oriented filter can obtained by smoothing with a Gaussian (or some other filter) and then taking a directional derivative u = u u (G f ) = u (G f ) = ( u G) f with u = (cos θ, sin θ). The Sobel operator is a simple approximation of this: Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

127 Practical Example [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

128 Finding Edges Figure: Gradient magnitude [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

129 Finding Edges!"#$#%&'%("#%#)*#+% Figure: Gradient magnitude [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

130 Non-Maxima Suppression Figure: Gradient magnitude Check if pixel is local maximum along gradient direction: requires interpolation [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

131 Finding Edges Figure: Thresholding [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

132 Finding Edges Figure: Thinning: Non-maxima suppression [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

133 Canny Edge Detector Matlab: edge(image, canny ) 1 Filter image with derivative of Gaussian 2 Find magnitude and orientation of gradient 3 Non-maximum suppression 4 Linking and thresholding (hysteresis): Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them [Source: D. Lowe and L. Fei-Fei] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

134 Canny edge detector Still one of the most widely used edge detectors in computer vision J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8: , Depends on several parameters: σ of the blur and the thresholds Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

135 Canny edge detector large σ detects large-scale edges small σ detects fine edges original Canny with Canny with [Source: S. Seitz] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

136 Scale Space (Witkin 83) first derivative peaks larger Gaussian filtered signal Properties of scale space (w/ Gaussian smoothing) edge position may shift with increasing scale (σ) two edges may merge with increasing scale an edge may not split into two with increasing scale [Source: N. Snavely] Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

137 Next class... more on filtering and image features Raquel Urtasun (TTI-C) Computer Vision Jan 10, / 82

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/ Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters

More information

Spatial filtering. Computer Vision & Digital Image Processing. Spatial filter masks. Spatial filters (examples) Smoothing filters

Spatial filtering. Computer Vision & Digital Image Processing. Spatial filter masks. Spatial filters (examples) Smoothing filters Spatial filtering Computer Vision & Digital Image Processing Spatial Filtering Use of spatial masks for filtering is called spatial filtering May be linear or nonlinear Linear filters Lowpass: attenuate

More information

What is an Edge? Computer Vision Week 4. How to detect edges? What is an Edge? Edge Detection techniques. Edge Detection techniques.

What is an Edge? Computer Vision Week 4. How to detect edges? What is an Edge? Edge Detection techniques. Edge Detection techniques. What is an Edge? Computer Vision Week 4 Edge Detection Linear filtering; pyramids, wavelets Interest Operators surface normal discontinuity depth discontinuity surface color discontinuity illumination

More information

Edge detection. (Trucco, Chapt 4 AND Jain et al., Chapt 5) -Edges are significant local changes of intensity in an image.

Edge detection. (Trucco, Chapt 4 AND Jain et al., Chapt 5) -Edges are significant local changes of intensity in an image. Edge detection (Trucco, Chapt 4 AND Jain et al., Chapt 5) Definition of edges -Edges are significant local changes of intensity in an image. -Edges typically occur on the boundary between two different

More information

Lecture 2: 2D Fourier transforms and applications

Lecture 2: 2D Fourier transforms and applications Lecture 2: 2D Fourier transforms and applications B14 Image Analysis Michaelmas 2014 A. Zisserman Fourier transforms and spatial frequencies in 2D Definition and meaning The Convolution Theorem Applications

More information

Computational Foundations of Cognitive Science

Computational Foundations of Cognitive Science Computational Foundations of Cognitive Science Lecture 15: Convolutions and Kernels Frank Keller School of Informatics University of Edinburgh keller@inf.ed.ac.uk February 23, 2010 Frank Keller Computational

More information

jorge s. marques image processing

jorge s. marques image processing image processing images images: what are they? what is shown in this image? What is this? what is an image images describe the evolution of physical variables (intensity, color, reflectance, condutivity)

More information

Today s Topics. Lecture 11: LoG and DoG Filters. Recall: First Derivative Filters. Second-Derivative Filters

Today s Topics. Lecture 11: LoG and DoG Filters. Recall: First Derivative Filters. Second-Derivative Filters Today s Topics Lecture : LoG and DoG Filters Laplacian of Gaussian (LoG) Filter - useful for finding edges - also useful for finding blobs! approimation using Difference of Gaussian (DoG) Recall: First

More information

Convolution, Noise and Filters

Convolution, Noise and Filters T H E U N I V E R S I T Y of T E X A S Convolution, Noise and Filters Philip Baldwin, Ph.D. Department of Biochemistry Response to an Entire Signal The response of a system with impulse response h(t) to

More information

- What is a feature? - Image processing essentials - Edge detection (Sobel & Canny) - Hough transform - Some images

- What is a feature? - Image processing essentials - Edge detection (Sobel & Canny) - Hough transform - Some images Seminar: Feature extraction by André Aichert I Feature detection - What is a feature? - Image processing essentials - Edge detection (Sobel & Canny) - Hough transform - Some images II An Entropy-based

More information

BIL Computer Vision Mar 5, 2014

BIL Computer Vision Mar 5, 2014 BIL 719 - Computer Vision Mar 5, 2014 Image pyramids Aykut Erdem Dept. of Computer Engineering Hacettepe University Image Scaling This image is too big to fit on the screen. How can we reduce it? How to

More information

Correlation and Convolution Class Notes for CMSC 426, Fall 2005 David Jacobs

Correlation and Convolution Class Notes for CMSC 426, Fall 2005 David Jacobs Correlation and Convolution Class otes for CMSC 46, Fall 5 David Jacobs Introduction Correlation and Convolution are basic operations that we will perform to extract information from images. They are in

More information

Digital Image Processing Using Matlab. Haris Papasaika-Hanusch Institute of Geodesy and Photogrammetry, ETH Zurich

Digital Image Processing Using Matlab. Haris Papasaika-Hanusch Institute of Geodesy and Photogrammetry, ETH Zurich Haris Papasaika-Hanusch Institute of Geodesy and Photogrammetry, ETH Zurich haris@geod.baug.ethz.ch Images and Digital Images A digital image differs from a photo in that the values are all discrete. Usually

More information

Image Segmentation Preview Segmentation subdivides an image to regions or objects Two basic properties of intensity values Discontinuity Edge detection Similarity Thresholding Region growing/splitting/merging

More information

Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition

Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition Bildverarbeitung und Mustererkennung Image Processing and Pattern Recognition 1. Image Pre-Processing - Pixel Brightness Transformation - Geometric Transformation - Image Denoising 1 1. Image Pre-Processing

More information

Digital Imaging and Multimedia. Filters. Ahmed Elgammal Dept. of Computer Science Rutgers University

Digital Imaging and Multimedia. Filters. Ahmed Elgammal Dept. of Computer Science Rutgers University Digital Imaging and Multimedia Filters Ahmed Elgammal Dept. of Computer Science Rutgers University Outlines What are Filters Linear Filters Convolution operation Properties of Linear Filters Application

More information

Image Filtering & Edge Detection

Image Filtering & Edge Detection What is imae filterin? Imae Filterin & Ede Detection Modify the pixels in an imae based on some function of a local neihborhood of the pixels. Some function Readin: Chapter 7 and 8, F&P Linear Functions

More information

Image Processing. Alpha Channel. Blending. Image Compositing. Blending Errors. Blending in OpenGL

Image Processing. Alpha Channel. Blending. Image Compositing. Blending Errors. Blending in OpenGL CSCI 42 Computer Graphics Lecture 22 Image Processing Blending Display Color Models Filters Dithering [Ch 7.13, 8.11-8.12] Jernej Barbic University of Southern California Alpha Channel Frame buffer Simple

More information

Linear Filtering Part II

Linear Filtering Part II Linear Filtering Part II Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Fourier theory Jean Baptiste Joseph Fourier had a crazy idea: Any periodic function can

More information

Picture Perfect: The Mathematics of JPEG Compression

Picture Perfect: The Mathematics of JPEG Compression Picture Perfect: The Mathematics of JPEG Compression May 19, 2011 1 2 3 in 2D Sampling and the DCT in 2D 2D Compression Images Outline A typical color image might be 600 by 800 pixels. Images Outline A

More information

Digital image processing

Digital image processing Digital image processing The two-dimensional discrete Fourier transform and applications: image filtering in the frequency domain Introduction Frequency domain filtering modifies the brightness values

More information

REAL TIME TRAFFIC LIGHT CONTROL USING IMAGE PROCESSING

REAL TIME TRAFFIC LIGHT CONTROL USING IMAGE PROCESSING REAL TIME TRAFFIC LIGHT CONTROL USING IMAGE PROCESSING Ms.PALLAVI CHOUDEKAR Ajay Kumar Garg Engineering College, Department of electrical and electronics Ms.SAYANTI BANERJEE Ajay Kumar Garg Engineering

More information

Implementation of Canny Edge Detector of color images on CELL/B.E. Architecture.

Implementation of Canny Edge Detector of color images on CELL/B.E. Architecture. Implementation of Canny Edge Detector of color images on CELL/B.E. Architecture. Chirag Gupta,Sumod Mohan K cgupta@clemson.edu, sumodm@clemson.edu Abstract In this project we propose a method to improve

More information

Image Processing with. ImageJ. Biology. Imaging

Image Processing with. ImageJ. Biology. Imaging Image Processing with ImageJ 1. Spatial filters Outlines background correction image denoising edges detection 2. Fourier domain filtering correction of periodic artefacts 3. Binary operations masks morphological

More information

Canny Edge Detection

Canny Edge Detection Canny Edge Detection 09gr820 March 23, 2009 1 Introduction The purpose of edge detection in general is to significantly reduce the amount of data in an image, while preserving the structural properties

More information

QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING

QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING QUALITY TESTING OF WATER PUMP PULLEY USING IMAGE PROCESSING MRS. A H. TIRMARE 1, MS.R.N.KULKARNI 2, MR. A R. BHOSALE 3 MR. C.S. MORE 4 MR.A.G.NIMBALKAR 5 1, 2 Assistant professor Bharati Vidyapeeth s college

More information

Image Gradients. Given a discrete image Á Òµ, consider the smoothed continuous image ܵ defined by

Image Gradients. Given a discrete image Á Òµ, consider the smoothed continuous image ܵ defined by Image Gradients Given a discrete image Á Òµ, consider the smoothed continuous image ܵ defined by ܵ Ü ¾ Ö µ Á Òµ Ü ¾ Ö µá µ (1) where Ü ¾ Ö Ô µ Ü ¾ Ý ¾. ½ ¾ ¾ Ö ¾ Ü ¾ ¾ Ö. Here Ü is the 2-norm for the

More information

Compressive Sensing. Examples in Image Compression. Lecture 4, July 30, Luiz Velho Eduardo A. B. da Silva Adriana Schulz

Compressive Sensing. Examples in Image Compression. Lecture 4, July 30, Luiz Velho Eduardo A. B. da Silva Adriana Schulz Compressive Sensing Examples in Image Compression Lecture 4, July, 09 Luiz Velho Eduardo A. B. da Silva Adriana Schulz Today s Lecture Discuss applications of CS in image compression Evaluate CS efficiency

More information

Admin stuff. 4 Image Pyramids. Spatial Domain. Projects. Fourier domain 2/26/2008. Fourier as a change of basis

Admin stuff. 4 Image Pyramids. Spatial Domain. Projects. Fourier domain 2/26/2008. Fourier as a change of basis Admin stuff 4 Image Pyramids Change of office hours on Wed 4 th April Mon 3 st March 9.3.3pm (right after class) Change of time/date t of last class Currently Mon 5 th May What about Thursday 8 th May?

More information

Lecture 9: Shape Description (Regions)

Lecture 9: Shape Description (Regions) Lecture 9: Shape Description (Regions) c Bryan S. Morse, Brigham Young University, 1998 2000 Last modified on February 16, 2000 at 4:00 PM Contents 9.1 What Are Descriptors?.........................................

More information

Sharpening through spatial filtering

Sharpening through spatial filtering Sharpening through spatial filtering Stefano Ferrari Università degli Studi di Milano stefano.ferrari@unimi.it Elaborazione delle immagini (Image processing I) academic year 2011 2012 Sharpening The term

More information

Image Enhancement in the Frequency Domain

Image Enhancement in the Frequency Domain Image Enhancement in the Frequency Domain Jesus J. Caban Outline! Assignment #! Paper Presentation & Schedule! Frequency Domain! Mathematical Morphology %& Assignment #! Questions?! How s OpenCV?! You

More information

One-Dimensional Processing for Edge Detection using Hilbert Transform P. Kiran Kumar, Sukhendu Das and B. Yegnanarayana Department of Computer Science and Engineering Indian Institute of Technology, Madras

More information

2D Image Processing. Edge and Corner Detection. Prof. Didier Stricker Dr. Alain Pagani

2D Image Processing. Edge and Corner Detection. Prof. Didier Stricker Dr. Alain Pagani 2D Image Processing Edge and Corner Detection Prof. Didier Stricker Dr. Alain Pagani Kaiserlautern University http://ags.cs.uni-kl.de/ DFKI Deutsches Forschungszentrum für Künstliche Intelligenz http://av.dfki.de

More information

Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 269 Class Project Report

Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 269 Class Project Report Automatic 3D Reconstruction via Object Detection and 3D Transformable Model Matching CS 69 Class Project Report Junhua Mao and Lunbo Xu University of California, Los Angeles mjhustc@ucla.edu and lunbo

More information

Analecta Vol. 8, No. 2 ISSN 2064-7964

Analecta Vol. 8, No. 2 ISSN 2064-7964 EXPERIMENTAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS IN ENGINEERING PROCESSING SYSTEM S. Dadvandipour Institute of Information Engineering, University of Miskolc, Egyetemváros, 3515, Miskolc, Hungary,

More information

IMPLEMENTATION OF IMAGE PROCESSING IN REAL TIME CAR PARKING SYSTEM

IMPLEMENTATION OF IMAGE PROCESSING IN REAL TIME CAR PARKING SYSTEM IMPLEMENTATION OF IMAGE PROCESSING IN REAL TIME CAR PARKING SYSTEM Ms.SAYANTI BANERJEE Ajay Kumar Garg Engineering College, Department of electrical and electronics Ms.PALLAVI CHOUDEKAR Ajay Kumar Garg

More information

Edge Detection Techniques: Evaluations and Comparisons

Edge Detection Techniques: Evaluations and Comparisons Applied Mathematical Sciences, Vol. 2, 2008, no. 31, 1507-1520 Edge Detection Techniques: Evaluations and Comparisons Ehsan Nadernejad Department of Computer Engineering, Faculty of Engineering Mazandaran

More information

Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

More information

Railway Expansion Joint Gaps and Hooks Detection Using Morphological Processing, Corner Points and Blobs

Railway Expansion Joint Gaps and Hooks Detection Using Morphological Processing, Corner Points and Blobs Railway Expansion Joint Gaps and Hooks Detection Using Morphological Processing, Corner Points and Blobs Samiul Islam ID: 12301053 Rubayat Ahmed Khan ID: 11301026 Supervisor RubelBiswas Department of Computer

More information

Zhang Wu Directional LMMSE Image Demosaicking

Zhang Wu Directional LMMSE Image Demosaicking 2014/07/01 v0.5 IPOL article class Published in Image Processing On Line on 2011 09 01. Submitted on 2011 00 00, accepted on 2011 00 00. ISSN 2105 1232 c 2011 IPOL & the authors CC BY NC SA This article

More information

MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr

MVA ENS Cachan. Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Machine Learning for Computer Vision 1 MVA ENS Cachan Lecture 2: Logistic regression & intro to MIL Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Department of Applied Mathematics Ecole Centrale Paris Galen

More information

Another Example: the Hubble Space Telescope

Another Example: the Hubble Space Telescope 296 DIP Chapter and 2: Introduction and Integral Equations Motivation: Why Inverse Problems? A large-scale example, coming from a collaboration with Università degli Studi di Napoli Federico II in Naples.

More information

Lectures 6&7: Image Enhancement

Lectures 6&7: Image Enhancement Lectures 6&7: Image Enhancement Leena Ikonen Pattern Recognition (MVPR) Lappeenranta University of Technology (LUT) leena.ikonen@lut.fi http://www.it.lut.fi/ip/research/mvpr/ 1 Content Background Spatial

More information

Image Segmentation and Registration

Image Segmentation and Registration Image Segmentation and Registration Dr. Christine Tanner (tanner@vision.ee.ethz.ch) Computer Vision Laboratory, ETH Zürich Dr. Verena Kaynig, Machine Learning Laboratory, ETH Zürich Outline Segmentation

More information

Digital Image Processing

Digital Image Processing Digital Image Processing Using MATLAB Second Edition Rafael C. Gonzalez University of Tennessee Richard E. Woods MedData Interactive Steven L. Eddins The MathWorks, Inc. Gatesmark Publishing A Division

More information

EECS 556 Image Processing W 09. Interpolation. Interpolation techniques B splines

EECS 556 Image Processing W 09. Interpolation. Interpolation techniques B splines EECS 556 Image Processing W 09 Interpolation Interpolation techniques B splines What is image processing? Image processing is the application of 2D signal processing methods to images Image representation

More information

Computer Vision & Digital Image Processing

Computer Vision & Digital Image Processing Intensity Transformations and Spatial Filtering Basics Computer Vision & Digital Image Processing Intensity Transformations and Spatial Filtering Operations take place in the spatial domain Operate directly

More information

Background: A CCD consists of an array of individual detector sites, as suggested by the sketch below: Array of independent detector sites

Background: A CCD consists of an array of individual detector sites, as suggested by the sketch below: Array of independent detector sites Q: Can I use PixelScope to measure the MTF of a CCD camera? Can I use PixelScope to map the actual sensitive area of a pixel? A: Yes to both. This note explains how. Approach This can be a complicated

More information

Applications of Convolution in Image Processing with MATLAB

Applications of Convolution in Image Processing with MATLAB University of Washington Applications of Convolution in Image Processing with MATLAB Author: Sung Kim Instructor: Riley Casper August 20, 2013 1 Abstract The article presents a short introduction to image

More information

Visualization and Feature Extraction, FLOW Spring School 2016 Prof. Dr. Tino Weinkauf. Flow Visualization. Image-Based Methods (integration-based)

Visualization and Feature Extraction, FLOW Spring School 2016 Prof. Dr. Tino Weinkauf. Flow Visualization. Image-Based Methods (integration-based) Visualization and Feature Extraction, FLOW Spring School 2016 Prof. Dr. Tino Weinkauf Flow Visualization Image-Based Methods (integration-based) Spot Noise (Jarke van Wijk, Siggraph 1991) Flow Visualization:

More information

Blind Deconvolution of Corrupted Barcode Signals

Blind Deconvolution of Corrupted Barcode Signals Blind Deconvolution of Corrupted Barcode Signals Everardo Uribe and Yifan Zhang Advisors: Ernie Esser and Yifei Lou Interdisciplinary Computational and Applied Mathematics Program University of California,

More information

Optical Flow as a property of moving objects used for their registration

Optical Flow as a property of moving objects used for their registration Optical Flow as a property of moving objects used for their registration Wolfgang Schulz Computer Vision Course Project York University Email:wschulz@cs.yorku.ca 1. Introduction A soccer game is a real

More information

SGN-1158 Introduction to Signal Processing Test. Solutions

SGN-1158 Introduction to Signal Processing Test. Solutions SGN-1158 Introduction to Signal Processing Test. Solutions 1. Convolve the function ( ) with itself and show that the Fourier transform of the result is the square of the Fourier transform of ( ). (Hints:

More information

APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (Copyright 2001, David T. Sandwell)

APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (Copyright 2001, David T. Sandwell) 1 APPLICATIONS AND REVIEW OF FOURIER TRANSFORM/SERIES (Copyright 2001, David T. Sandwell) (Reference The Fourier Transform and its Application, second edition, R.N. Bracewell, McGraw-Hill Book Co., New

More information

Module II: Multimedia Data Mining

Module II: Multimedia Data Mining ALMA MATER STUDIORUM - UNIVERSITÀ DI BOLOGNA Module II: Multimedia Data Mining Laurea Magistrale in Ingegneria Informatica University of Bologna Multimedia Data Retrieval Home page: http://www-db.disi.unibo.it/courses/dm/

More information

Maximum and Minimum Values

Maximum and Minimum Values Jim Lambers MAT 280 Spring Semester 2009-10 Lecture 8 Notes These notes correspond to Section 11.7 in Stewart and Section 3.3 in Marsden and Tromba. Maximum and Minimum Values In single-variable calculus,

More information

ColorCrack: Identifying Cracks in Glass

ColorCrack: Identifying Cracks in Glass ColorCrack: Identifying Cracks in Glass James Max Kanter Massachusetts Institute of Technology 77 Massachusetts Ave Cambridge, MA 02139 kanter@mit.edu Figure 1: ColorCrack automatically identifies cracks

More information

Image Enhancement: Frequency domain methods

Image Enhancement: Frequency domain methods Image Enhancement: Frequency domain methods The concept of filtering is easier to visualize in the frequency domain. Therefore, enhancement of image f ( m, n) can be done in the frequency domain, based

More information

Digital Image Processing. Prof. P. K. Biswas. Department of Electronics & Electrical Communication Engineering

Digital Image Processing. Prof. P. K. Biswas. Department of Electronics & Electrical Communication Engineering Digital Image Processing Prof. P. K. Biswas Department of Electronics & Electrical Communication Engineering Indian Institute of Technology, Kharagpur Lecture - 28 Colour Image Processing - III Hello,

More information

Implementation of Image Processing Algorithms on the Graphics Processing Units

Implementation of Image Processing Algorithms on the Graphics Processing Units Implementation of Image Processing Algorithms on the Graphics Processing Units Natalia Papulovskaya, Kirill Breslavskiy, and Valentin Kashitsin Department of Information Technologies of the Ural Federal

More information

Brightness and geometric transformations

Brightness and geometric transformations Brightness and geometric transformations Václav Hlaváč Czech Technical University in Prague Center for Machine Perception (bridging groups of the) Czech Institute of Informatics, Robotics and Cybernetics

More information

Armstrong Atlantic State University Engineering Studies MATLAB Marina Image Processing Primer

Armstrong Atlantic State University Engineering Studies MATLAB Marina Image Processing Primer Armstrong Atlantic State University Engineering Studies MATLAB Marina Image Processing Primer Prerequisites The Image Processing Primer assumes nowledge of the MATLAB IDE, MATLAB help, arithmetic operations,

More information

IMAGE MODIFICATION DEVELOPMENT AND IMPLEMENTATION: A SOFTWARE MODELING USING MATLAB

IMAGE MODIFICATION DEVELOPMENT AND IMPLEMENTATION: A SOFTWARE MODELING USING MATLAB Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 4, Issue. 8, August 2015,

More information

Mean-Shift Tracking with Random Sampling

Mean-Shift Tracking with Random Sampling 1 Mean-Shift Tracking with Random Sampling Alex Po Leung, Shaogang Gong Department of Computer Science Queen Mary, University of London, London, E1 4NS Abstract In this work, boosting the efficiency of

More information

Assessment. Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall

Assessment. Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall Automatic Photo Quality Assessment Presenter: Yupu Zhang, Guoliang Jin, Tuo Wang Computer Vision 2008 Fall Estimating i the photorealism of images: Distinguishing i i paintings from photographs h Florin

More information

A New Image Edge Detection Method using Quality-based Clustering. Bijay Neupane Zeyar Aung Wei Lee Woon. Technical Report DNA #2012-01.

A New Image Edge Detection Method using Quality-based Clustering. Bijay Neupane Zeyar Aung Wei Lee Woon. Technical Report DNA #2012-01. A New Image Edge Detection Method using Quality-based Clustering Bijay Neupane Zeyar Aung Wei Lee Woon Technical Report DNA #2012-01 April 2012 Data & Network Analytics Research Group (DNA) Computing and

More information

Computer Graphics. Course SS 2007 Antialiasing. computer graphics & visualization

Computer Graphics. Course SS 2007 Antialiasing. computer graphics & visualization Computer Graphics Course SS 2007 Antialiasing How to avoid spatial aliasing caused by an undersampling of the signal, i.e. the sampling frequency is not high enough to cover all details Supersampling -

More information

Computational Optical Imaging - Optique Numerique. -- Deconvolution --

Computational Optical Imaging - Optique Numerique. -- Deconvolution -- Computational Optical Imaging - Optique Numerique -- Deconvolution -- Winter 2014 Ivo Ihrke Deconvolution Ivo Ihrke Outline Deconvolution Theory example 1D deconvolution Fourier method Algebraic method

More information

1 Notes on Markov Chains

1 Notes on Markov Chains Notes on Markov Chains A Markov chain is a finite-state process that can be characterized completely by two objects the finite set of states Z = {z,, z n } and the transition matrix Π = [π ij i,j=,n that

More information

Sachin Patel HOD I.T Department PCST, Indore, India. Parth Bhatt I.T Department, PCST, Indore, India. Ankit Shah CSE Department, KITE, Jaipur, India

Sachin Patel HOD I.T Department PCST, Indore, India. Parth Bhatt I.T Department, PCST, Indore, India. Ankit Shah CSE Department, KITE, Jaipur, India Image Enhancement Using Various Interpolation Methods Parth Bhatt I.T Department, PCST, Indore, India Ankit Shah CSE Department, KITE, Jaipur, India Sachin Patel HOD I.T Department PCST, Indore, India

More information

Image Compression. Chapter 6 JORGE REBAZA

Image Compression. Chapter 6 JORGE REBAZA Chapter 6 Image Compression JORGE REBAZA One of the central issues in information technology is the representation of data by arrays of bits in the most efficient way possible, a never-ending quest for

More information

222 The International Arab Journal of Information Technology, Vol. 1, No. 2, July 2004 particular pixels. High pass filter and low pass filter are gen

222 The International Arab Journal of Information Technology, Vol. 1, No. 2, July 2004 particular pixels. High pass filter and low pass filter are gen The International Arab Journal of Information Technology, Vol. 1, No. 2, July 2004 221 A New Approach for Contrast Enhancement Using Sigmoid Function Naglaa Hassan 1&2 and Norio Akamatsu 1 1 Department

More information

Chapter 4: Time-Domain Representation for Linear Time-Invariant Systems

Chapter 4: Time-Domain Representation for Linear Time-Invariant Systems Chapter 4: Time-Domain Representation for Linear Time-Invariant Systems In this chapter we will consider several methods for describing relationship between the input and output of linear time-invariant

More information

Automated Sudoku Solver. Marty Otzenberger EGGN 510 December 4, 2012

Automated Sudoku Solver. Marty Otzenberger EGGN 510 December 4, 2012 Automated Sudoku Solver Marty Otzenberger EGGN 510 December 4, 2012 Outline Goal Problem Elements Initial Testing Test Images Approaches Final Algorithm Results/Statistics Conclusions Problems/Limits Future

More information

suggestive contours and abstracted shading Daniel Arias

suggestive contours and abstracted shading Daniel Arias suggestive contours and abstracted shading Daniel Arias LINES AND SHADES Shape, volume, shades and texture in drawing lines as nonphotorealistic rendering technique Luis Caballero Suggestive contours Many

More information

Segmentation & Clustering

Segmentation & Clustering EECS 442 Computer vision Segmentation & Clustering Segmentation in human vision K-mean clustering Mean-shift Graph-cut Reading: Chapters 14 [FP] Some slides of this lectures are courtesy of prof F. Li,

More information

Robert Collins CSE598C, PSU. Introduction to Mean-Shift Tracking

Robert Collins CSE598C, PSU. Introduction to Mean-Shift Tracking Introduction to Mean-Shift Tracking Appearance-Based Tracking current frame + previous location likelihood over object location appearance model (e.g. image template, or Mode-Seeking (e.g. mean-shift;

More information

3D Scanner using Line Laser. 1. Introduction. 2. Theory

3D Scanner using Line Laser. 1. Introduction. 2. Theory . Introduction 3D Scanner using Line Laser Di Lu Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute The goal of 3D reconstruction is to recover the 3D properties of a geometric

More information

Image Enhancement - Frequency Domain

Image Enhancement - Frequency Domain Image Processing - Lesson 7 Image Enhancement - Frequency Domain Low Pass Filter High Pass Filter Band pass Filter Blurring Sharpening Image FFT Filtered FFT -1 Transform Transform Filtered Image filter

More information

Antialiasing. CS 319 Advanced Topics in Computer Graphics John C. Hart

Antialiasing. CS 319 Advanced Topics in Computer Graphics John C. Hart Antialiasing CS 319 Advanced Topics in Computer Graphics John C. Hart Aliasing Aliasing occurs when signals are sampled too infrequently, giving the illusion of a lower frequency signal alias noun (c.

More information

The Image Deblurring Problem

The Image Deblurring Problem page 1 Chapter 1 The Image Deblurring Problem You cannot depend on your eyes when your imagination is out of focus. Mark Twain When we use a camera, we want the recorded image to be a faithful representation

More information

Math 56 Homework 4 Michael Downs

Math 56 Homework 4 Michael Downs Fourier series theory. As in lecture, let be the L 2 -norm on (, 2π). (a) Find a unit-norm function orthogonal to f(x) = x on x < 2π. [Hint: do what you d do in vector-land, eg pick some simple new function

More information

Lecture 4: Thresholding

Lecture 4: Thresholding Lecture 4: Thresholding c Bryan S. Morse, Brigham Young University, 1998 2000 Last modified on Wednesday, January 12, 2000 at 10:00 AM. Reading SH&B, Section 5.1 4.1 Introduction Segmentation involves

More information

MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0,

MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0, MATH 23: SYSTEMS OF LINEAR EQUATIONS Systems of Linear Equations In the plane R 2 the general form of the equation of a line is ax + by = c and that the general equation of a plane in R 3 will be we call

More information

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB. Sohail A. Dianat. Rochester Institute of Technology, New York, U.S.A. Eli S.

ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB. Sohail A. Dianat. Rochester Institute of Technology, New York, U.S.A. Eli S. ADVANCED LINEAR ALGEBRA FOR ENGINEERS WITH MATLAB Sohail A. Dianat Rochester Institute of Technology, New York, U.S.A. Eli S. Saber Rochester Institute of Technology, New York, U.S.A. (g) CRC Press Taylor

More information

OBJECT TRACKING USING LOG-POLAR TRANSFORMATION

OBJECT TRACKING USING LOG-POLAR TRANSFORMATION OBJECT TRACKING USING LOG-POLAR TRANSFORMATION A Thesis Submitted to the Gradual Faculty of the Louisiana State University and Agricultural and Mechanical College in partial fulfillment of the requirements

More information

Lecture 14. Point Spread Function (PSF)

Lecture 14. Point Spread Function (PSF) Lecture 14 Point Spread Function (PSF), Modulation Transfer Function (MTF), Signal-to-noise Ratio (SNR), Contrast-to-noise Ratio (CNR), and Receiver Operating Curves (ROC) Point Spread Function (PSF) Recollect

More information

Deinterlacing/interpolation of TV signals. Elham Shahinfard Advisors: Prof. M. Ahmadi Prof. M. Sid-Ahmed

Deinterlacing/interpolation of TV signals. Elham Shahinfard Advisors: Prof. M. Ahmadi Prof. M. Sid-Ahmed Deinterlacing/interpolation of TV signals Elham Shahinfard Advisors: Prof. M. Ahmadi Prof. M. Sid-Ahmed Outline A Review of some terminologies Converting from NTSC to HDTV; What changes need to be considered?

More information

Frequency domain filtering fundamentals

Frequency domain filtering fundamentals Frequency domain filtering fundamentals by Gleb V. Tcheslavski: gleb@ee.lamar.edu http://ee.lamar.edu/gleb/dip/index.htm Spring 2008 ELEN 4304/5365 DIP 1 Preliminaries For a digital image f(x,y) the basic

More information

Digital Image Processing (CS/ECE 545) Lecture 2: Histograms and Point Operations (Part 1)

Digital Image Processing (CS/ECE 545) Lecture 2: Histograms and Point Operations (Part 1) Digital Image Processing (CS/ECE 545) Lecture 2: Histograms and Point Operations (Part 1) Prof Emmanuel Agu Computer Science Dept. Worcester Polytechnic Institute (WPI) Histograms Histograms plots how

More information

Feature Tracking and Optical Flow

Feature Tracking and Optical Flow 02/09/12 Feature Tracking and Optical Flow Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Many slides adapted from Lana Lazebnik, Silvio Saverse, who in turn adapted slides from Steve

More information

Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold:

Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold: Linear Systems Example: Find x, x, x such that the following three equations hold: x + x + x = 4x + x + x = x + x + x = 6 We can write this using matrix-vector notation as 4 {{ A x x x {{ x = 6 {{ b General

More information

Face Recognition using Principle Component Analysis

Face Recognition using Principle Component Analysis Face Recognition using Principle Component Analysis Kyungnam Kim Department of Computer Science University of Maryland, College Park MD 20742, USA Summary This is the summary of the basic idea about PCA

More information

CSCI 445 Amin Atrash. Ultrasound, Laser and Vision Sensors. Introduction to Robotics L. Itti & M. J. Mataric

CSCI 445 Amin Atrash. Ultrasound, Laser and Vision Sensors. Introduction to Robotics L. Itti & M. J. Mataric Introduction to Robotics CSCI 445 Amin Atrash Ultrasound, Laser and Vision Sensors Today s Lecture Outline Ultrasound (sonar) Laser range-finders (ladar, not lidar) Vision Stereo vision Ultrasound/Sonar

More information

An Iterative Image Registration Technique with an Application to Stereo Vision

An Iterative Image Registration Technique with an Application to Stereo Vision An Iterative Image Registration Technique with an Application to Stereo Vision Bruce D. Lucas Takeo Kanade Computer Science Department Carnegie-Mellon University Pittsburgh, Pennsylvania 15213 Abstract

More information

Real Time Traffic Light Control System (Hardware and Software Implementation)

Real Time Traffic Light Control System (Hardware and Software Implementation) International Journal of Electronic and Electrical Engineering. ISSN 0974-2174, Volume 7, Number 5 (2014), pp. 505-510 International Research Publication House http://www.irphouse.com Real Time Traffic

More information

Monte Carlo Method: Probability

Monte Carlo Method: Probability John (ARC/ICAM) Virginia Tech... Math/CS 4414: The Monte Carlo Method: PROBABILITY http://people.sc.fsu.edu/ jburkardt/presentations/ monte carlo probability.pdf... ARC: Advanced Research Computing ICAM:

More information

Computer Vision - part II

Computer Vision - part II Computer Vision - part II Review of main parts of Section B of the course School of Computer Science & Statistics Trinity College Dublin Dublin 2 Ireland www.scss.tcd.ie Lecture Name Course Name 1 1 2

More information

Enhanced High Resolution Colour Image Using Discrete and Stationary Wavelet Transform

Enhanced High Resolution Colour Image Using Discrete and Stationary Wavelet Transform Enhanced High Resolution Colour Image Using Discrete and Stationary Wavelet Transform Sri M.V. Bhavani Sankar 1, Sri K. Suresh Babu 2 1 (Professor of Dept of ECE, DVR & Dr HS MIC college of Technology,

More information