jorge s. marques image processing

Size: px

Start display at page:

Download "jorge s. marques image processing"

Teresa McDowell
7 years ago
Views:

1 image processing

2 images

3 images: what are they?

4 what is shown in this image? What is this?

5 what is an image images describe the evolution of physical variables (intensity, color, reflectance, condutivity) in a plane or in a 3D volume.

6 coordinate systems y y y x x x most common choice

7 discrete images Most electronic sensors produce discrete images defined on a grid of points n sccd sensor m f(m,n) pixel pixel picture element each pixel is characterized by its position and value

102 110 97 106 109 112 113 109 105 107 101 108 116 115 111 109 107 109 112 113 101 119 128 143 121 122 114 114 115 115 113 119 146 154 115 110 116 pixel 125

8 image types gray scale images binary images multi-spectral images color images

9 topological concepts what is the neighborhood of a pixel(i,j)? neighborhood 4 N4( i, j) {( i 1, j),( i 1, j),( i, j 1),( i, j 1)} neighborhood 8 N8 ( i, j) {( i ( i, j 1),( i 1, j 1),( i 1, j),( i 1, j 1),( i 1, j 1),( i, j 1), 1, j),( i 1, j 1)}

10 connectivity y x are x, y connected? NO binary image conectivity: two pixels x, y of a binary image u(m,n) are connected iif (if and only if) there a sequence of points z 1, z 2,..., z n : 1 z n x, z y i i 1 z, z are neighbors i { 1, 2,..., n 1} i u( z ) 1 i 1,...,n

u(m,n) are connected iif (if and only if) there a sequence of

11 regions regions Region R is a set of fully connected pixels i.e., x, y R, x,y are connected A maximal set of connected pixels is called a connected component.

12 extraction of connected components The algorithm assigns different labels to pixels belonging to different connected components ssign the label of the previous neighbors if they are equal If the neighbors have different labels create na equivalence 12 table

previous neighbors if they are equal If the neighbors have different labels

13 boundary The boundary of a region R is the set of R pixels whose neighborhood is not contained in R.

14 linear filtering

15 examples original blurred sharpened edge preserving blurred

16 noise reduction how can we remove the noise in an (homogeneous) image? idea: replace each pixel by an average g(i, j) in 1 2 ( 2N1) kin j N l j N f(k, l) N=1 2N+1 2N+1 N=2 mean filter

17 gaussian filter filtered image g(i, j) k, l G(k i,l j)f(k, l) 2 2 k l 2 2 (, l) C e G k f(i,j) j g(i,j) j weighted average i i

18 Gaussian filtering orig. =0.2 =1 =2 the gaussian filter destroys the transitions!!

19 linear filtering ), ( ), ( ), ( ), (, ), ( j i f j M i l k f j l i M k l k j i g ), ( j i M mask the mask/impulse response is shifted by an amount (i,j) mask / impulse response f g i j ), ( ) *, ( ), ( ), (, ), ( j i f j i h l k f l j k i h l k j i g ), ( ), ( j i M j i h impulse response 2D convolution sum 2D correlation

impulse response f g i j ), ( ) *, ( ), ( ), (, ), ( j i f j i h l k f l j k i h

20 Separable filters Separable filters have the following property: M( i, j) M1( i) M2( j), h( i, j) h1( i) h2( j) Separable filters can be implemented in a very efficient way. Algorithm: 1. filter all the image columns using filter h 1 (1D) 2. filter all the lines of the previous image using filter h 2 (1D) original image column filtering line filtering

Algorithm: 1. filter all the image columns using filter h 1 (1D) 2.

21 border effects

22 matrix notation Linear filtering can be written in matrix notation. g Hf g, f p IR, pp H IR g, f are column vectors with all the pixel intensities; H is a (huge) square matrix; p is the number of pixels in the image. This notation is very elegant but image filtering cannot be implemented in this way.

23 examples M(i,j) = M(i,j) = weighted average of neighboring pixels vertical difference, horizontal mean

24 example J( m, n) h( m, n) I( m, n) h( m, n) δ( m, n) G( m, n) original filtered

25 1 st order derivatives: gradient vector y f x x x f x g x g x g ) ( ) ( 2 1 ) ( ) ( ) ( gradient vector: ) ( ) ( arctan ) ( ) ( ) ( ) ( x g x g x g x g x g x g x=(x 1,x 2 ) continuous variables f(x) continuous image length and direction

26 gradient discretization 1) Derivatives can be approximated by finite differences I( m 1, n) I( m 1, n) g1( m, n) 2 I( m, n 1) I( m, n 1) g2( m, n) filtragem linear! 2) a better choice: Sobel masks High pass filtering in one direction and low pass filtering in the orthogonal direction

27 example

28 example

29 example: gradient g 1 g 2 g

30 2nd order derivatives: Laplacian filter Laplacian of a 2D image f x, y = 2 f x f y 2 zero crossings are associated to transitions Since the derivatives increase noise, the image is girst filtered using a Gaussian filter G x, y f x, y = [ G x, y ] f x, y where G x, y = x2 + y 2 σ 4 2 σ2 G(x, y)

31 Directional filters: Gabor filters Some cells of the visual cortex are activated by directional stimula. Gabor filters approximate the behavior of these cells. Their impulse response is the multiplication of a sinusoid by a Gaussian envelope in a rotated 2D space. h 1 x, y = exp x 2 + γ 2 y 2 2σ 2 h 2 x, y = exp x 2 + γ 2 y 2 2σ 2 cos 2π x λ + ψ sin 2π x λ + ψ rotation x = x cos θ + y sin θ y = x sin θ + y cos θ

32 Directional analysis: difference of Gaussians Difference of Gaussians Skin lesion dermoscopy images Barata, Transactions on Biomedical Engineering, 2012

33 nonlinear filtering

34 linear filters reduce noise but they destroy image contours. Is it possible to solve this conflict i.e., removing the noise, preserving the contours?

35 median filter

36 1d median filter f filter g g( n) median{ f ( n M),..., f ( n M)} f g non linear! example

37 2d median filter f filter g g( m, n) median{ f ( m M, n N),..., f ( m M, n N)}

38 example original image

39 sequence filtering video sequence time median filter can be applied in space, in time or in space-time.

40 example time filtering output: background image

41 non-linear diffusion

42 non linear diffusion Perona & Malik how is this possible?

43 heat equation u t c u u( x, t) 2 u u 2 x 2 y temperature u 2 diffusion

44 linear diffusion t 0 =0 t 1 >0 t 2 >t 1 Solution of heat equation: Gσ ( x) σ 2t 1 2πσ 2 x 2 e 2σ 2 (check it!)

45 non linear diffusion definition: g g t g(x, 0) f(x) initial condition The equation has a unique solution g( x, t) ( G f ( x) t 2t * f )( x) t 0 0 ( G * I)( x) G ( x y) f ( y) dy R 2 x G ( ) x e 2 2 Gaussian filtering beautiful!! Linear diffusion leads to Gaussian filtering with parameter σ 2t

46 gaussian filtering time

47 Space-scale Many images contain structures with different sizes. Therefore it is useful to represent it as a family of images with increasingly less details. Space scale of an image f(x) is a family of images indexed by a parameter s { T s f, s 0} such that T T 0 f st f f T s ( T t f ) semi-group property

48 Gaussian space-scale The Gaussian space-scale is obtained by filtering the image using a Gaussian kernel and changing the parameter s from 0 to infinity. Ts f Gs * f The semi-group property holds. Prove it! Limitations: It destroys the contour information how can we overcome this difficulty? pode ser gerado através de um processo de difusão linear

49 Generalization g(x) - concentration mass flows from regions of high concentration to regions with smaller concentration taking the gradient opposite direction. Fick s law j cg Mass conservation law g t div j j(x,t) diffusion flux c(x,t) diffusion coefficient g(x,t) concentration homogeneous material: constant diffusivity. Perona & Malik: space dependent diffusivity Idea: filter homogeneous regions and not transitions

50 condutivity condutivity (Perona & Malik) c 1 c( x, t) g( x, t) g Condutivity depends on the concentration at a given position and time instant and changes along the time. In homogeneous regions ( f =0) condutivity is constant c=1. At the contour points, condutivity is almost zero.

51 non linear diffusion Diffusion equation (Perona & Malik) g t div (c(x, t) g) c( x, t) 1 1 g( x, t) 2 remark: div v A B x y v (A,B)

52 discretization Perona & Malik algorithm is based on a discretization of the diffusion equation in which the partial derivatives are replaced by first order differences. t c g c g c g c g 0 1/ 4 t1 t gm, n gm, n N N S S E E W W m, n N u m-1,n where u m,n-1 W u m,n E u m,n+1 N E u g m, n m, n u g m-1,n m,n1 - u - g m,n m,n S S u m, n g m, n u g m1,n m,n1 - u - g m,n m,n S u m+1,n c Nm,n g( ( g) m1/ 2,n )f m,n c Sm,n g( ( g) m1/ 2,n ) c Em,n g( ( g) m,n1/ 2 )f m,n c W m,n g( ( g) m,n1/ 2 )

53 example original Gaussian filter Perona-Malik

54 interpolation and pyramids

55 bilinear interpolation Data: f x, y x, y {0,1} 0 d h 1 1 f 0, δ = 1 δ f 0,0 + δf 0,1 f 1, δ = 1 δ f 1,0 + δf 1,1 f η, δ = 1 η f 0, δ + ηf 1, δ Replacing f η, δ = 1 η 1 δ f 0,0 + 1 η δf 0,1 +η 1 δ f 1,0 + ηδf 1,

56 Interpolation at equally spaced points Sometimes we wish to increase the resolution of images. This can be done by interpolation. f(i) 1D g(i) i g i = k f k h(i kr) 0 r 2r 3r i f(i) r d(i) h(i) g(i) 2D d i = f i r i = kr 0 otherwise g i = d i h i = d p h(i p) = f k h(i kr) p = kr k g i, j = k,l f k, l h(i kr, j lr)

57 Interpolation results filters bilinear bicubic (a=-1) bicubic (a=-0.5) windowed sinc extracted from Szeliski book

58 decimation Decimation reduces the image resolution by a factor k. It involves two steps: 1) lowpass filtering (anti-aliasing) and 2) subsampling. 1D f(i) f(i) h(i) d(i) r g(i) 0 r 2r 3r i d(i) 0 r 2r 3r i g(i) i d i = f k h(i k) k g i = d(ri) = f k h(ri k) k 2D g i, j = f k, l h(ri k, rj l) g i, j = f k, l h(i k/r, j l/r) k,l k,l h(k,l) is the decimation kernel if h(k,l) is the interpolation kernel

59 Gaussian pyramid (Burt & Adelson) lowpass filter

60 Laplacian pyramid (Burt & Adelson) reconstruction reconstruction Block diagram perfect reconstruction ineficient representation Szeliski book

61 Credits Some slides are based on: Richard Szeliski, Computer Vision: Algorithms and Application, Springer, 2011

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/

Convolution. 1D Formula: 2D Formula: Example on the web: http://www.jhu.edu/~signals/convolve/ Basic Filters (7) Convolution/correlation/Linear filtering Gaussian filters Smoothing and noise reduction First derivatives of Gaussian Second derivative of Gaussian: Laplacian Oriented Gaussian filters