Spatial Transforms 2

Spatial Transforms Reading: Chapter 6 ECE/OPTI 531 Image Processing Lab for Remote Sensing Spatial Transforms Introduction Convolution and Linear Filters Spatial Filtering Fourier Transforms Scale-Space Transforms Summary Spatial Transforms 2 1

Introduction Spatial transforms provide a way to access image information according to size, shape, etc. Spatial transforms operate on different scales Local pixel neighborhood (convolution) Global image (Fourier filters) All scales (scale-space filters) Image Model for Spatial Filtering Any digital image can be written as the sum of two images, The Low-Pass (LP) image contains the large area variations, which determines the global image contrast ( macro-contrast ) The High-Pass (HP) image contains the small area variations, which determines the sharpness and local image contrast ( micro-contrast ) Spatial Transforms 3 LP and HP Image Components image(x,y) - LP(x,y) = HP(x,y) 3 3 neighborhood 7 7 neighborhood LP and HP image components can be calculated by either convolution or Fourier filters Spatial Transforms 4 2

Spatial Transforms Introduction Convolution and Linear Filters Spatial Filtering Fourier Transforms Scale-Space Transforms Summary Spatial Transforms 5 Convolution Filters Local processing within a moving window output at: pixel 2, row2 pixel 3, row 2 pixel 2, row 3 Result of the calculation at each location is the output value at the center pixel of the moving window Processing within the window uses the original pixels, not the previously-calculated values Spatial Transforms 6 3

Linear Filters Any type of calculation can be done within the moving window If the calculation satisfies the principal of superposition (Chapter 3, Section 3.3.1), then the filter is termed linear The output of the sum of two inputs is equal to the sum of their respective outputs Discrete form of 2-D convolution Symbolically, Spatial Transforms 7 2-D Convolution Alternatively, we can write for a W x W y window, which emphasizes that the convolution is a weighting of local pixels 2-D Convolution Procedure 1. flip the window in x and y 2. shift the window 3. multiply the window weights times the corresponding image pixels 4. add the weighted pixels and write to the output pixel 5. repeat steps 2-3-4 (shift-multiply-add) until finished Spatial Transforms 8 4

Cascaded Linear Filters Sequential linear filters can be cascaded into a single filter Because convolution operator does not depend on the order applied (cummutative and associative) Example with two filters where w net is the equivalent filter, which can be precomputed efficiently because the kernels are generally much smaller than the image Spatial Transforms 9 Box-Filter Algorithm Very efficient for certain configurations of window weights For column-sums, the weights within each window row must be constant (but do not have to be equal from row-to-row) Computational savings varies with window size, but at least a factor of 2 Box-Filter Algorithm (along rows) 1. generate column sum of weighted input pixels 2. add column sums to obtain current output pixel (leftmost output pixel in row only) 3. move window one pixel along row 4. subtract leftmost column sum and add new rightmost column sum to obtain current output pixel 5. repeat steps 3 and 4 until done Spatial Transforms 1 5

Box-Filter Algorithm (cont.) Graphical depiction of column calculation C 1 C 2 C 3 C 1 C 2 C 3 C 4 Spatial Transforms 11 Border Region There is a problem with the moving window when it runs out of pixels near the image border Several possible solutions: repeat the nearest valid output pixel reflect the input pixels outside the border and calculate the convolution reduce the window size set the border pixels to zero or mean image DN wrap the window around to the opposite side of the image (circular convolution, same as produced by Fourier filtering) All solutions are a trick to get around the problem; no real solution Spatial Transforms 12 border region for 3x3 window 6

Spatial Transforms Introduction Convolution and Linear Filters Spatial Filtering Fourier Transforms Scale-Space Transforms Summary Spatial Transforms 13 Spatial Filters Low-Pass Filters (LPF) Preserve slowly varying spatial details (signal mean) and smooth sudden transitions (edges, noise) Simple example: 3-pixel window with equal weights (moving average) [1/3, 1/3, 1/3] Weights sum to one High-Pass Filters (HPF) Remove the signal mean and preserve edges Complimentary to the LPF [-1/3, 2/3, -1/3] = [, 1, ] - [1/3, 1/3, 1/3] Weights sum to zero Spatial Transforms 14 7

Example: 1-D LPF and HPF 14 15 12 input input 1 1x3 LP 1 1x3 HP 8 DN DN 5 6 4 2 1 2 3 4 5 6-5 1 2 3 4 5 6 index index 14 15 12 input input 1 1x7 LP 1 1x7 HP 8 DN DN 5 6 4 2 1 2 3 4 5 6 index Spatial Transforms 15-5 1 2 3 4 5 6 index 2-D LPF and HPF Kernels Weight array for pixels within the moving window is known as the kernel of the spatial filter Spatial Transforms 16 8

Filtered Image Histograms number of pixels 1 8 6 4 2-1 -5 5 1 15 2 DN 1 Original LP-component LP-filtered images same mean smaller standard deviation similar histogram HP-filtered images zero mean much smaller standard deviation nearly symmetric histogram 35 HP-component number of pixels 8 6 4 2 number of pixels 3 25 2 15 1 5-1 -5 5 1 15 2 DN Spatial Transforms 17-1 -5 5 1 15 2 DN High-Boost Filters (HBF) Corresponds to a boosting of the high spatial frequency components (details) of the image Kernel consists of edges extracted with a HPF and boosted by a gain K, then added back to the original DN at the center of the window Spatial Transforms 18 9

Example: HBF Varying K Spatial Transforms 19 Directional Filters Emphasize (HP) or suppress (LP) spatial features in a given direction orientation of window Spatial Transforms 2 Directional enhancements in 3 directions 1

Gradient Filters Find image gradient (derivative) in any direction by combining the x and y gradients in a vector calculation y g y g g! g x x Spatial Transforms 21 Example: Edge Detection Gradient magnitude of different filters Thresholding Roberts gradient to find edges original Prewitt Roberts DN T = 6 Roberts Sobel Spatial Transforms 22 DN T = 9 DN T = 12 11

Statistical Filters Similar to convolution with moving window, but output is a statistical quantity, e.g. local minimum or maximum local standard deviation local histogram mode (DN at peak) Morphological filters Application to spatial segmentation and noise filtering Operations are composed of two basic elements similar to minimum and maximum filters, but defined for dark objects in binary images dilation (minimum) erosion (maximum) Two filters applied in series to binary image yield opening = dilation[erosion(binary image)] Eq. 6 9 closing = erosion[dilation(binary image)] Eq. 6 1 Can use non-rectangular windows to achieve template matching ( structuring element ) Spatial Transforms 23 Example: Morphological Filters original local minimum local maximum thresholded original dilation erosion closing opening Spatial Transforms 24 12

Median Filter Output at each pixel is the median of the DNs within the window insensitive to outliers suppresses impulse (single pixel or line) noise 1-D example with 5-pixel moving window input = 1, 12, 9, 11, 21, 12, 1, 1 output =.., 11, 12, 11, 11,.. 14 12 1 input 1X3 median 8 DN 6 4 2 1 2 3 4 5 6 index Spatial Transforms 25 Spatial Transforms Introduction Convolution and Linear Filters Spatial Filtering Fourier Transforms Scale-Space Transforms Summary Spatial Transforms 26 13

Fourier Transforms Old theory (18th century), with wide application to signal analysis Represent a function as a linear combination (superposition) of basis functions, namely sines and cosines Fourier Synthesis The two-component image model is a simple example Extend to many components, i.e. individual sines and cosines Classic example is synthesis of a 1-D square wave with a sum of sines and cosines In our case, the functions are 2-D images, but the synthesis is the same An image is a sum of 2-D sine and cosine components The more components included in the sum, the better the synthesis Spatial Transforms 27 Square Wave Synthesis 1.2 1.8 Ideal square wave amplitude.6.4.2 -.2 5 1 15 2 position.8.6.4 fundamental third harmonic fifth harmonic Sine wave components amplitude.2 -.2 -.4 -.6 -.8 5 1 15 2 position 1.2 1.8 DC+fundamental +third harmonic +fifth harmonic Partial component sum amplitude.6.4.2 Spatial Transforms 28 -.2 5 1 15 2 position 14

2-D Image Synthesis synthesized image error image RMS error (DN) synthesized image error image RMS error DN DC 127.5 DC 8.96 + 2 components 74.2 + 12 components 6.45 + 8 components 43.1 + 84 components 4.54 + 18 components 29.1 + 22 components 3.7 + 32 components 16 + 42 components 1.67 + 144 components + 248 components Spatial Transforms 29 Discrete Fourier Transform 2-D discrete Fourier series (synthesis) inverse Fourier transform complex exponential is the sum of a sine and cosine 2-D discrete Fourier transform (analysis) forward Fourier transform Spatial Transforms 3 15

DFT (cont.) Eq. 6 14 and 6 16 are the Discrete Fourier Transform (DFT) pair f is in the spatial domain and F is in the spatial frequency domain The arrays in the DFT are assumed periodic in both domains Fig. 6 18 example postage stamp replication of arrays Image Domain Spatial Transforms 31 Spatial Frequency Domain Frequency Domain Relations among frequency domain parameters spatial frequency intervals (not normalized) spatial frequency intervals (normalized) spatial frequencies (normalized) Spatial Transforms 32 16

Fourier Components The Fourier transform produces a complex array, with real and imaginary components A complex number can also be written in terms of amplitude, A kl, and phase, φ kl Both components are important Amplitude determines contrast/brightness Phase determines location Spatial Transforms 33 Fourier Components (cont.) inverse Fourier transform of only one component Re(F kl) A kl phase only (A kl = 1) amplitude only (! kl = ) Im(F kl) Spatial Transforms 34! kl 17

Fourier Filters Filtering in the frequency domain is the product of two functions W is called the transfer function, or simply filter W acts like a mask on the image frequency components F, modifying their amplitudes and phases for discrete case, applies only to circular convolution Writing in the amplitude/phase representation W is called the Modulation Transfer Function (MTF) and is the amplitude filter Spatial Transforms 35 Fourier Filter Data Flow original image FT image spectrum F kl filtered image spectrum G kl FT -1 filtered image zero-padded window FT transfer function W kl spatial domain Fourier domain spatial domain Spatial Transforms 36 18

Transfer Functions An ideal LPF in the spatial frequency domain has uniform MTF, It transmits all frequency components below the cutoff frequency k c unchanged, and removes all higher frequency components Spatial Transforms 37 Example: MTFs MTFs for Truncated 3x3 Box Filters MTF Comparison for LPFs Gaussian LPF HPF HBF 3x3 Gaussian 3x3 Box Filter Spatial Transforms 38 19

Power Spectrum The square of the Fourier amplitude Indicates the strength of each frequency component Certain image spatial patterns have distinctive power spectra spatial description periodic patterns linear, quasi-periodic features nonlinear, aperiodic features spatial frequency description high amplitude spikes, localized at the frequencies of the patterns high amplitude line through DC, oriented orthogonal to the spatial patterns high amplitude cloud, primarily at lower frequencies Spatial Transforms 39 Example: Power Spectra desert streets fields railroad Spatial Transforms 4 2

Spatial Transforms Introduction Convolution and Linear Filters Spatial Filtering Fourier Transforms Scale-Space Transforms Summary Spatial Transforms 41 Scale-Space Transforms Access an image at different resolutions (scales) Image pyramids Define a reduce operation that incorporates resolution reduction and downsampling to smaller image L 3 L 5 L 4 L 3 L 2 L 2 L 1 L 1 L L Spatial Transforms 42 21

Reduction Filters The Burt-Adelson REDUCE function (convolution weights) is separable where w in each direction is parametric function of a Gaussian-like if a=.4, Spatial Transforms 43 Reduction Comparison Gaussian pyramid produces smoother (AKA wellbehaved ) reducedresolution images box pyramid Gaussian pyramid Spatial Transforms 44 22

Expansion EXPAND the REDUCEd image by interpolation REDUCE EXPAND level L-1 * w w down-sample up-sample level L Spatial Transforms 45 Net Convolution Filter level 3 29 x 29 level 2 13 x 13 level 1 Spatial Transforms 46 5 x 5 23

Wavelet Transforms level 2 Similar to resolution pyramids Successive convolutions with LPFs and HPFs to decompose image into components at different resolution scales down-sample rows filter columns w H H x H y H x L y L x H y L x L y * * w * * L w H w L level 1 down-sample columns filter rows w H * * w L level Spatial Transforms 47 Wavelet Components Calculation of one of the 4 wavelet components at each level level level 1 L x H y down-sample * w L w H L x down-sample Spatial Transforms 48 24

Spatial Transforms Introduction Convolution and Linear Filters Spatial Filtering Fourier Transforms Scale-Space Transforms Summary Spatial Transforms 49 Summary Convolution and Fourier transform filtering are equivalent (given the same border algorithm) Small neighborhood convolution filters can accomplish noise removal (LPF, median filter), edge enhancement (HPF, HBF, gradient filter) Scale-space filters combine convolution filters with resolution pyramid Examples of scale-space filters include Gaussian (LPF), Laplacian (HPF) and wavelets (LPF and HPF) Spatial Transforms 5 25