Introduction TU Berlin November, 21st 2006
About Me Introduction Student at TU Berlin since 2002 eitz@cs.tu-berlin.de
Outline Introduction 1 Introduction Smoothing Filters Comparison 2 Intuition Mathematical 3 Pictures
Outline Introduction Smoothing Filters Comparison 1 Introduction Smoothing Filters Comparison 2 Intuition Mathematical 3 Pictures
Introduction Image Smoothing Filters Smoothing Filters Comparison Aim: Smooth image to reduce noise Problem: Details are reduced as well Examples Mean (average) filter Median filter Gaussian filter
Introduction Smoothing Filters - Mean Filter Smoothing Filters Comparison Replace pixel value by mean (average) of its neigbours 97 103 83 82 81 98 103 105 108 97 99 255 102 101 95 101 103 107 255 93 93 101 112 108 107 1/9 1/9 1/9 * 1/9 1/9 1/9 = 1/9 1/9 1/9 97 103 83 82 81 98 103 105 108 97 99 255 137 101 95 101 103 107 255 93 93 101 112 108 107 Figure: Computing mean of central pixel
Introduction Smoothing Filters - Median Filter Smoothing Filters Comparison Replace pixel value by median of its neigbours Variance in neigbouring values does not influence mean Sort values, find the middle one 97 103 83 82 81 98 103 105 108 97 99 255 102 101 95 101 103 107 255 93 93 101 112 108 107 101 102 103 103 105 107 108 255 255 97 103 83 82 81 98 103 105 108 97 99 255 105 101 95 101 103 107 255 93 93 101 112 108 107 Figure: Computing median of central pixel
Introduction Smoothing Filters - Gaussian Smoothing Filters Comparison Replace pixel value by weighted average Pixels near center of kernel are weighted higher Pixels near border of kernel are weighted lower Weighting function G (x, y) = 1 x 2 +y 2 2πσ 2 e 2σ 2
Gaussian Function Introduction Smoothing Filters Comparison 3 2.5 2 1.5 1 0.5 0-2 -1.5-1 -0.5 0 0.5 1 1.5 2-1 -0.5 0-1.5-2 0.5 1 1.5 2 Figure: Gaussian, σ = 0.25 Figure: Filtered, σ = 0.25
Gaussian Function Introduction Smoothing Filters Comparison 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0-2 -1.5-1 -0.5 0 0.5 1 1.5 2-1 -0.5 0-1.5-2 0.5 1 1.5 2 Figure: Gaussian, σ = 0.5 Figure: Filtered, σ = 0.5
Gaussian Function Introduction Smoothing Filters Comparison 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0-2 -1.5-1 -0.5 0 0.5 1 1.5 2-1 -0.5 0-1.5-2 0.5 1 1.5 2 Figure: Gaussian, σ = 1.0 Figure: Filtered, σ = 1.0
Gaussian Function Introduction Smoothing Filters Comparison 0.04 0.035 0.03 0.025 0.02 0.015 0.01-2 -1.5-1 -0.5 0 0.5 1 1.5 2-1 -0.5 0-1.5-2 0.5 1 1.5 2 Figure: Gaussian, σ = 2.0 Figure: Filtered, σ = 2.0
Gaussian Function Introduction Smoothing Filters Comparison 0.01 0.0095 0.009 0.0085 0.008 0.0075-2 -1.5-1 -0.5 0 0.5 1 1.5 2-1 -0.5 0-1.5-2 0.5 1 1.5 2 Figure: Gaussian, σ = 4.0 Figure: Filtered, σ = 4.0
Gaussian Function Introduction Smoothing Filters Comparison 0.0025 0.00248 0.00246 0.00244 0.00242 0.0024 0.00238 0.00236 0.00234 0.00232-2 -1.5-1 -0.5 0 0.5 1 1.5 2-1 -0.5 0-1.5-2 0.5 1 1.5 2 Figure: Gaussian, σ = 8.0 Figure: Filtered, σ = 8.0
Outline Introduction Smoothing Filters Comparison 1 Introduction Smoothing Filters Comparison 2 Intuition Mathematical 3 Pictures
Introduction Smoothing Filters Comparison Comparison of Mean, Median and Gaussian Figure: Original Figure: Mean, radius 6px
Introduction Smoothing Filters Comparison Comparison of Mean, Median and Gaussian Figure: Original Figure: Gaussian, σ = 4.0
Introduction Smoothing Filters Comparison Comparison of Mean, Median and Gaussian Figure: Original Figure: Median, radius 6px
Common Problems Introduction Smoothing Filters Comparison Aim Mean Blurs the image, removes simple noise, no details are preserved Gaussian Blurs the image, results related to the mean filter, preserves details only for small σ Median Preserves some details, good at removing strong noise We need a filter that only smooths regions but does not smooth edges
Outline Introduction Intuition Mathematical 1 Introduction Smoothing Filters Comparison 2 Intuition Mathematical 3 Pictures
Introduction Intuition Mathematical What Is a Bilateral Filter - Definition Some properties Convolution filter Smooth image but preserve edges Operates both in the domain and the range of the image Definition Bilateral Affecting or undertaken by two sides equally
Introduction Intuition Mathematical What Is a Bilateral Filter - Definition Some properties Convolution filter Smooth image but preserve edges Operates both in the domain and the range of the image Definition Bilateral Affecting or undertaken by two sides equally
Introduction - Basic Idea Intuition Mathematical Algorithm Idea Smooth as usual in the domain of the image (e.g. Gaussian) Do not smooth when pixels are not similar (edge) Similarity Function Determines the amount of smoothing Similar pixels: Strong smoothing Otherwise (edges): No smoothing Similarity based on human perception Simplest example: based on intensity values of pixel, two pixels considered similar if they have the same value
Introduction - Basic Idea Intuition Mathematical Algorithm Idea Smooth as usual in the domain of the image (e.g. Gaussian) Do not smooth when pixels are not similar (edge) Similarity Function Determines the amount of smoothing Similar pixels: Strong smoothing Otherwise (edges): No smoothing Similarity based on human perception Simplest example: based on intensity values of pixel, two pixels considered similar if they have the same value
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.9 * 1.0 * 0.9 Resulting Weight: 0.30 0.30 Weight * PixelValue: 0.30*0 *15 0.30*21 sum() normalize() 0 12 21 7 0 255 240 231 242 251
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.9 * 1.0 * 0.9 Resulting Weight: 0.30 0.30 Weight * PixelValue: 0.30*15 *21 0.30*7 sum() normalize() 0 12 14 7 0 255 240 231 242 251
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.9 * 1.0 * 0.9 Resulting Weight: 0.30 0.30 Weight * PixelValue: 0.30*21 *7 0.30*0 sum() normalize() 0 12 14 9 0 255 240 231 242 251
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.9 * 1.0 * 0.0 Resulting Weight: 0.30 0.0 Weight * PixelValue: 0.30*7 *0 0.0*255 sum() normalize() 0 12 14 9 3 255 240 231 242 251
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.0 * 1.0 * 0.9 Resulting Weight: 0.0 0.30 Weight * PixelValue: 0.0*0 *255 0.30*240 sum() normalize() 0 12 14 9 3 248 240 231 242 251
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.9 * 1.0 * 0.9 Resulting Weight: 0.30 0.30 Weight * PixelValue: 0.30*255 *240 0.30*231 sum() normalize() 0 12 14 9 3 248 242 231 242 251
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.9 * 1.0 * 0.9 Resulting Weight: 0.30 0.30 Weight * PixelValue: 0.30*240 *231 0.30*242 sum() normalize() 0 12 14 9 3 248 242 237 242 251
Introduction of a Picture Intuition Mathematical 0 15 21 7 0 255 240 231 242 251 Smoothing Weight: Similarity Weight: * 0.9 * 1.0 * 0.9 Resulting Weight: 0.30 0.30 Weight * PixelValue: 0.30*231 *242 0.30*251 sum() normalize() 0 12 14 9 3 248 242 237 241 251
Outline Introduction Intuition Mathematical 1 Introduction Smoothing Filters Comparison 2 Intuition Mathematical 3 Pictures
Introduction - Math Intuition Mathematical Formula J s = 1 f (p s) g (I p I s ) I p k s p Ω k s = p Ω f (p s) g (I p I s ) s coord. of center pixel, p coord. of current pixel, Ω set of all pixel coord. in the local neighbourhood (under kernel) J s resulting pixel intensity. I s, I p intensities of p and s f (p s) measures geometric distance between p and s g (I p I s ) measures photometric similarity betw. I p and I s
Introduction Domain Weighting Functions Intuition Mathematical Domain weighting Usually a standard Gaussian Filter f (p s) = e d(p s) 2 2σ d 2 d (p s) = p s = p 2 + s 2 d (p s) is the Euclidean distance between p and s
Introduction Range Weighting Function Intuition Mathematical Range weighting Usually Gaussian of the intensity difference g (I p I s ) = e δ(ip Is) 2 2σr 2 δ (I p I s ) = I p I s With I p I s suitable measure of the difference between two pixel values Simplest approach: Difference in intensity values I p I s = I p I s
Outline Introduction Pictures References 1 Introduction Smoothing Filters Comparison 2 Intuition Mathematical 3 Pictures
Lena Figure: Original Figure: σ d = 3.0, σ r = 3.0
Lena Figure: Original Figure: σ d = 6.0, σ r = 3.0
Lena Figure: Original Figure: σ d = 12.0, σ r = 3.0
Lena Figure: Original Figure: σ d = 12.0, σ r = 6.0
Lena Figure: Original Figure: σ d = 15.0, σ r = 8.0
Water Lily Figure: Original Figure: σ d = 3.0, σ r = 3.0
Water Lily Figure: Original Figure: σ d = 6.0, σ r = 3.0
Water Lily Figure: Original Figure: σ d = 12.0, σ r = 3.0
Water Lily Figure: Original Figure: σ d = 12.0, σ r = 6.0
Water Lily Figure: Original Figure: σ d = 15.0, σ r = 8.0
Chinese Handle
Chinese Handle
Chinese Handle
Chinese Handle
Chinese Handle
Peppers Figure: Original Figure: σ d = 3.0, σ r = 3.0
Peppers Figure: Original Figure: σ d = 6.0, σ r = 3.0
Peppers Figure: Original Figure: σ d = 12.0, σ r = 3.0
Peppers Figure: Original Figure: σ d = 12.0, σ r = 6.0
Peppers Figure: Original Figure: σ d = 15.0, σ r = 8.0
Questions Introduction Pictures References Any questions? Slides can be downloaded at http://user.cs.tu-berlin.de/ eitz References: [1, 2, 3, 4, 5]
Bibliography I Introduction Pictures References Barash, D. and Anisotropic Diffusion: Towards a Unified Viewpoint. Springer, 2000. Barash, D. Fundamental relationship between bilateral filtering, adaptivesmoothing, and the nonlinear diffusion equation. Pattern Analysis and Machine Intelligence, IEEE Transactions on 24, 6 (2002), 844 847.
Bibliography II Introduction Pictures References Durand, F., and Dorsey, J. Fast bilateral filtering for the display of high-dynamic-range images. Proceedings of the 29th annual conference on Computer graphics and interactive techniques (2002), 257 266. Fleishman, S., Drori, I., and Cohen-Or, D. Bilateral mesh denoising. ACM Transactions on Graphics (TOG) 22, 3 (2003), 950 953. Tomasi, C., and Manduchi, R. Bilateral filtering for gray and color images. Proceedings of the Sixth International Conference on Computer Vision (1998), 839 846.