A Histogram Modification Framework and Its Application for Image Contrast Enhancement

Transcription

1 A Histogram Modification Framework and Its Application for Image Contrast Enhancement IEEE Transactions on Image Processing, Vol. 18, No. 9, 2009 Tarik Arici, Salih Dikbas, and Yucel Altunbasak Presented by Jung Yul Choi School of Electrical Engineering and Computer Science Kyungpook National Univ.

2 Abstract General framework based on histogram equalization for image contrast enhancement Optimization problem that minimizes a cost function Conventional histogram equalization Effective technique for contrast enhancement Resulte of excessive contrast enhancement Adjusting level of contrast enhancement Introducing specifically designed penalty terms 2 / 37

3 Introduction Contrast enhancement Occurring poor contrast of image and video No reveal all the details in captured scene Washed-out and unnatural look Target of contrast enhancement Eliminating these problems Obtaining more visually-pleasing or informative image or both Broadly categorizing contrast enhancement techniques Direct methods and indirect methods 3 / 37

4 Direct methods Defining contrast measure and trying to improve it Indirect methods Improving the contrast through exploiting the underutilized regions of dynamic range Most methods in the literature Dividing into several subgroups 1. Decomposing image into high and low frequency signals 2. Histogram modification techniques 3. Transform-based techniques 4 / 37

5 Global contrast enhancement (GCE) Use of single mapping derived from the image Impossible enhancing local contrast One of most popular GCE techniques Histogram equalization» Effective technique to transform narrow histogram by spreading gray-level clusters in histogram» Excessively enhanced output image for some applications Local contrast enhancement (LCE) Use of neighborhood of each pixel to obtain local mapping function More computationally complex than GCE 5 / 37

6 Histogram modification techniques Obtaining through modification on HE Bi-histogram equalization Reducing mean brightness change Dualistic sub-image histogram equalization (DSIHE) Using median intensity instead of mean intensity Becoming problem when histogram has spikes One method to deal with histogram spikes Histogram low-pass filtering and modifying cumulation function of histogram Still sensitive to problem created by histogram spikes LCE methods 6 / 37

7 Recent method proposed by Wang and Ward (2007) Modifying image histogram by weighting and thresholding Before histogram equalization Gray-level grouping (GLG) Grouping histogram bins and redistributing groups iteratively Robust to histogram spikes Mainly designed for still images 7 / 37

8 Aforementioned techniques Performing well on some images Creating problems When sequence of images is enhanced When the histogram has spikes When natural looking enhanced image is strictly required Goal in this paper Obtaining visually pleasing enhancement method Low computational complexity Easily implementing on FPGAs or ASICs Working well with both video and still images 8 / 37

9 Contrast Enhancement Enhancement mapping function Histogram-based methods Obtaining image with histogram of uniform distribution Mapping function in discrete form n B Tn [ ] = (2 1) pj [ ] j= 0 where B is number of bits used to represent the pixel values, p[j] is normalized histogram, and n [0, 2 B -1] As uniform as possible, no exactly uniform (1) Because of the discrete nature of the pixel intensities 9 / 37

10 Black stretching and white stretching Making dark pixels darker, while bright pixels brighter Linear black and white stretching n sb, n b Tn [ ] = n gn [ ], b< n< w w + ( n w) sw, w n where b is maximum gray-level to be stretched to black, w is minimum gray-level to be stretched to white, g[n] is any function mapping intensities in between, and s b, s w are black and white stretching factors both of which are less than one (2) 10 / 37

11 Histogram Modification Fully exploit available dynamic range on HE Creating uniformly distributed output histogram Using cumulated histogram as its mapping function One problem with HE Large backward-difference values of T[n] Unusually large T[n]- T[n-1] Modifying input histogram Then accumulating histogram Already uniform input distribution Mapping function is T[n]=n» Identically mapping input to output 11 / 37

12 Lessening level of enhancement Altering input histogram Modified histogram h» Closer to uniformly distributed histogram u Solution of bi-criteria optimization problem Fining modified histogram h h h Making residual i h i ( h hi h u ) minλ + hh,, h, u R where i and, and λ varies over [0, ). (3) 12 / 37

13 Adjustable histogram equalization Obtaining analytical solution to (3) Using squared sum of the Euclidean norm ( 2 2 ) i 2 2 h = arg minλ h h + h u h (4) Quadratic optimization problem argmin ( λ( ) T T h = h hi )( h( hi) ) + h u h u h Solution of (5) hi + λu 1λ h = = hi + u 1λ1λ1λ (5) (6) 13 / 37

14 Example image and enhanced images Using modified histogram equalization Fig. 1. Modified histogram equalization results using (6) for image Door. (a) Original image, (b) enhanced image using (6) with λ = 0, (c) enhanced image using (6) with λ = 1, (d) enhanced image using (6) with λ = 2. Fig. 2. The mappings and histograms for Fig / 37

15 Problem of Tn [ *] having large slope Arising from spikes in the input histogram Observing sensitivity to spikes Because l 2 norm heavily penalizes large Using l 1 norm instead of l 2 norm in (4) ( 2 ) i 1 2 h = arg minλ h h + h u h (7) Transforming into quadratic programming problem h T T = arg minλ( t 1 + ) ( h u ) h u h where t, 1 R, t ( h h ) t Another way to deal with the histogram spikes i Using one more penalty term to measure the smoothness of h 15 / 37

16 Histogram smoothing Measuring histogram smoothness Backward-difference of histogram, h[i]-h[i-1] D = ( ) ( ) i minλγh h + h u + Dh (8) 16 / 37

17 Solution of (8) T ((1λ)γ(λ) ) 1 h = + I+ DD h+ u Low-pass filtering operation on averaged histogram T ((1λ)γ ) 1 1 S = + I+ DD 2γ(1λ) + + γ γ4γ(1λ) γ S = 0 γ4γ(1λ) γ Existence of computational complex i (9) (10) 17 / 37

18 Performance of histogram smoothing Fig. 3. Histogram smoothing results using (9) for image Palermo. (a) Original image, (b) enhanced image using (9) with γ=0 and λ=1, (c) enhanced image using (9) with γ=0 and λ=3, (d) enhanced image using (9) with γ=1000 and λ=1. Fig. 4. The mappings for the enhanced images given in Fig / 37

19 Weighted histogram approximation Large number of pixels of exactly same gray-level Due to large smooth areas in the image Average local variance of all the pixels Using to weight approximation error, h - h i ( λ( T T h hi W) ( h h) i + h u h u ) min ( ) ( ) where W R 256X256 is diagonal error weight matrix. Solution of (11) 1 h = (λ) Wλ) + ( I Wh + u» Computationally simpler than (9) i (11) (12) 19 / 37

20 Comparison of weighted histogram approximation and histogram smoothing Fig. 5. Comparison results of histogram smoothing and weighted histogram approximation for image Palermo. (a) Histogram smoothing using(9) with γ=1000 and λ=1, (b) weighted approximation using (12) with λ=1000. Fig. 6. Mappings for the enhanced images given in Fig / 37

21 Black and White (B&W) stretching Decreasing histogram bin length [0, b] and [w, 255] Incorporating B&W stretching into histogram modification Adding additional penalty term to (5) T T T B ( hλ( hi ) h( hi + ) h u h u +αh I h) min ( ) ( ) where I B is diagonal matrix. I B ( ii, ) = 1 for i {[0, b] [ w, 255]} Solution to minimization problem (1λ) λ) B h = + I+ αi ( h + u ( ) 1 i (13) (14) 21 / 37

22 Comparison of histogram smoothing with and without B&W Fig. 7. (a) Original image, (b) enhanced image using (9) with γ=1000 and λ=1, (c) enhanced image using (14) with γ=1000, λ=1, and α=5, (d) mappings for the two enhanced images in (b) and (c). 22 / 37

23 Low complexity histogram modification algorithm Histogram computation High complex computation Because of histogram spike problem Simple way instead of complex computation Using conditional probability of pixel where pi [ C] denotes probability of pixel having gray-level i given event C Robust noise Obtaining pi [ C] hi [] = pi [ C] by counting only those pixels 23 / 37

24 Adjusting the level of enhancement GCE histogram modification algorithm User controlled parameter 1λ h = h + u 1λ1λ + i + Measuring input contrast hi [] = pi [ C] 24 / 37

25 1 1λ+ Limitation of very low slope Modification of histogram B&W stretching 25 / 37

26 Results and discussion Assessment of image enhancement Hard task Absence of any accepted objective criterion Proposed method of quantitative measures Absolute mean brightness error (AMBE) Absolute difference between input and output mean Discrete entropy (H) Measuring content of image Measure of enhancement (EME) Dividing image into blocks Finding measure Min and Max intensity values in each block, and averaging them 26 / 37

27 Subjective assessment Gray-scale images Fig. 8. Results for image Beach. (a) Original image, (b) enhanced image obtained using HE, (c) enhanced image obtained using WTHE, (d) enhanced image obtained using the proposed algorithm. Fig. 12. (a) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dotted line indicates the no change mapping. 27 / 37

28 Fig. 9. Results for image Beach. (a) Original image, (b) enhanced image obtained using HE, (c) enhanced image obtained using WTHE, (d) enhanced image obtained using the proposed algorithm. Fig. 12. (b) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dotted line indicates the no change mapping. 28 / 37

29 Fig. 10. Results for image Beach. (a) Original image, (b) enhanced image obtained using HE, (c) enhanced image obtained using WTHE, (d) enhanced image obtained using the proposed algorithm. Fig. 12. (c) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dotted line indicates the no change mapping. 29 / 37

30 Color images Fig. 11. Results for image Beach. (a) Original image, (b) enhanced image obtained using HE, (c) enhanced image obtained using WTHE, (d) enhanced image obtained using the proposed algorithm. Fig. 12. (b) Solid line indicates the HE mapping, red dashed line indicates the WTHE mapping, blue dash-dotted line indicates the proposed method, and the dotted line indicates the no change mapping. 30 / 37

31 Fig. 13. Results for image Hats. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm. 31 / 37

32 Fig. 14. Results for image Window. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm. 32 / 37

33 Fig. 14. Results for image Island. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm. 33 / 37

34 Fig. 14. Results for image Face. (a) Original image, (b) Enhanced image obtained using HE, (c) Enhanced image obtained using WTHE, (d) Enhanced image obtained using the proposed algorithm. 34 / 37

35 Objective assessment Table. 1. Quantitative measurement results. AMBE denotes the absolute mean brightness error, H denotes the discrete entropy, and EME denotes the measure of enhancement 35 / 37

36 Complexity comparison Analyzing time complexities of HE, WTHE, and proposed algorithm for M x N image Algorithm Histogram computation Mapping function Obtaining result image Total HE O(MN) O(2 B ) O(MN) O(2MN + 2 B ) WTHE O(MN) O(2 B ) + O(2 B ) O(MN) O(2MN + 2 B+1 ) Proposed algorithm O(MN) O(2 B ) + O(2 B ) O(MN) O(2MN + 2 B+1 ) 36 / 37

37 Conclusion Low-complexity algorithm Suitable for video display applications Improving contrast of image and video No introducing visual artifacts No decreasing visual quality of image No introducing flickering for video applications 37 / 37

38 Norm Assigning length or size of vectors in vector space Euclidean norm x = x1 + x2 + + x n = T xx Taxicab norm or Manhattan norm p-norm or l p norm l 2 norm x x p 1 n = i= 1 x p n = x i i= 1 i 1 p x n 2 2 n 2 2 xi xi i= 1 i= 1 = = = T xx

39 Absolute Mean Brightness Error (AMBE) AMBE = E( X) E( Y) Gray level of input image p( x) = 1/( X X ) for X x X L L 1 Computing statistical expectation E( Y) = E( Y X X ) Pr( X X ) + E( Y X> X ) Pr( X> X ) 1 2 m m m m { E( Y X X ) E( Y X X )} = + > m m where X m is mean brightness of input image. E( Y X X ) = ( X + X )/2 m E( Y X> X ) = ( X + X )/2 0 m m m L 1 where X G = ( X 0 + X L-1 ) / 2 E( Y) = ( X + X )/2 m G 39 / 37

40 Measure of enhancement (EME) Let image x(m, n) be split into k 1 k 2 blocks w k, l(i, j) of sizes l 1 xl 2, and let α, β, and γ are fixed enhancement parameters. EME EME = maxχ( EME ( Φ)) αβλ,,, k, k 1 2 Φ { Φ} w w Imax;, ( Φ) Imin;, ( Φ) αβλ,,, k, k where kl and kl respectively are the minimum and maximum of image x(m, n) inside block w k,l, after processing block by Φ transform based enhancement algorithm. Function is sign function. χ 1 2 I ( Φ ) = 20log kk k2 k w 1 1 max; kl, w 1 2 l= 1 k= 1 Imin; kl, ( Φ) ( Φ) 40 / 37

41 41 / 37