UNIVERSITY OF ENGINEERING AND TECHNOLOGY TAXILA DEPARTMENT OF ELECTRICAL ENGINEERING

Transcription

1 UNIVERSITY OF ENGINEERING AND TECHNOLOGY TAXILA DEPARTMENT OF ELECTRICAL ENGINEERING DESIGN AND IMPLEMENTATION OF IMPROVED QUALITY LOW BIT RATE VIDEO CODING A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical Engineering by Gulistan Raja 03-UET/PhD-EE-14 Research Committee in charge: Prof. Dr. Muhammad Javed Mirza Supervisor Prof. Dr. Habibullah Jamal Prof. Dr. Muhammad Khawar Islam Dr. Shoab A. Khan 2008

2

3 Design and Implementation of Improved Quality Low Bit Rate Video Coding Copyright 2008 by Gulistan Raja All rights reserved iii

4 Dedicated to my family iv

5 SUMMARY Design and Implementation of Improved Quality Low Bit Rate Video Coding Gulistan Raja 03-UET/PhD-EE-14 Today s most video coding standards use block based discrete cosine transform coding schemes to exploit spatial redundancy. The basic approach is: partitioning of the whole image into blocks, transformation and quantization. Loss of correlation occurs between adjacent blocks due to coarse quantization at low bit rates. This introduces visually disturbing block discontinuities along block edges, known as blocking artifacts. The latest H.264/AVC video coding standard employs normative adaptive loop deblocking filter algorithm for reduction of blocking artifacts. Performance analysis of deblocking filter has proved its effectiveness for suppression of artifacts. However, it is highly computationally complex. Therefore, there is need to reduce this computing complexity to make it suitable for low bit rate applications, e.g., real time mobile video. Various attempts have been made to reduce computing complexity of deblocking algorithm but most of them deal with hardware implementation using efficient architecture. We propose an optimized deblocking algorithm based on motion activity of video sequences. v

6 First, we have done performance analysis of latest H.264/AVC with other video coding standards for low bit rate applications and the results show a significant performance gain of H.264/AVC in comparison with other standards. Second, we have shown that H.264/AVC deblocking filter is very effective in suppressing blocking artifacts generated at low bit rates. However, it takes one third computing resources of decoder due to high computational cost. The main cause of this enormously high computing complexity is boundary strength computations, which are primarily used to select one type of filter out of two filters: normal and strong. More than 90% of computational resources are spent on boundary strength computations in H.264/AVC deblocking filter. Third, by considering computing complexity reduction of H.264/AVC deblocking filter as an objective, a motion activity based deblocking algorithm is proposed. Based on sum of motion vectors at frame level, the thresholds have been set through experimentation for categorization of video sequences according to their motion activity into three groups as: (1) low motion (2) moderate motion (3) high motion. It has been observed through experimentation in proposed research that strong filter of H.264/AVC deblocking filter can be replaced by normal filter for low to moderate motion sequences. The new decision criteria for application of filter based on motion activity of video sequences has been proposed. As a result, boundary strength computations are not used for low to moderate motion sequences in proposed deblocking algorithm. Various simulations are conducted to evaluate the candidacy of the proposed technique. A significant reduction in average number of operations is achieved without losing subjective quality of the video. A reduction of 45.29% in average number of operations is attained. The objective and subjective results are in vi

7 conformity of the original H.264/AVC deblocking filter for low and moderate motion video sequences. The proposed research can be used for real time low bit rate video applications. For example, mobile video on portable devices, video telephony, video conferencing on Internet using low bandwidth lines. Keywords: Video coding, H.264/AVC, deblocking filter, motion activity, computing complexity. vii

8 ACKNOWLEDGMENTS First of all, I would like to express my overflowing gratitude to Almighty Allah for granting me wisdom, resources and strength to complete this work. The first acknowledgment goes to my supervisor, Prof. Dr. Muhammad Javed Mirza, for his invaluable guidance, constructive advise, accurate criticism and encouragement during the course of this research. I have great appreciation for Professor Mirza s wisdom as he has lended great support to me in academic matters. I would like to express my deep gratitude to members of my research committee, Prof. Dr. Habibullah Jamal, Prof. Dr. Muhammad Khawar Islam and Prof. Dr. Shoab A. Khan for their interesting discussions and helpful comments in the evaluation of this research work. They always encouraged me and were the significant force during my dissertation work. Special thanks to Tian Song, with whom I studied together during my master studies in Osaka University, Osaka, for his valuable comments and suggestions. I am grateful to Prof. Ahmad Khalil Khan for useful discussions, encouragement and motivation during the course of studies. I would also like to thank all my colleagues and friends especially Prof. Dr. Muhammad Amin, Prof. Dr. Umar Farooq, Prof. Dr. Zafrullah, Prof. Dr. Muhammad Ahmad, Prof. Tahir Nadeem Malik, Prof. Aftab Ahmad, Prof. Iram Baig, Prof. Dr. viii

9 Adeel Akram, Tahir Mahmood, Ilyas Ahmad, Amir Hanif, Zahid Suleman Butt, Riffat Asim Pasha and Dr. Mirza Jahanzaib for their encouragement. I am thankful to all the people who had given me support during my research especially Prof. Dr. Qaiser-uz-Zaman, Director ASR & TD, Zafar Iqbal Sabir, Admin Officer, ASR & TD office and their staff. At the end, my heartfelt appreciation is expressed to all the members of my family especially my mother and wife for their love, inspiration, patience, continuous support, encouragement and prayers during my PhD studies. ix

10 CURRICULUM VITA Education 1996 B.Sc. Electrical Engineering, University of Engineering and Technology, Taxila 2002 M.S. Information Systems Engineering, Osaka University, Osaka 2008 Ph.D. Electrical Engineering, University of Engineering and Technology, Taxila Professional Experience 1997 ~ 2003 Research Associate, Electrical Engineering Department, University of Engineering and Technology, Taxila 2000~2002 Intern, Synthesis Corporation, Osaka 2003 ~ to date Assistant Professor, Electrical Engineering Department, University of Engineering and Technology, Taxila Pertinent Publications 1. Gulistan Raja, M. J. Mirza, and T. Song, H.264/AVC Deblocking Filter based on Motion Activity in Video Sequences Journal of IEICE Electronics Express, Japan, Vol. 5, No. 19, 2008, pp Gulistan Raja and M. J. Mirza, A New Scheme of Suppressing Blocking Artifacts in H.264/AVC Deblocking Filter for Low Bit Rate Video Coding, World Scientific and Engineering Academy and Society Transactions on Circuits and Systems, Greece, Issue 1, Vol. 6, 2007, pp x

11 3. Gulistan Raja and M. J. Mirza, Evaluation of Loop Filtering for Reduction of Blocking Effects in Real Time Low Bit Rate Video Coding, MUET Research Journal of Engineering & Technology, Pakistan, Vol. 26, No. 3, 2007, pp Gulistan Raja and M. J. Mirza, In-Loop Deblocking Filter for JVT H.264/AVC, World Scientific and Engineering Academy and Society Transactions on Signal Processing, Greece, (selected paper from International Conference on Signal Processing, Robotics and Automation, ISPRA, 06, Madrid, Spain), Issue 2, Vol. 2, pp Gulistan Raja, M. J. Mirza, JVT H.264/AVC: Evaluation with Existing Standards for Low Bit Rate Video Coding, Proceedings of 17th IEEE International Conference on Microelectronics, Islamabad, Pakistan, December 13-15, 2005, pp Gulistan Raja, M. J. Mirza, Evaluation of Emerging JVT H.264/AVC with MPEG Video, Proceedings of 9th IEEE International Multi-topic Conference, Karachi, Pakistan, December 24-25, 2005, pp Gulistan Raja, M. J. Mirza, Performance Comparison of Advanced Video Coding H.264 Standard with Baseline H.263 and H.263+ Standards, Proceedings of 4th IEEE International Symposium on Communications & Information Technologies, Sapporo, Japan, October 26-29, 2004, pp xi

12 TABLE OF CONTENTS Summary... v Acknowledgments... viii Curriculum Vita... x Table of Contents... xii List of Figures...xiv List of Tables...xvi Chapter 1: Introduction Background Objectives Approach Thesis Outline... 4 Chapter 2: Literature Review Theory of Blocking Artifacts at Low Bit Rates Methods to Reduce Blocking Artifacts Deblocking Filters Categories of Techniques for Reducing Blocking Artifacts Literature Review for Deblocking Filters Summary of Salient Techniques used in Deblocking Filters Motion Activity Detection Metrics - A Deblocking Filters Perspective Review of Significant Motion Activity Detection Approaches Analysis of Motion Activity Detection Techniques xii

13 Chapter 3: Case Studies Analysis with respect to Low Bit Rate Video Coding Performance Analysis of H.264/AVC with Existing Standards H.264/AVC Profiles and Levels Main Blocks of H.264/AVC Test Environment and Simulation Results Evaluation of H.264/AVC Deblocking Filter H.264/AVC Loop Deblocking Filter Experimental Methodology and Results Chapter 4: Design and Implementation of Proposed Deblocking Filter for Improved Quality Low Bit Rate Video Coding Analysis of Strong Filter and Normal Filter Employment in H.264/AVC Deblocking Filter Classification using Motion Activity in Video Sequences Motion Vectors Thresholds for Motion Activity Proposed Deblocking Filter Experimental Environment Computational Complexity Comparison Objective Comparison Subjective Comparison Conclusions Future Recommendations References xiii

14 LIST OF FIGURES Fig. 2.1 Blocking Artifacts at 40 Kbps... 8 Fig. 2.2 Post processing for reduction of blocking artifacts... 9 Fig. 2.3 Loop filter for reduction of blocking artifacts Fig. 3.1 JVT H.264/AVC encoder Fig. 3.2 Rate distortion comparison of H.264/AVC at low bit rates with MPEG2 and MPEG Fig. 3.3 Rate distortion comparison of H.264/AVC at low bit rates with H.263 Baseline and H Fig. 3.4 Subjective comparison at low bit rats: QCIF CARPHONE frame 57 encoded at 22 Kbps Fig. 3.5 Subjective comparison at low bit rates: QCIF FOREMAN frame 134 encoded at 40 Kbps. 46 Fig. 3.6 Position of deblocking filter in H.264/AVC encoder Fig. 3.7 Filtering order at macroblock level Fig. 3.8 Boundary strength (bs) computation flowchart Fig. 3.9 H.264/AVC deblocking filter Fig Rate-PSNR Comparison at Low Bit Rates: with- & without deblocking filter.56 Fig Subjective comparison for various QCIF sequences Fig Subjective comparison for various QCIF sequences Fig Subjective comparison for various CIF sequences Fig Subjective comparison for various CIF sequences Fig. 4.1 QCIF CONTAINER at 30 Kbps: Use of (a) Normal Filter (b) Strong Filter Fig. 4.2 QCIF SALESMAN at 30 Kbps: Use of (a) Normal Filter (b) Strong Filter Fig. 4.3 QCIF MOTHER DAUGHTER at 30 Kbps: Use of (a) Normal Filter (b) Strong Filter...67 Fig. 4.4 QCIF FOOTBALL at 30 Kbps: Use of (a) Normal Filter (b) Strong Filter Fig. 4.5 Use of Strong and Normal Filter at Frame Level in H.264/AVC Deblocking Filter encoded at 30 Kbps (a) QCIF CONTAINER (b) QCIF SALESMAN xiv

15 Fig. 4.6 Use of Strong and Normal Filter at Frame Level in H.264/AVC Deblocking Filter encoded at 30 Kbps (a) QCIF MOTHER DAUGHTER (b) QCIF CARPHONE.. 70 Fig. 4.7 Use of Strong and Normal Filter at Frame Level in H.264/AVC Deblocking Filter encoded at 30 Kbps (a) QCIF FOREMAN (b) QCIF FOOTBALL Fig. 4.8 Thresholds for classification of QCIF video sequences Fig. 4.9 Thresholds for classification of CIF video sequences Fig Adjacent samples to vertical & horizontal edge Fig Proposed Deblocking Filter Fig Comparison of addition operations (a) QCIF sequences (b) CIF sequences Fig Comparison of shift operations (a) QCIF sequences (b) CIF sequences Fig Comparison operations (a) QCIF sequences (b) CIF sequences Fig Objective comparison between H.264/AVC deblocking filter and proposed deblocking filter for various (a) QCIF sequences (b) CIF sequences Fig Subjective comparison for various QCIF sequences Fig Subjective comparison for various QCIF sequences Fig Subjective comparison for various QCIF sequences Fig Subjective comparison for various QCIF sequences Fig CONTAINER frame 1 encoded at 35 Kbps Fig CIF BRIDGE frame 4 encoded at 40 Kbps Fig CIF MOTHER DAUGHTER frame 6 encoded at 40 Kbps Fig CIF HIGHWAY frame 5 encoded at 40 Kbps Fig CIF SILENT frame 3 encoded at 40 Kbps Fig CIF IRENE frame 14 encoded at 40 Kbps Fig CIF FOREMAN frame 9 encoded at 35 Kbps xv

16 LIST OF TABLES Table 2.1 Deblocking filters for various standards Table 2.2 Comparison of post- and loop filtering Table 2.3 Comparison of deblocking algorithms Table 2.4 SD thresholds of motion vector magnitude Table 3.1 Coding tools supported by baseline, main and extended profile Table 3.2 Objective comparison of H.264/AVC with MPEG-2 and MPEG-4 at different bit rates for various QCIF sequences Table 3.3 Objective comparison of H.264/AVC with H.263 Baseline and H.263+ at different bit rates for various QCIF sequences Table 3.4 Average luminance PSNR at different low bit rates for QCIF sequences withand without deblocking filter Table 3.5 Average luminance PSNR at different low bit rates for CIF sequences withand without deblocking filter Table 4.1 MV sum for QCIF video sequences Table 4.2 MV sum for CIF video sequences Table 4.3 Various parameters for experimental environment Table 4.4 Average number of operations spent on QCIF sequences using H.264/AVC deblocking filter and proposed filter Table 4.5 Average number of operations spent on CIF sequences using H.264/AVC deblocking filter and proposed filter Table 4.6 Computing complexity analysis of proposed filter with H.264/AVC deblocking filter Table 4.7 Average Luminance PSNR at different bit rates for QCIF sequences Table 4.8 Average Luminance PSNR at different bit rates for CIF sequences xvi

17 CHAPTER 1 Introduction 1.1 Background Imagine that you want to transmit or store a TV quality digital video. Transmission or storage capability of Mega bytes is required for 1 second of video and Giga bytes are required for 1-hour uncompressed (raw) video program. This requires enormously high data transmission and/or storage medium, which is beyond the capabilities of today s systems. Therefore, there is a need for compression to deal with this kind of high-bit rate data. Moreover, the demand for digital video communication applications such as video conferencing, video , network games and other value added services has increased considerably. However, transmission rates over public switched telephone networks (PSTN) and wireless networks are still very restricted due to bandwidth limitations. Consequently, separate international video coding standards have been recommended for different applications such as H.261 [1-2, 4], MPEG-1 [2-4], MPEG-2 [2, 5-6], H.263 [7], MPEG-4 [8], H.263+ [9-10]. These standards address wide range of applications having different requirements in terms of bit rates, picture quality, error resilience and delay, etc. The latest video coding standard, H.264/AVC is developed by Joint Video Team (JVT) that includes experts from Motion Picture Expert Group (MPEG) and ITU-T Video Coding Expert Group (VCEG). The official title of the new standard is Advanced Video Coding (AVC); however, it is widely known by its ITU document 1

18 number, H.264 or MPEG-4 Part 10. The final drafting work on the first version of the standard was completed in May of H.264/AVC supersedes previous video coding standards in almost every aspect. The salient enhancements made by this standard are: variable block size motion compensation with small block sizes, quarterpixel accurate motion compensation, multiple reference picture motion compensation, decoupling of referencing order from display order, weighted prediction skipped and direct motion influence and loop deblocking filtering [11]. The wide range of target applications that can be categorized as: (1) Broadcast over cable, satellite communication, cable modem, DSL, terrestrial communication, etc. (2) Storage on optical and magnetic devices, DVD etc. (3). Conversational services important networks. (4) Video-on-demand (5) multimedia streaming services over IDSN, cable mode, DSL, LAN, wireless Network etc. and multimedia messaging services over ISDN, DSL, Ethernet etc [12]. 1.2 Objectives H.264/AVC video coding standard along with other standards use block base transform coding scheme to exploit spatial redundancy. However, loss of correlation occurs between adjacent blocks due to coarse quantization at low bit rates. This produces visually disturbing discontinuities along the block edges, known as blocking artifacts. H.264/AVC employs normative adaptive loop deblocking filter for the reduction of blocking artifacts [13]. The filter is applied to the reconstructed frame in both, encoder and decoder. The filtered frames are used as reference frames for motion compensation for subsequent coded frame. Performance analysis of H.26/AVC deblocking filter shows that it reduces the blocking artifacts significantly at low bit rates [14-17]. However, it is highly computationally complex, as it takes 2

19 one-third of computing resources of the decoder [18].The main reason for high computing complexity of filter is heavy conditional processing on block edge and at pixel level required for filtering decision, and to select one type of filter out of two filters: normal and strong. Most of research reported in literature for reduction of blocking artifacts is by use of efficient architecture and hardware implementation of deblocking filter [19-22] but very little work has been reported for algorithmic optimization of deblocking algorithm. The main focus of our research is to reduce the computing complexity of deblocking algorithm for H.264/AVC video, so that it can be used for real time low bit rate applications like mobile video effectively. 1.3 Approach This thesis describes the design and implementation of reduced computing deblocking filter for low bit rate video coding. The novel idea of incorporating motion activity of video sequences in deblocking filter to reduce the computing complexity is proposed. It has been found that motion compensation vectors can be used to detect the motion activity of video sequences. Based on this criterion, different video sequences are categorized into three groups: low motion activity, moderate motion activity and high motion activity. The thresholds using absolute sum of motion vectors has been set to classify video sequences in these three groups. Using this criterion, new modified conditions for filtering edge pixels are implemented. This results in significant reduction in computational complexity of deblocking algorithm as decision to select between two types of filters (strong/normal filter) takes considerable computing operations. Experimental simulations conducted in our research show significant 3

20 reduction in computing complexity without loss of subjective quality of video for low to moderate motion video sequences. 1.4 Thesis Outline The rest of the thesis is organized as follows: Chapter 2 provides literature review for thesis. First the mathematical background for occurrence of blocking artifacts at low bit rates is introduced. Second, various schemes for reduction of blocking artifacts in literature is discussed. Third, we introduce some important methods used for detection of motion activity in video sequences with perspective of incorporating it into deblocking filters. In chapter 3, performance evaluations for low bit rate video coding are carried out. Initially, performance analysis of H.264/AVC standard with existing video coding standards for low bit rate video coding is done. Finally, effectiveness of H.264/AVC deblocking filter for reduction of blocking artifacts at low bit rates is evaluated. In chapter 4, design and implementation of a new criterion for deblocking algorithm for low bit rate video coding is presented. First, examination of strong and normal filter usage in original H.264/AVC deblocking filter is described. Second, categorization of various video sequences according to motion activity is introduced. Third, thresholds on the basis of absolute sum of motion compensation vectors for low, moderate and high motion sequences are provided. Fourth, design and implementation of proposed deblocking algorithm is discussed. Fifth, experimental 4

21 results for computing complexity, subjective and objective comparison of proposed scheme with original H.264/AVC deblocking algorithm are given. Finally, conclusions and future research directions are provided. 5

22 CHAPTER 2 Literature Review This chapter describes literature review of central methods for reduction of blocking artifacts. Section 2.1 describes the theory related to occurrence of blocking artifacts at low bit rates while categories of deblocking techniques and some central methods found in literature for blocking artifacts reduction are discussed in section 2.2. As this thesis presents a novel approach of incorporating motion activity of video sequences in deblocking filters; section 2.3 elaborates some central schemes for detection of motion activity found in literature. 2.1 Theory of Blocking Artifacts at Low Bit Rates The basic approach in block based discrete cosine transform schemes for image and video coding is to divide the whole image into blocks, transform each block using discrete cosine transform, quantize and entropy coded [23]. An image is divided in M x N blocks, generally 8 x 8 blocks. The Discrete Cosine Transform (DCT) for 8 x 8 block is given by Eq ( 2i 1) x ( 2 j 1) 7 7 C( x) C( y) + π + yπ Wi, j ( x, y) = wi, j ( x, y) cos cos i= 0 j= where w i,j (x,y) are the 64 samples of ij th input sample block and W x,y are the 64 DCT coefficients (x,y) and C(x), C(y) are constants as described by Eq

23 C( x) = 1/ 1 2 x = 0 x After this transform, the DCT coefficients are quantized. The inverse DCT (IDCT) reconstructs a block of image samples from an array of DCT coefficients. The IDCT takes as input a block of 8 x 8 DCT coefficients W x,y and reconstructs a block of 8 x 8 image samples w i,j by Eq ( 2i 1) x ( 2 j 1) 7 7 C( x) C( y) + π + yπ Wx, y ( i, j) = Wx, y ( i, j)cos cos 2.3 x= 0 y= Quantization step divides transformed coefficients by quantization table and are rounded to an integer. At low bit rates, high-order DCT coefficients are more severely quantized (usually to zero). In video coding, motion compensation is another source of propagation of these blocking artifacts [14]. Copied interpolated pixel data from various locations of different reference frames can be used for generation of motion compensated blocks. Discontinuities on the edges of copied blocks of data arise, as there is never a perfect fit for this data. Moreover, during copying process, existing edge discontinuities in reference frames are passed into the interior of the block to be motion compensated. Blocking artifacts makes the decompressed images/video unacceptable for human eyes at low bit-rates and often limits the maximum compression performance that can be achieved. Fig. 2.1 shows comparison of uncompressed (raw) and the reconstructed frames of for CIF MOTEHR DAUGHTER, CIF CONTAINER, and CIF FOREMAN encoded at 40 Kbps respectively. It is apparent that the reconstructed frames contain blocking artifacts. 7

24 Fig. 2.1 Blocking Artifacts at 40 Kbps (a) Uncompressed (raw) frame 3 of sequence CIF Mother and Daughter (b) Reconstructed frame of (a) by H.264/AVC (c) Original frame 2 of sequence CIF Container (d) Reconstructed frame of (c) by H.264/AVC (e) Uncompressed (raw) frame 4 of sequence CIF Foreman (f) Reconstructed frame of (e) by H.264/AVC 8

25 2.2 Methods to Reduce Blocking Artifacts Deblocking Filters This section outlines the categories of deblocking filters and reviews some core algorithms used for suppression of blocking artifacts in existing literature. Section provides overview of two main types used for reduction of blocking artifacts. Literature review of some significant methods used for deblocking are explained in section while summary of these methods is given in section Categories of Techniques for Reducing Blocking Artifacts There are two types of techniques employed for reduction of blocking artifacts [14]. : 1. Post filtering 2. Loop filtering In post filtering, as shown in Fig. 2.2, deblocking filter is applied after the decoder and utilizes decoded parameters. It operates on display buffer outside the coding loop. The frame is decoded into reference frame buffer and filtered before passing it to display device. An additional buffer may be required for implementation of post filter. Fig. 2.2 Post processing for reduction of blocking artifacts The use of post filter is optional in most standards as it is not a normative part of standards. In loop filtering, the deblocking filter works within the coding loop. For motion compensation of following frames, filtered frames are used as reference 9

26 frames. As a result, standard conformant decoder is needed to carry out filtering identical to that of encoder. Filtering takes place for each macroblock during decoding process and reference frame buffer is used to store the filtered output. Fig. 2.3 shows the position of loop deblocking filter in coding loop at encoder and decoder respectively. Fig. 2.3 Loop filter for reduction of blocking artifacts (a) encoder (b) decoder Different video coding standards proposed deblocking filters for blocking artifacts reduction. Table 2.1 shows deblocking filters used by various standards [11, 23]. Table 2.1 Deblocking filters for various standards Standard Deblocking Filter H.261 Optional in-loop filter MPEG-1 No filter MPEG-2 No Filter, post-filter processing often used H.263 No filter, post-filter using H.263+ MPEG-4 Optional in-loop filter, post-filter processing suggested H.264 Mandatory in-loop filter, post- filter processing may also be used 10

27 The reported research for post- and loop-filtering for reduction of blocking artifacts in literature is very diverse. Lot of attention has been given to post-filtering but very little work has been reported in the area of loop filtering in literature. Table 2.2 compare pros and cons of loop filtering and post filtering. Table 2.2 Comparison of post- and loop filtering Post filtering Loop filtering Independent of coding standard Implementation without any increment in bit rate or any modification in encoding procedure Filtering only at the decoder No compatibility issues as it works outside coding loop Extra buffer required at decoder Improvement in quality of reconstructed frame results in accurate motion compensation Exactly same filtering at encoder and decoder Extra frame buffer not required at decoder Compatibility with coding standard required Difficult to incorporate in commercial coding products with existing standards Use of either post filtering or loop filtering has some pros and cons on both sides. For example, in decoder implementations, maximum independence is offered by post filtering and no amendment in video coding standard is needed. However it requires an extra buffer at the decoder. On the other hand, there are also some advantages of loop filtering i.e., applying deblocking filter within coding loop [14]. First, by using loop filtering, the quality of reconstructed frame can be improved. The outcome is quality improvement of prediction frame and as a consequence, accuracy in motion compensated prediction for next encoded frame can be achieved. Second, the quality level of deblocking is guaranteed as exactly same filtering is done at encoder & decoder respectively, resulting in expected (predicted) quality of video at the decoder 11

28 side. Third, extra frame buffer is not required at decoder as was the case for post filters. Fourth, empirical results revealed that usage of loop filtering results in improvement of objective and subjective quality of video with major reduction in decoder complexity in comparison with post filtering [24-25] Literature Review for Deblocking Filters Many algorithms are proposed for reduction of blocking artifacts for block based transform coding schemes. Among them are: 1. Projection on Convex Set (POCS) based algorithms 2. Maximum a Posteriori (MAP) technique 3. Constrained Least Square (CLS) deblocking 4. Combined Transform Coding (CTC) scheme 5. AC prediction based deblocking 6. Wavelet based deblocking algorithms 7. Multilayer perceptron (MLP) neural network based deblocking method 8. Deblocking filtering using weighted sum of symmetrically aligned pixels 9. Deblocking using gradient projection method 10. Lapped orthogonal transform (LOT) based deblocking 11. Deblocking using genetic algorithm (GA) 12. Deblocking based on Human Visual System (HVS) 13. Non-linear spatial filters deblocking 14. Adaptive linear spatial filters deblocking The brief description of above mentioned approaches is as follows: 12

29 Projection on Convex Set (POCS) based Algorithms The POCS algorithms [26] use iterative block reduction technique based on theory of projection onto convex sets. A number of constraints on coded image are used for restoration into original form. For example, one constraint can be devised from the information that blocking artifact image has high frequency components across boundary of neighboring blocks. These high frequency components are omitted from original image, so projection of artifact image onto original image is performed by iterative procedure. These iterations are repeated until artifact free image is obtained. In POCS based algorithm proposed by Yang et al [27], based on line processes modeling of the image edge structure, a new family of directional smoothness constraint sets is described. Because of the fact that visibility of artifacts in an image is spatially varying, the authors have also taken definition of smoothness sets. The numerical difficulty of computing the projections onto these sets is overcome by a divide-and-conquer (DAC) strategy. In DAC, new smoothness sets are derived such that their projections are easier to compute. The algorithm can remove blocking artifacts from compressed image and video. The highly correlated images are assumed by Paek et al [28] to reduce blocking artifacts based on POCS. As assumed images are highly correlated, the global frequency characteristics in two adjacent blocks are similar to the local ones in each block. The high frequency components in global characteristics of a decoded image, which are not found in local ones, results from blocking artifacts are considered. N-point DCT to obtain the local characteristics, and 2N-point DCT to obtain the global ones, and then relation between N-point and 2Npoint DCT coefficients are employed. The undesired high frequency components caused by blocking artifacts are detected by comparison of N-point with 2N-point DCT coefficients. Then novel convex sets and their projection operators in the DCT 13

30 domain are proposed by authors and they claim that it yields significantly better performance than the conventional techniques in terms of objective quality, subjective quality, and convergence behavior. Maximum a Posteriori (MAP) Technique MAP based technique is based on stochastic model of image data [29]. It selects the best image from a set of better images. Quantization step partitions the transform coefficients and maps all points in a partition cell to a reconstruction point, taken as centeriod of cell. The technique selects the reconstruction point within quantization partition cell which results in reconstructed image that best fits a non-gaussian Markov random field (MRF) image model. The gradient projection method is used to update the estimate based on image model iteratively. In paper [30], probabilistic models are used for both the degradation introduced by the coding and for a good image. The restored video sequence is the MAP estimate based on these models. The authors first describe a generic model for video compression. It also explains the effects of motion compensation which is used in many video compression techniques. A decompression algorithm is then outlined based on a previously proposed image model. From experimental results, reconstructed image sequence shows a reduction in many of the most noticeable artifacts. Constrained Least Square (CLS) deblocking Yang et al [31] describes reconstruction of images from incomplete block discrete cosine transform (BDCT) data. In it, prior knowledge about the smoothness of the original image is transmitted along with the image data. The decoder reconstructs the image by using both of them. Two methods are proposed in this paper based on POCS 14

31 and CLS respectively. In CLS, the proposed objective function captures the smoothness properties of original image. The recovered image is obtained by minimizing an objective function, which is the weighted sum of two functions that impose conflicting requirements on the recovered image. Thus, if one of these functions penalizes deviation from the available data the other must penalize the undesired effects if an image is reconstructed only from the available data. In this sense, the second function introduces prior knowledge that complements the available data or, in other words, constrains the behavior of the reconstructed image. Iterative algorithms are introduced for its minimization. The authors claim with the help of experimental results that blocking artifacts can be reduced drastically. In another paper based on adaptive constrained least squares restoration by Andre' Kaup [32], a numerically simple post-processing scheme is proposed. The spatial adaptation of post processing to local image structure preserves high frequency details of image. The authors claim that proposed technique almost completely removes blocking artifacts. Combined Transform Coding (CTC) scheme In Combined Transform Coding (CTC) scheme [33], image is divided into two sets that contain different correlation properties, i.e., the upper image set (UIS) and lower image set (LIS). The UIS contains the most significant information and tends to be highly correlated whereas; LIS contains the less significant information and carries less correlation. Then the UIS is compressed noiselessly without dividing into blocks and LIS is coded by conventional block transform coding. This results in suppression of blocking effects in image due to the fact that correlation in UIS is reduced without distortion and thus as a result the inter-block correlation is significantly reduced.the additional advantage of the CTC scheme is removal of ringing effects. 15

32 AC Prediction based Deblocking Taehwan Shin et al [34] proposed a blocking effect reduction method based on content-based AC prediction for MPEG-2 video. The algorithm, first detects the block, which has caused blocking artifact. Then DC sequence is generated and position of block is searched in image content. The AC coefficients are predicted by contentbased AC prediction algorithm. Simulations performed by authors show that proposed algorithm reduces the blocking artifacts effectively. The research by Changick Kim [35], proposes another AC prediction based blocking artifact reduction method. For each block, its DC value and DC values of the surrounding eight neighbor blocks are exploited to predict low frequency AC coefficients. Each block is categorized into low activity or high activity block by use of these predicted AC coefficients. Then two types of low pass filters are adaptively applied based on the categorized result of each block. A strong low pass filter is applied in low activity region, where blocking artifacts are most noticeable. High activity regions are filtered by weak low pass filter. Computer simulations performed by author show that proposed algorithm is effective in reducing blocking artifacts as well as ringing artifacts. Hadamard transform is used by K. Veeraswamy et al [36] for AC coefficients prediction to reduce the blocking artifacts. In proposed method, Hadamard transform DC values are transmitted. AC restoration method is used for image reconstruction. The proposed method improves the peak signal to noise ratio and reduces the blocking effects significantly. Wavelet Based Deblocking Algorithms Wavelet based deblocking algorithm [37] computes soft threshold values based on difference between wavelet transform coefficients of image blocks and coefficients of entire image to threshold high-frequency wavelet coefficients in different sub-bands 16

33 using different values and strategies. An adaptive threshold value is employed for different images and characteristics of blocking effects. The filtered image is obtained by thresholding of different sub bands by three-level decomposition. Liew et al [38] proposed a non-iterative wavelet-based deblocking algorithm. The algorithm exploits the fact that block discontinuities are constrained by the dc quantization interval of the quantization table, as well as the behavior of wavelet modulus maxima evolution across wavelet scales to derive appropriate threshold maps at different wavelet scales. The algorithm can suppress blocking artifacts as well as ringing artifacts effectively while preserving true edges and textural information. Multilayer Perceptron (MLP) Neural Network based Deblocking Method Multilayer Perceptron (MLP) neural network deblocking is based on adaptive learning by examples concept. In this scheme [39], relevant information from the image is extracted and given as input to neural network. The MLP neural network tries to learn to reconstruct the original image. On the encoder side, the image is compressed and decompressed by image compression algorithms. By the decompressed image, features representing the occurrence of blocking effects, the numerical artifacts indicators (NAIs), are taken out and as an input given to the MLP network. The MLP will try to produce an output approximating the difference between the original image and the decompressed image. To train the MLP network, a suitable supervised learning algorithm and difference between the original and the decompressed image as a desired output is used. After the completion of training, the weights of the MLP network together with the compressed image data are transmitted or stored. When compressed data is received at the decoder, decompression and extraction of blocking 17

34 effect features is done and given as input to MLP network. The output of MLP network is added in the decompressed image for final decoded image formation. Deblocking Filtering using Weighted Sum of Symmetrically Aligned Pixels In deblocking filtering using sum of symmetrically aligned pixels [40], a new class of deblocking algorithms for reduction of blocking artifacts in images and video is proposed. A symmetrically aligned weighted sum of pixel quartets with respect to block boundaries is employed for image deblocking. The basic weights are obtained from a function which obeys predefined constraints. A deblocked image is produced using these weights which contain blurred edges near real edges. The authors refer these blurred edges as ghosting phenomenon. To prevent this, non-monotone area weights of pixels is modified by dividing each pixel s weight by predefined factor called a grade. This scheme is referred as weight adaptation by grading (WABG). Better deblocking of monotone areas is done by doing three iterations of WABG. The fourth iteration is done on rest of image to deblock the detailed blocks. The authors call this as deblocking frames of variable size i.e., DFOVS. The WABG and the DFOVS approaches automatically adapt themselves to different bit rates. It produces very good results for decompressed images ranging from extremely low to medium bit rates as claimed by authors. Deblocking using Gradient Projection Method The gradient projection based method [41] exploits the correlation between the intensity values of boundary pixels of two neighboring blocks. It is based on the theoretical and empirical observation that under mild assumptions, quantization of the DCT coefficients of two neighboring blocks increases the expected value of the Mean 18

35 Squared Difference of Slope (MSDS) between the slope across two adjacent blocks, and the average between the boundary slopes of each of the two blocks. This increase in expected value of MSDS is dependent on the width of quantization intervals of transform coefficients. Consequently, amongst all permitted inverse quantized coefficients, the set which reduces the expected value of this MSDS by a suitable amount is most likely to decrease the blocking artifacts. In order to estimate the set of unquantized coefficients, a constrained quadratic programming problem in which the quantization decision intervals provide upper and lower bound constraints on the coefficients is solved. The authors with the help of simulations claim that from a subjective viewpoint, the blocking effect is less noticeable in processed images than in the ones using existing filtering techniques. Lapped Orthogonal Transform (LOT) based Deblocking Lapped orthogonal transform (LOT) [42] can reduce blocking artifacts to very low levels. It is tool with basis functions that overlap adjacent blocks. Malvar et al [43] proposed an optimal LOT that is related to the DCT in such a way that a fast algorithm for a nearly optimal LOT is derived. The LOT is distinguished by the fact that each block of size N is mapped into a set of N basis functions, each one being longer than N samples. As coding noise is mainly a function of quantization process, therefore, it is virtually unaffected by LOT. The blocking effects are reduced to a level where they can hardly be detected by the human eye. However, it requires about percent more computations, mostly additions in comparison with DCT [43]. In another research by Malvar [44], the lapped bi-orthogonal transform (LBT) and hierarchical lapped bi-orthogonal transform (HLBT) are used for image coding. The HLBT has a significantly lower computational complexity than the lapped orthogonal transform 19

36 (LOT), with almost no blocking artifacts in comparison with DCT. Experimental results performed by author show better performance of the LBT and HLBT and they have fewer ringing artifacts. Deblocking using Genetic Algorithm (GA) Chih-Chin et al [45] proposed a hybrid approach of using L-filter (modified linear finite impulse response (FIR) filter or a generalization of median filter) and genetic algorithm (GA) to reduce the blocking artifacts. The authors consider the blocking artifact removal as a de-noising problem since the blocking artifacts can be thought as the superposition of an image and a quantization noise. An L-filter is an order statistic filter that combines order information of the observation data and applies linear operation to the ranked data. An L-filter can be used to remove different types of noises if its parameters are properly chosen [46]. The search for proper parameters for L-filter is done with the help of genetic algorithms (GAs). GAs are well known for their ability to perform parallel search in complex solution spaces and have the following advantages over traditional search methods: (i) GAs directly work with a coding of the parameter set; (ii) search is carried out from a population of points; (iii) payoff information is used instead of derivatives or auxiliary knowledge; and (iv) probabilistic transition rules are used instead of deterministic ones [47]. In proposed method by Chih-Chin et al, the L-filter is used for reduction of blocking artifacts and the GA is used to search the proper parameters for the L-filter. The proposed approach is used as follows: At the sender side, a reconstructed image is obtained by taking the inverse transform of the transmitted transform data. The proper L-filter parameters are found by using a GA between the original and reconstructed images. The L-filter parameters are then transmitted to the receiver side for removing the blocking 20

37 artifacts. The authors claim with the experimental results that the proposed approach is a practicable technique to reduce the blocking artifacts in the block-based compressed images. Deblocking based on Human Visual System (HVS) B. Macq et al [48] proposed a criterion based on visual model for reduction of blocking artifacts. The target is to decompose the corrupted image into perceptual channels and to cancel the channels where the noise is above the visibility threshold. Then image is reconstructed with the only channels where the estimated noise is below the visibility threshold. More specifically, the noisy picture is first split up into several perceptual channels by means of filters tuned to specific spatial frequencies and orientations. Each resulting filtered picture is then weighted by a masking function in order to cancel the visible noise. The masking is a function of the perceptual component contrast of the original picture. Difference of noisy picture and noise estimation is used for carrying out the contrast. The addition of each masked pictures provides at last the restored picture. Tao Chen et al [49] proposed an approach that works in transform domain for reduction of quantization noise. The adaptive weighting mechanism is integrated by considering the masking effect of human visual system. The proposed approach makes use of transform coefficients of shifted blocks, rather than those of the neighboring blocks, in order to obtain a close correlation between the DCT coefficients at the same frequency. The filtering is operated location-variantly based on the local activity of blocks to achieve the artifacts reduction and detail preservation simultaneously. More exactly, an adaptively weighted low-pass filtering technique is activated to image blocks of different activities, which represent the inherent masking abilities for artifacts. Human visual 21

38 system sensitivity at different frequencies is used to characterize the block activity. Blocking artifacts are more noticeable for low-activity blocks and post-filtering of the transform coefficients is applied within a large neighborhood to smooth out the artifacts. For high activity blocks, a small window and a large central weight are used to preserve the image details since the eye has difficulty discerning small intensity variations in portions of an image where strong edges and other abrupt intensity changes occur. Finally, the quantization constraint is also applied to the filtered DCT coefficients prior to the reconstruction of the image from coefficients. Another approach for reduction of blocking artifacts based on masking effect of human visual system is proposed by Shen-Chuan Tai et al [50]. The proposed scheme is based on three separate modes that classify local characteristics of images. Region classification with respect to activity across block boundary is done before the application of one of three modes of deblocking filter. The classification of regions is: smooth regions, complex regions and intermediate regions. Flat areas of block boundary are strong filtered whereas, weak filter is applied is areas of high spatial or temporal activity. An intermediate mode is used for solving problem of either excessive blurring or inadequate removal of blocking effect. Deblocking using Non-linear Spatial Filters An algorithm based on non-linear smoothing of pixels for deblocking is proposed by Jim Chou et al [51]. The deblocking is performed in two steps. In step 1, difference between actual image edges and artificial discontinuities produced by quantization noise at block boundaries is taken into account. A probabilistic framework is used to derive estimates for the reconstructed DCT coefficients and for the quantization error of each image coefficient. While removal of blockiness by reducing discontinuities at 22

39 block edges is done in step 2. The principal used is to reduce discontinuities of artificial edges at block boundaries to a level that is imperceptible to the eye. First, the discontinuities are computed by differencing the pixels across each block boundary and then, authors attempted to reduce these discontinuities below visibility threshold. Experimental results show significant improvement in visual quality of images. Gaetano Scognamiglio et al [52] proposed a technique based on unsharp masking (UM) for noise smoothing and edge enhancing. The authors used the approach described in [53] with additional new features. Important new feature in this technique is that amount of coding artifacts and fact that blocking artifacts can be located at any position in video sequence is taken into account. The method does not need any information about position and size of blocks. In another approach by Kee-Koo Kwon et al [54], an adaptive post processing algorithm using block boundary classification and simple adaptive filter (SAF) is proposed. The method of deblocking can be described as follows: First, classification of each block boundary into smooth or complex sub-region is done. For smooth-smooth sub-regions with blocking artifacts, a non-linear 1-D 8 tap filter is applied while a nonlinear 1-D variant filter is applied to smooth-complex and complex-smooth regions for suppression of artifacts. For complex-complex sub-regions, a nonlinear 1-D 2-tap filter is only applied to adjust two block boundary pixels so as to preserve the image details. Authors experimental simulations show that proposed algorithm produces better results than those of the conventional algorithms, both subjectively and objectively. Adaptive Linear Spatial Filters Deblocking A deblocking algorithm by adaptively using spatial frequency and temporal information extracted from the compressed data is proposed by Hyun Wook Park et al 23