A Novel Hierarchical Search Algorithm for Video Compression

Transcription

1 International Conference on Advances in Computer and Electrical Engineering (ICACEE'01) ov , 01 Manila (Philippines) A ovel Hierarchical Search Algorithm for Video Compression ijad Al-ajdawi Abstract Motion estimation algorithms have proven to be effective in the reduction of video bit-rates while preserving the good quality. The most popular technique for motion estimation is block matching. Block matching algorithms involve searching techniques for block movements between consecutive video frames. Researchers try to develop fast search motion estimation algorithms to reduce the computational cost required by full-search algorithms. In this research, the author presents a new fast search algorithm based on the hierarchical search approach. The original image is sub-sampled into additional two levels. The full search is performed on the highest level where the complexity is relatively low. The enhanced Three- Step Search algorithm and a new proposed searching algorithm are used in the consecutive two levels. The results show that, the performance of the proposed hierarchal search algorithm is close to the full search with 16.6% complexity and a high matching quality. Keywords Hierarchical Search, Motion Estimation, Video Compression V I. ITRODUCTIO IDEO Compression is the process of representing the video data using fewer bits than the original representation. The captured video frames consist of redundant data in the spatial and temporal domains. Therefore, researchers have developed inter and intra-frame coding methods to eliminate the temporal and spatial domains redundancy, respectively. In inter-frame coding, motion estimation and compensation are powerful techniques to eliminate the temporal redundancy, due to the high correlation between consecutive frames. Motion estimation describes the transformation from one frame to another using motion vectors. Block Matching Motion estimation algorithms assume that all the pixels within a block have the same motion activity, and estimates motion on the basis of rectangular blocks, producing one motion vector for each block [1]. In block matching algorithms, the current frame of a video sequence is divided into non-overlapping square blocks of pixels. For each reference block in the current frame, the algorithm searches for the best matched block within a search window in the previous frame of size. The relative position between the reference and its best matched block is represented as the motion vector of the reference block []. A simple method of block matching algorithm is the Full-Search Algorithm (FSA), which requires computing all positions of ijad Al-ajdawi is an assistant professor at the department of computer science, Prince Abdullah Bin Ghazi Faculty of Information Technology, Al- Balqa Applied University, Jordan ( n.al-najdawi@bau.edu.jo). candidate blocks in the search window. Although FSAs yield the best possible results, they require intensive computation processes, hence limiting their practical applications. Therefore, researchers have investigated the use of computationally efficient fast search motion estimation algorithms. Fast search algorithms are classified as suboptimal to the FSAs as they do not result in a quality that is as good [3]. The BMAs are built depending on search patterns, step sizes, and the total number of required searches, attempting to reduce the complexity of FSA, by mostly reducing the number of required comparisons. Many BMAs have been developed in literature with various searching strategies that involve: searching patterns such as cross, and diamond patterns. The former pattern involves searching the diagonal neighbouring points, and the latter involves searching the adjacent vertical/horizontal neighbouring points. Another criterion that differentiates between BMAs is the step size, which indicates the distance (in pixels) between the searching points and the center of search. Some algorithms are based on fixed distances while others calculate step sizes based on the searching window size. The total number of required searches (comparisons) is algorithm dependent, and it indicates how many comparisons the algorithm will perform before it concludes with the best match. The total number of required comparisons has a great impact on the algorithm s complexity; therefore researchers have attempted to develop algorithms that require low number of comparisons. This research presents a new fast search algorithm based on the hierarchical search approach, where the number of searched locations is reduced compared to the Full Search. In addition to the introduction presented in section I, section II presents the related literature. Section II introduces the proposed method and section IV presents the experimental results and evaluation. Finally section V concludes this work. II. LITERATURE REVIEW Hierarchical Search is considered as one of the most efficient block matching algorithms. Hierarchical block matching techniques attempt to combine the advantages of large blocks with those of small blocks. The reliability of motion vectors is predisposed by block size. Large blocks are more likely to track actual motion than small blocks, and thus are less likely to converge on local minima. Hierarchical block matching algorithms exploit the motion tracking capabilities 46

2 International Conference on Advances in Computer and Electrical Engineering (ICACEE'01) ov , 01 Manila (Philippines) of large blocks and use their motion vectors as starting points for searches for small blocks. There has been some research on the hierarchical search algorithms such as the mean pyramids proposed by Lee and Chen [14], and the multiresolution video coding proposed by Kuo et al., [13]. In [14] the authors applied the concept of Mean Pyramid Data structure with each hierarchical level corresponding to a reduced-resolution approximation. In their work, an image pyramid is formed by reduced resolution levels and each pyramid level data structure is formed using an averaging filter over corresponding neighboring pixels, where the bottom level is the original image and the top level represents the mean value of the image. The Hierarchical Stochastic Fast Search Motion Estimation algorithm [15] requires 5% of the total computations required by the full search algorithm. The measured motion vectors are obtained by a simplified hierarchical search block-matching algorithm, and are used as the measurement part of the Kalman filter. A ew Two-Step Hierarchical BMA [16] shows that in a continuous motion, the adjacent blocks can be candidates in the motion vector. The two-step hierarchical BMA uses spatial correlation in a motion field. Using a partial searching block rather than the whole block, the partial mean absolute difference is used as a matching criterion. A Hierarchical Partial Distortion Search [17] minimizes the computation of each distortion measure instead of reducing the number of checking points by using a partial distortion measure. The algorithm divides the motion vector search into three levels, where the lower levels use partial distortions higher decimation ratios. For every level, the candidate motion vectors with minimum partial distortion are selected to enter the next level. The wide range of algorithms available for block-based motion estimation makes the process of selecting the optimal algorithm difficult. The choice can depend on different criteria, such as: matching performance, rate-distortion performance, complexity, scalability, and implementation [18]. As motion estimation is subject to noise, researchers have attempted to use the Kalman Filter in order to enhance the motion vectors predictions and measurements and to obtain a better performance [8]. Many approaches combine the Kalman filter with available BMAs such as [13], [19], [0], [1], [], [3], [4], [5], [6], and [9] however the use of Kalman filter is considered intuitive. Analyzing the quality and performance of motion estimation algorithms has been a popular research area since video codecs were released and different results have been obtained by different researchers. According to Ghanbari [1] in terms of speed, the Two-Dimensional-Logarithmic algorithm outperforms the rest of the algorithms at the cost of quality. The Three-Steps- Search achieves a marginal improvement in terms of quality but has a high computational complexity in comparison with the Two-Dimensional-Logarithmic algorithm. The Four- Steps-Search algorithm outperforms the Three-Steps-Search algorithm in terms of complexity; however, its quality does not approach that of Full-Search as the hierarchical algorithms do. Although the complexity of the hierarchical algorithms is worse than some of other fast search algorithms, they outperform any other algorithm in terms of quality and they almost have the same quality as the Full-Search algorithms, with a significant reduction in complexity. According to Kuhn et al. [7], alternate pixel sub-sampling depicts very similar results as the original full search algorithm, where no extreme case of performance degradation occurs. For the search area sub-sampling algorithms, the Three Step Search results in the best results. The five step diamond search performs well, but suffers in some cases from a very small search range of pixels. Amongst the available BMAs, the Three-Step-Search (TSS) is the most popular algorithm [4]. The TSS algorithm however, uses a uniformly allocated checking point pattern, making it inefficient for searching small motion video sequences. The ew Three Step Search (TSS) Algorithm [5] modifies the existing TSS algorithm. Whilst TSS algorithm requires predefined checking points pattern, the TSS algorithm uses center-biased checking point pattern, by making the search adaptive to the motion vector distribution, and a halfway-stop technique is used to reduce the computation cost. Another approach that enhances the existing TSS algorithm is the Efficient Three Step Search (ETSS) algorithm [6]. The Four-Step-Search algorithm [7] uses center-biased checking point pattern; and the Halfway-stop technique. It utilizes a small initial step compared to the TSS. The Two Dimensional Logarithmic Search (TDLS) algorithm [8] works by dividing a frame into blocks and then finds the minimum distortion for each block. The TDLS employs the cross search pattern in each step. The Binary Search algorithm [9], works by dividing the search window into a number of regions and then performs a full search only on one of these regions. A ew Diamond Search algorithm (DSA) [10] uses two search patterns. The first pattern is a large diamond search comprising nine checking points. The second pattern is a small diamond search pattern which consists of five checking points. Orthogonal Search Algorithm [11] has three checking points to be searched in horizontal direction. The best match becomes the center of the three point vertical search, where the step size is decreased by half and the same strategy is used. In Cross Search Algorithm [1], each step involves 4 locations to be searched. Starting from the center point and forming searching points at the end of cross (X) pattern, the best match is selected to be the new center point, and the step size is decreased by half. The same procedure is repeated until the step size becomes equal to 1. At this step size, a (+) cross search pattern is used, if the best match of the previous step is either the center, upper left or lower right checking point, otherwise, a (X) cross search pattern is used. III. THE PROPOSED HIERARCHICAL SEARCH MOTIO ESTIMATIO ALGORITHM Hierarchical search algorithms use a coarsely sub-sampled version of the image, followed by successively higher- 47

3 International Conference on Advances in Computer and Electrical Engineering (ICACEE'01) ov , 01 Manila (Philippines) resolution versions until the full image resolution is reached. After the images are constructed by sub-sampling, the hierarchical search motion vector estimation proceeding from the higher level to the lower levels reduces the computational complexity and gets high quality motion vectors. Different motion estimation algorithms can be used at each level of the HS algorithm. At the highest (top) level, Full Search is intuitively used, since the complexity is relatively low, due to the low number of possible searches that is reduced by image sub-sampling. The rest of the hierarchy levels may contain any of the fast search BMAs. In this work, the three levels hierarchy is used in order to simplify the process. In the top level, full search is used. The middle and the bottom levels are processed using the new proposed BMA and the Enhanced three-step-search algorithm, respectively. A new algorithmic search pattern will be used at the Middle level of the hierarchy. The following illustrates the steps of the proposed work for each block in the image: Step 1: the lowest level (level-1) consists of the original frame at its full resolution. This step involves sub-sampling level-1 by a factor of in vertical and horizontal directions to produce level-. Step : This step involves sub-sampling level- in the same way to produce level-3 (highest level). The sub-sampling process ends, by getting level-1, level- and level-3. Step 3: In this step, the search starts from the highest level (level-3) using block sizes, where a FS algorithm will be performed to get the initial coarse motion vector and the best match position will be passed to the lower level (level-). Step 4: This step involves searching level- by using the new proposed search pattern (using 8 8 block size) to get a motion new motion vector, and the best match position will be passed to level-1 (lowest level). Step 4 involves the following phases: Phase 1: involves setting the window size to 1, where is the number of levels in the hierarchical search (i.e., = 3 in the proposed algorithm), and setting the step size to the standard 1 (i.e., step size= 4). Phase : Starting at the center point location around the obtained coarse motion vector, this step involves searching the four points located at the corners of the (X) shape pattern (see Fig. 1). The best match will be passed to step-3 as a new center of search. Phase 3: involves setting the step size to ( 1) /, searching the four neighboring points around the new center obtained from step-, and forming the diamond shape (see Fig. 1). If the step size >1, then set the step size to ( 1) / and repeat step-3; otherwise, the best match point that is found is passed to level-1 of the hierarchy. Step 5: In this step, the Enhanced Three-Step-Search algorithm is used on level-1, to get the final motion vector that will be added to the previous image to get the next predicted image frame. Fig.1: Left: is the first step of the proposed algorithm, where four locations are searched around the obtained motion vector forming a cross shape pattern. Right: is the second step of the proposed algorithm, where four locations are searched around the best match point obtained from (a). IV. EXPERIMETAL RESULTS The proposed Hierarchal Search algorithm is tested on several benchmark video sequences. The format used in the experiments is the CIF. TABLE I shows the used test videos, in addition to the total number of frames. More than 600 video frames are tested from different sequences a. The results are evaluated subjectively and objectively. The PSR is used to objectively evaluate the system performance, where PSR = 10log10 ( L / MSE) is measured in decibel units (db units), where L is the range of pixel values (when the luminance component is only used L=55, and MSE = 1/ = ( x i y ) and where is the number of the pixels per frame, and x i, yi are the pixels within the original and predicted frames, respectively. A standard measurement states that, if the PSR result is >30db, then the difference between the original image and the resulted processed image will not be recognized through the i 1 TABLE I STADARD CIF VIDEO SEQUECES YUV Video Sequence Total number of frames Akiyo 100 Claire 100 Flower 50 Foreman 50 Hall Monitor 50 ews 100 Salesman 50 Suzie 100 Standard video sequences used in this research with available CIF format i 48

4 International Conference on Advances in Computer and Electrical Engineering (ICACEE'01) ov , 01 Manila (Philippines) TABLE II PSR RESULTS Video Sequene FS TSS Proposed HS algorithm Akiyo Claire Flower Foreman Hall Monitor ews Salesman Suzie Average PSR PSR results of the proposed algorithm compared to the standard Three-Step-Search Algorithm and the Full Search algorithm human visual system. Using the original and reconstructed frames, Table II, illustrates the PSR values for the proposed HS and other benchmark algorithms. The results shown in Table II show that, using the standard set of test videos, the proposed HS algorithm outperforms the benchmark Three-Step-Search. The average PSR shows an enhancement of 7db units in some particular cases, signifying and enormous enhancement of quality. Even without the use of an additional set of filters, the proposed algorithm has results comparable to those of the full search algorithm. In addition to the objective evaluation, a subjective evaluation of the proposed work can be seen in Fig. and Fig.3 which illustrates the results of the proposed HS algorithm when applied a sample set of standard videos illustrated in Table I. proposed algorithm requires only 16.6% of the total OPB required by the FSA. Although the OPB in the proposed work exceeds the other illustrated algorithms in Table III, due to the high number of searches, the quality of the proposed algorithm is similar to the FSA. Additions (a) (w + 1) ( 1) w = 7, = 16 (b) Level(3) (w + 1) ( 1) w = 3, = 4 Level() 8 ( 1), = 8 Level(1) 3 ( 1), = 16 (c) P ( 1) P = 3, = 16 TABLE III COMPLEXITY Absolute Difference OPB ( w + 1) w = 7, = 16 =17575 Level(3) ( w + 1) w = 3, = 4 Level() 8, = 8 Level(1) 3, = 8 P P = 3, = 16 =8605 =13801 Algorithms complexity: (a) the Full Search algorithm, (b) the proposed HS algorithm, (c) the Three-Step-Search algorithm Fig.: Reconstructed frame-47 of the Flowers video sequence The complexity of the proposed HS algorithm is evaluated and compared against some of the benchmark searching methods. Compared the FSA, the proposed algorithm require 18.6% of the total number of additions, and 1.4% of the total absolute differences. This can be summarized with a total umber of Operations Per Block (OPB), where the Fig.3: Reconstructed frame-8 of the ews video sequence 49

5 International Conference on Advances in Computer and Electrical Engineering (ICACEE'01) ov , 01 Manila (Philippines) V. COCLUSIO Motion estimation algorithms intend to construct the current video frame as accurately as possible while keeping the computational complexity acceptable. Full Search algorithm often converges to the global minima, at the cost of complexity. Therefore researchers have investigated the use of Fast Search motion estimation algorithms in order to reduce the Full Search complexity. Many fast search algorithms have been proposed with different search patterns and varying numbers of required searches. Whilst the searching algorithms proposed in literature vary in terms of approach, design, and results, hierarchical Search algorithms have shown comparative results. In this work, a new Hierarchal Search algorithm is used, where the original image is iteratively subsampled in horizontal and vertical directions to produce three levels. The three levels combine a full search, a new proposed searching pattern, and the Enhanced three-step-search algorithms from the highest to the lowest levels respectively with variable block sizes. A complexity evaluation, in addition to the experiments, has shown that the proposed work achieves a quality that approaches the Full Search algorithm with 16.6% of the required complexity. REFERECES [1] M. Ghanbari, Video coding: an introduction to standard codecs, Institution of Electrical Engineers, [] I. Richardson, Video Codec Design, John Wiley & Sons, 00. [3] R. Gonzales, R.Woods,. Digital Image Processing, second ed., Prentice Hall, 00. [4] T. Koga, K. Iinuma, A. Hirano, Y. Iijima, T. Ishiguro, Motion compensated interframe coding for video conferencing, Proc. ut. Telecommun. Conf. (1981) G [5] R. Li, B. Zeng, M.L. Liou, A new three-step search algorithm for block motion estimation, IEEE Trans. Circuits Syst. Video Technol. 4(4), 1994, pp [6] X. Jing, L.P. Chau, An efficient three-step search algorithm for block motion estimation, IEEE Trans. Multimedia 6(3), 004, pp [7] L.M. Po, W.C. Ma, A ovel Four-Step Search Algorithm for Fast Block Motion Estimation, IEEE Trans. Circuits Syst. Video Technol. 6(3), 1996, pp [8] J.R. Jain, A.K. Jain, Displacement measurement and its application in interframe image coding, IEEE Trans. Commun. 9(1), 1981, [9] T. Zahariadis, D. Kalivas, Spiral Search Algorithm For Fast Estimation Of Block Motion Vectors, Proc. of the Eighth Eur. Signal Process Conf., 1996 pp [10] S. Zhu, K.K. Ma, A new diamond search algorithm for fast block matching motion Estimation, IEEE Trans. Image Process. 9(), 000, pp [11] A. Puri, H.M. Hang, D. Schilling, An efficient block-matching algorithm for motion-compensated coding, Proc. IEEE Int. Conf. Acoust. Speech and Signal Process. 1987, pp [1] M. Ghanbari, The cross search algorithm for motion estimation, IEEE Trans. Commun. 38(7), 1990, pp [13] C. Kuo, C. Hsieh, C. Chao, Multiresolution video coding based on Kalman filtering motion estimation, J. Vis. Commun. Image Represent. 13(1) 00, pp [14] K.M. am, J.S. Kim, R.H. Park, Y.S. Shim, A fast hierarchical motion vector estimation algorithm using mean pyramid, IEEE Trans. Circuits Syst. Video Technol. 5(4), 1995, pp [15] S. Tedmori,. Al-ajdawi, Hierarchical stochastic fast search motion estimation algorithm, IET Comput. Vis. 6(1), 01, pp [16] B. C. Song, J.B. Ra, A hierarchical block matching algorithm using partial distortion criterion, Proc. SPIE 3309, 1997, pp [17] C.K. Cheung, L.M. Po, A hierarchical block motion estimation algorithm using partial distortion measure, Proc. Int. Conf. Image Process. 3, 1997, pp [18] S. Basah, A. Bab-Hadiashar, R. Hoseinnezhad, Conditions for motionbackground segmentation using fundamental matrix, IET Comput. Vis. 3(4), 009, pp [19] C. Kuo, C. Chao, C. Hsieh, An efficient motion estimation algorithm for video coding using Kalman filter, J. Real-Time Imaging 8(4), 00, pp [0] C. Kuo, C. Chao, C. Hsieh, A new motion estimation algorithm for video coding using adaptive Kalman filter, J. Real-Time Imaging 8(5), 00, pp [1] C. Kuo, S. Chung, P. Shih, Kalman filtering-based rate-constrained motion estimation for very low bit rate video coding, IEEE Trans. Circuits Syst. Video Technol. 16(1), 006, pp [] Z. Wang, H. Sheikh, A. Bovik, Objective video quality assessment, in Furht, B., Marques, O. (Eds.): The handbook of video databases: design and applications, CRC Press, 003, pp [3] C. Kuo, C. Hsieh, H. Lin, P. Lu, Motion estimation algorithm with Kalman filter, IEEE Electron. Lett. 30(7), 1994, pp [4] C. Kuo, C. Hsieh, Y. Jou, H. Lin, P. Lu, Motion estimation for video compression using Kalman filtering, IEEE Trans. Broadcast. 4(), 1996, pp [5] V. Ruiz, V. Fotopoulos, A. Skodras, H. Constantinides, An 8 8-block based motion estimation using Kalman Filter, Proc. IEEE Int. Conf. Image Process. 1997, Greece. [6] J. Kim, J. Woods, 3-D Kalman filter for image motion estimation, IEEE Trans. Image Process. 3(1), 1998, pp.4 5. [7] P. Kuhn, G. Diebel, S. Herrmann, A. Keil, H. Mooshofer, A. Kaup, R. Mayer, W. Stechele, Complexity and PSR comparison of several fast motion estimation algorithms for MPEG-4, Proc. SPIE, 1998, pp [8] J. Gao, A. Kosaka, A. C. Kak, A multi-kalman filtering approach for video tracking of human-delineated objects in cluttered environments, Elsevier Comput. Vision Image Understanding 99(1), 005, pp [9] S. Yin, J. a, J. Choi, S. Oh, Hierarchical Kalman-particle filter with adaptation to motion changes for object tracking, Elsevier Comput. Vision Image Understanding 115(6), 011, pp ijad Al-ajdawi received his BSc degree in Computer Science from Mu tah University, Jordan in He obtained his MSc degree in Multimedia and Internet Computing in 003 and a PhD degree in Machine Vision and Autonomous Systems in 006, from Loughborough University, UK. After which he joined Loughborough University as a research Associate (Post-Doc position) in the Electronic and Electrical Engineering department. Currently, he is appointed as an assistant professor of computer science at Al-Balqa Applied University, Jordan. His research interests include: image processing, video coding objects tracking and recognition. 50