Project Proposal Spring 2014 Course: EE5359 Multimedia Processing Study and Performance Comparison of HEVC and H.264 Video CODECs Uma Sagar Madhugiri Dayananda Email: uma.sagar@mavs.uta.edu UTA ID: 1001002964
Contents Scope of Project HEVC Video Coding Standard H.264 Video Coding Standard Comparison Metrics Profiles for Comparison Test Sequences References Acronyms and Abbreviations
Scope of the Project This project aims at studying the state-of-the-art High Efficiency Video Coding(HEVC)[12] and H.264/Advanced Video Coding(AVC)[13] video codecs and gaining an understanding of various techniques in these codecs such as prediction, transform, quantization and coding. Comparison of these video codecs based on various performance metrics such as computational time, PSNR, SSIM, BD-Bitrate and BD-PSNR will be carried out. HM 13.0 and JM 18.6 reference software for HEVC and H.264 respectively will be used for this purpose.
Introduction to HEVC High Efficiency Video Coding (HEVC) [12] is an international standard for video compression developed by a working group of ISO/IEC MPEG (Moving Picture Experts Group) and ITU-T VCEG (Video Coding Experts Group). The main goal of HEVC standard is to significantly improve compression performance compared to existing standards (such as H.264/Advanced Video Coding [13]) in the range of 50% bit rate reduction [5]. HEVC is designed to address existing applications of H.264/MPEG-4 AVC and to focus on two key issues: increased video resolution and increased used of parallel processing architectures [5].
Block Diagram of HEVC CODEC [17]
Steps in HEVC Encoding Partitioning each picture into multiple units Predicting each unit using inter or intra prediction, and subtracting the prediction from the unit Transforming and quantizing the residual (the difference between the original picture unit and the prediction) Entropy encoding the transform output, prediction information, mode information and headers.
Block Diagram of HEVC Encoder [9]
Steps in HEVC Decoding Entropy decoding and extracting the elements of the coded sequence Rescaling and inverting the transform stage Predicting each unit and adding the prediction to the output of the inverse transform Reconstructing a decoded video image.
Block Diagram of HEVC Decoder [31]
H.264/Advanced Video Coding Standard H.264/Advanced Video Coding (AVC) [13] is video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goals of the H.264/AVC standard were to enhance compression performance and provision of a network-friendly video representation addressing conversational (video telephony) and non-conversational (storage, broadcast, or streaming) applications [8].
Block Diagram of H.264 CODEC [18]
Encoding in H.264 The encoder processes a frame of video in units of a macroblock (16x16 displayed pixels). It forms a prediction of the macro-block based on previously-coded data, either from the current frame (intra prediction) or from other frames that have already been coded and transmitted (inter prediction). The encoder subtracts the prediction from the current macro-block to form a residual. A block of residual samples is transformed using a 4x4 or 8x8 integer transform, an approximate form of the Discrete Cosine Transform (DCT). The output of the transform, a block of transform coefficients, is quantized, i.e. each coefficient is divided by an integer value. The values produced in these processes are encoded to form the compressed bit stream.
Block Diagram of H.264 Encoder [10]
Decoding in H.264 A video decoder receives the compressed H.264 bit stream, decodes each of the syntax elements and extracts the information such as quantized transform coefficients, prediction information, etc. This information is then used to reverse the coding process and recreate a sequence of video images.
Block Diagram of H.264 Decoder [10]
Comparison of HEVC and H.264 Coding Tools [20] Coding Tool H.264/AVC HEVC Partition size Macro block 16x16 Coding Unit 8x8 to 64x64 Partitioning Sub-block down to 4x4 Prediction Unit Quad tree down to 4x4 Square, symmetric and asymmetric (only square for intra) Intra prediction Up to 9 predictors Up to 35 predictors Transform Integer DCT 8x8, 4x4 Transform Unit square IDCT from 32x32 to 4x4 + DST Luma Intra 4x4 Entropy coding CABAC, CAVLC CABAC Filters De-blocking Filter De-blocking Filter Sample Adaptive Offset
Metrics to be used for comparison of CODECs Peak Signal to Noise Ratio (PSNR) Structural Similarity Index (SSIM) BD-Bitrate BD-PSNR Computational time as a measure of implementation complexity
Profiles for Comparison The following profiles for HEVC and H.264 respectively will be used for comparison in this project: HM 13.0 main profile [5] JM 18.6 high profile [8]
Test Sequences The following test sequences [23] are used for the study in the project: RaceHorses_416x240_30.yuv BasketballDrill_832x480_50.yuv
Test Sequences KristenAndSara_1280x720_60.yuv Kimono1_1920x1080_24.yuv
References [1] I.E.G. Richardson, Video Codec Design: Developing Image and Video Compression Systems, Wiley, 2002. [2] K.R. Rao, D.N. Kim and J.J. Hwang, Video Coding Standards: AVS China, H.264/MPEG-4 Part 10, HEVC, VP6, DIRAC and VC-1, Springer, 2014. [3] I.E.G. Richardson, H.264 and MPEG-4 Video Compression, Hoboken, NJ, Wiley, 2003. [4] I.E.G. Richardson, The H.264 advanced video compression standard, 2 nd Edition, Hoboken, NJ, Wiley, 2010. [5] G.J Sullivan et al, Overview of the High Efficiency Video Coding (HEVC) Standard, IEEE Transactions on Circuits and Systems for Video Technology, Vol.22, No. 12, pp. 1649-1668, Dec 2012. [6] F. Bossen et al, HEVC Complexity and Implementation Analysis, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1685-1696, Dec. 2012. [7] J. R. Ohm et al, Comparison of the Coding Efficiency of Video Coding Standards Including High Efficiency Video Coding (HEVC), IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1669-1684, Dec. 2012. [8] T. Wiegand et al, Overview of the H.264/AVC Video Coding Standard, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, Jul. 2003. [9] D. Marpe et al, The H.264/MPEG4 advanced video coding standard and its applications, IEEE Communications Magazine, Vol. 44, pp. 134-143, Aug. 2006. [10] A. Puri et al, Video coding using the H.264/MPEG-4 AVC compression standard, Signal Processing: Image Communication, vol. 19, pp. 793-849, Oct. 2004. [11] X. Li et al, Rate-complexity-distortion evaluation for hybrid video coding, IEEE International Conference on Multimedia and Expo (ICME), pp. 685-690, Jul. 2010.
References(Continued) [12] B. Bross et al, High Efficiency Video Coding (HEVC) Text Specification Draft 10, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.itsudparis.eu/jct/doc_end_user/current_document.php?id=7243 [13] JVT Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264-ISO/IEC 14496-10 AVC), March 2003, JVT-G050 available on http://ip.hhi.de/imagecom_g1/assets/pdfs/jvt-g050.pdf [14] J. Vanne et al, Comparative Rate-Distortion-Complexity Analysis of HEVC and AVC Video Codecs, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1885-1898, Dec. 2012. [15] G. Bjontegaard, Calculation of Average PSNR Differences between RD Curves, document VCEG-M33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001. [16] D. Grois et al, Performance Comparison of H.265/ MPEG-HEVC, VP9, and H.264/ MPEG-AVC Encoders, available on: http://iphome.hhi.de/marpe/download/performance_hevc_vp9_x264_pcs_2013_preprint.pdf [17] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html [18] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html [19] HEVC white paper-ittiam Systems: http://www.ittiam.com/downloads/en/documentation.aspx [20] HEVC white paper-ateme: http://www.ateme.com/an-introduction-to-uhdtv-and-hevc [21] HEVC white paper-elemental Technologies: http://www.elementaltechnologies.com/lp/hevc-h265-demystified-whitepaper [22] White paper on PSNR-NI: http://www.ni.com/white-paper/13306/en/
References(Continued) [23] Test Sequences: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ [24] Access to HM 13.0 Reference Software: http://hevc.hhi.fraunhofer.de/ [25] Access to JM 18.6 Reference Software: http://iphome.hhi.de/suehring/tml/ [26] Website on PSNR: http://en.wikipedia.org/wiki/peak_signal-to-noise_ratio [27] Website on SSIM: http://en.wikipedia.org/wiki/structural_similarity [28] Access the website http://www-ee.uta.edu/dip/courses/ee5359/ and refer to the project by S. Kulkarni on Transcoding from H.264/AVC to High Efficiency Video Coding (HEVC), University of Texas, Arlington, Spring 2013. [29] Access the website http://www-ee.uta.edu/dip/courses/ee5359/ and refer to the project by H. B. Jain on Comparative and performance analysis of HEVC and H.264 intra frame coding and JPEG 2000, University of Texas, Arlington, Spring 2013. [30]Access the website http://www-ee.uta.edu/dip/courses/ee5359/ and refer to the thesis by S. Vasudevan on Fast intra prediction and fast residual quad-tree encoding implementation in HEVC, University of Texas, Arlington, 2013. [31] C. Fogg, Suggested figures for the HEVC specification, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012. [32] Z. Wang et al, Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE Transactions on Image Processing, Vol. 13, No. 4, pp. 600-612, Apr. 2004. [33] G. Sullivan et al, Standardized Extensions of High Efficiency Video Coding (HEVC), IEEE Journal of selected topics in Signal Processing, Vol. 7, No. 6, pp. 1001-1016, Dec. 2013. [34] N. Ahmed et al, Discrete Cosine Transform, IEEE Transactions on Computers, Vol. C-23, pp. 90-93, Jan. 1974.
Acronyms and Abbreviations AVC: Advanced Video Coding. BD-BR: Bjontegaard Delta Bitrate. BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. CABAC: Context Adaptive Binary Arithmetic Coding. CTB: Coding Tree Block. CTU: Coding Tree Unit. CU: Coding Unit. DBF: De-blocking Filter. DCT: Discrete Cosine Transform. HEVC: High Efficiency Video Coding. HM: HEVC Test Model. IEC: International Electro-technical Commission. ISO: International Organization for Standardization.
Acronyms and Abbreviations JCT-VC: Joint Collaborative Team on Video Coding. JM: H.264 Test Model. JPEG: Joint Photographic Experts Group. MPEG: Motion Picture Experts Group. MSE: Mean Square Error. PB: Prediction Block. PSNR: Peak Signal to Noise Ratio. QP: Quantization Parameter SAO: Sample Adaptive Offset. SSIM: Structural Similarity Index. TB: Transform Block. TU: Transform Unit.