EE 5359 Multimedia Processing. Research Project Proposal- Performance analysis of H.264/MPEG4 based of different profiles

Transcription

1 EE 5359 Multimedia Processing Research Project Proposal- Performance analysis of H.264/MPEG4 based of different profiles Submitted By- Bhumika Makwana Under the guidance of Dr. K.R.Rao 1

2 Table of Contents 1. Background 3 2. Introduction 3 3. Technical Details 4 4. Profiles and Level References 10 Figures 1. H.264 encoder 4 2. Diagram depicting how the loop filter works on the edges of the blocks and sub-blocks Intra prediction 4x H.264 decoder 7 5. The specific coding parts of the Profiles in H A typical sequence with I-, B- and P-frames...9 2

3 Background: The number of devices using multimedia application is getting bigger day by day. End users constantly demand for rich multimedia content. For instance, user demanding for broadcast TV channel on mobile handset, user playing games online or surfing the web, user constantly checking stock exchange, etc. To meet this exponentially increasing demand for rich and high definition multimedia contents there is a need for advance codecs which must be capable of providing below features: Less network bandwidth Less storage space especially for video files Higher video quality for a given bitrates High resolution Less complexes Easy to implement One can find video compression technology in great diversity of products and services just because of its own advances in past 2 decades. From mp3 players to Blue Ray players, compressed video is now an indispensible part of our daily life. The new H.264 / AVC standard is at the forefront of this technology. H.264/MPEG 4 AVC codec introduction: H.264 is an open, licensed standard developed by the JVT (Joint Video Team) that supports the most efficient video compression techniques available today. Without compromising image quality, an H.264 encoder can reduce the size of a digital video file by more than 80% compared with the Motion JPEG format and as much as 50% more than with the MPEG-4 Part 2 standard. This means that much less network bandwidth and storage space are required for a video file. Or seen another way, much higher video quality can be achieved for a given bit rate[1] [2]. H.264 standard can deliver high-quality video to a variety of devices ranging from low-powered cell phones to high-powered Blu-ray devices because of its flexible bit stream control. This has enable the H.264 standard to supersede some of the video compression formats that are commonplace today. 3

4 Technical Details: Figure 1. H.264 encoder [3] 4x4 Integer transform The H.264 employs a 4x4 integer DCT as compared to 8x8 DCT adopted by the previous standards. The smaller block size leads to a significant reduction in ringing artifacts. Also, the 4 x 4 transform has the additional benefit of removing the need for multiplications. Quantization and scan The H.264 standard specifies the mathematical formulae of the quantization process. The scale factor for each element in each sub block varies as a function of the quantization parameter associated with the macroblock and as a function of the position of the element within the sub block. The rate control algorithm controls the value of the quantization parameter. Two types of scan pattern are used for 4x4 blocks one for frame coded macroblocks and one for field coded macroblocks. 4

5 Context-based adaptive variable length coding (CAVLC) and Context-based adaptive binary arithmetic coding (CABAC) entropy coding H.264 uses different variable length coding methods in order to match a symbol to a code based on the context characteristics. They are context-based adaptive variable length coding (CAVLC) and context-based adaptive binary arithmetic coding (CABAC). All syntax elements except for the residual data are encoded by the Exp-Golomb codes. In order to read the residual data (quantized transform coefficients), zig-zag scan (interlaced) or alternate scan (non-interlaced or field) is used. For coding the residual data, a more sophistical method called CAVLC is employed. Also, CABAC is employed in Main and High profiles, CABAC has more coding efficiency but higher complexity compared to CAVLC. Deblocking filter H.264 employs a deblocking filter to reduce the blocking artifacts in the block boundaries and stops the propagation of accumulated coded noise. The filter is applied after the inverse transform (before reconstructing and storing the macroblock for future predictions) and in the decoder (before reconstructing and displaying the macroblocks). The deblocking filter is applied across the edges of the macroblocks and the sub-blocks. The filtered image is used in motion compensated prediction of future frames and helps achieve more compression. Figure 2. Diagram depicting how the loop filter works on the edges of the blocks and sub-blocks [] Intra prediction During intra prediction, the encoder derives a predicted block based on its prediction with previously decoded samples. The predicted block is then subtracted from the current block and then encoded. There are a total of nine prediction modes (Figure 3) for each 4x4 luma block, four prediction modes for each 16x16 luma block and four modes for each chroma block. 5

6 Figure 3. Intra prediction 4x4[5] Inter prediction Inter prediction is performed on the basis of temporal correlation and consists of motion estimation and motion compensation. As compared to the previous standards, H.264 supports a large number of block sizes from 16x16 to 4x4. Moreover H.264 supports motion vector accuracy of one-quarter of the luma sample. Reference pictures Unlike the previous standards that just use the immediate previous I or P picture for inter prediction, H.264 has the ability to use more than one previous reference picture for inter prediction thus enabling the encoder to search for the best match for the current picture from a wider set of reference pictures than just the previously encoded one. 6

7 H.264 Decoder Figure 4. H.264 decoder ] It includes all the control information such as picture or slice type, macroblock types and subtypes, reference frames index, motion vectors, loop filter control, quantizer step size etc, as well as coded data comprising of quantized transform coefficients. The decoder of Figure works similar to the local decoder at the encoder; a simplified description is as follows. After entropy (CABAC or CAVLC) decoding, the transform coefficients are inverse scanned and inverse quantized prior to being inverse transformed. To the resulting 4_4 blocks of residual signal, an appropriate prediction signal (intra or motion compensated inter) is added depending on the macroblock type mbtyp (and submbtype) mode, the reference frame, the motion vector/s, and decoded pictures store, or in intra mode. The reconstructed video frames undergo deblock filtering prior to being stored for future use for prediction. The frames at the output of deblocking filter may need to undergo reordering prior to display. [6] Profiles and Levels: The H.264 / AVC standard is broken up into 3 "profiles": Baseline, Extended, and Main. These profiles are slightly different implementations of the compression technology within the same standard. In this way, the user can select one of the three profiles that is most synergistic with the intended application. Baseline profile: Designed with those applications in mind that run on the platforms with low 7

8 processing power and in an environment with large packet losses. Among the three Profiles, it has the least coding efficiency. Extended profile: A Super set of Baseline, more complex, and provides better coding efficiency than Baseline. Figure 1. The specific coding parts of the Profiles in H.264 [7]. Main Profile: Designed to provide the highest possible coding efficiency. Understanding frames Depending on the H.264 profile, different types of frames such as I-frames, P-frames and B- frames, may be used by an encoder. An I-frame, or intra frame, is a self-contained frame that can be independently decoded without any reference to other images. The first image in a video sequence is always an I-frame. I-frames are needed as starting points for new viewers or resynchronization points if the transmitted bit stream is damaged. I-frames can be used to implement fast-forward, rewind and other random access functions. An encoder will automatically insert I-frames at regular intervals or on demand if new clients are expected to join in viewing a stream. The drawback of I-frames is that they consume much more bits, but on the other hand, they do not generate many artifacts. A P-frame, which stands for predictive inter 8

9 frame, makes references to parts of earlier I and/or P frame(s) to code the frame. P-frames usually require fewer bits than I-frames, but a drawback is that they are very sensitive to transmission errors because of the complex dependency on earlier P and I reference frames. A B-frame, or bi-predictive inter frame, is a frame that makes references to both an earlier reference frame and a future frame.[1] Figure 6 A typical sequence with I-, B- and P-frames. A P-frame may only reference preceding I- or P-frames, while a B-frame may reference both preceding and succeeding I- or P-frames.[1] Why important to study profiles: All of H.264 profiles has its own benefit and disadvantages and based on the Rate Distortion curve a user can identify optimization point and accordingly can select the desired profile. In this project we will study these profiles in details including its implementation, tradeoffs, and application. As we have seen that it important to study not only the H.264 standard but also its profile. In this project I will do detail analysis of 3 profiles in terms if bitrate, PSNR, MSE, compression ratio. I will use JM software to encode 3 video clips; slow motion, fast motion, medium motion. Output of the encoded video will result in different PSNR, bitrate, MSE and compression for different profile and video clips. These data can be plotted on RD curve and can be compared against each other in order to choose the optimum profile for a given video/application. 9

10 References: [1] H.264 video compression standard, New possibilities within video surveillance, Axis communication. [2] Soon-kak Kwon, A. Tamhankar and K.R. Rao, Overview of H.264 / MPEG-4 Part 10 (pp ), Special issue on Emerging H.264/AVC video coding standard, J. Visual Communication and Image Representation, vol. 17, pp , Apr [3] P.N.Tudor, Tutorial on MPEG-2 Video Compression, IEE J Langham Thomson Prize, Electronics and Communication Engineering Journal, Dec [4] T. Wiegand et. al., Overview of the H.264/AVC Video Coding Standard, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, Issue 7, pp , July [5] S. Sharma, Transcoding of H.264 bitstream to MPEG 2 bitstream, Master s Thesis, May 2006, EE Department, University of Texas at Arlington. [6] S. Wagston and A. Susin, IP core for an H.264 Decoder SoC, 2007, Available at< [7] A. Puri, H. Chen and A. Luthra, Video Coding using the H.264/MPEG-4 AVC compression standard, Signal Processing: Image Communication, vol.19, pp , Oct [8] K. Sayood, Introduction to Data compression, III edition, Morgan Kauffmann publishers, [9] I.E.G. Richardson, H.264 and MPEG-4 video compression: video coding for nextgeneration multimedia, Wiley, [10] K. R. Rao and P. C. Yip, The transform and data compression handbook, Boca Raton, FL: CRC press, [11] K.R. Rao and J.J. Hwang Techniques and standards for image, video, and audio coding - Prentice Hall, [12] G. Sullivan, P. Topiwalla and A. Luthra, The H.264/AVC video coding standard: overview and introduction to the fidelity range extensions, SPIE Conference on Applications of Digital Image Processing XXVII, vol. 5558, pp Aug