2007 IEEE International Conference on Signal Processing and Communications November 24-27, 2007, Dubai, United Arab Emirates

Transcription

1 PROCEEDINGS 2007 IEEE International Conference on Signal Processing and Communications November 24-27, 2007, Dubai, United Arab Emirates Region 8 UAE Section UAE SP/COM Chapter 2007 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. IEEE Catalog Number: 07EX1787C, ISBN: , Library of Congress: For any problems or questions related to the usage of this CD please contact: ICSPC07 Secretariat Tel: , Fax: secretariat@icspc07.org

2 2007 IEEE International Conference on Signal Processing and Communications (ICSPC 2007), November 2007, Dubai, United Arab Emirates A REAL-TIME H.264 BP DECODER BASED ON A DM642 DSP F. Pescador, M. J. Garrido, C. Sanz, E. Juárez, A.M. Groba and D. Samper {pescador, matias, cesar, ejuarez, amgroba,dsamper}@sec.upm.es Universidad Politécnica de Madrid. Ctra de Valencia, Km , Madrid (Spain) ABSTRACT In this paper, the implementation of a Baseline Profile H.264 decoder based on a DM642 digital signal processor is described. An initial standard compliant raw-c decoder has been optimized in speed for the target processor. The parallelism between algorithm execution and data movement has been fully exploited using. Also, critical parts of the algorithm have been encoded directly in assembly code to increase the number of instructions per cycle. The decoder has been tested in simulation with actual (transcoded) DVD and digital TV streams. According to these tests, standard definition real time decoding can be obtained with a DM642@600MHz 1 Index Terms H.264; Digital Signal Processor; video codecs; digital TV; real time systems. 1. INTRODUCTION In the last years, new video coding standards [1,2] have been adopted allowing more data compression but increasing the complexity for both encoders and decoders as well [3]. Non-programmable decoders offer enough computational power at a low cost but cannot cope with the quick evolution of the video decoding algorithms. On the other hand, the latest generation of Digital Signal Processors (DSPs) [4-6] can support very flexible decoders like the multi-format one proposed in [7] at a relative low cost. The complexity of an H.264 (MPEG-4/AVC) decoder may increase by a factor up to 2 regarding to MPEG-4 SP [3], which in turn is more complex than an MPEG-2 decoder. With these figures, a real-time H.264 standard definition DSP-based decoder is still a challenge [7-9]. In this paper, the implementation of a Baseline Profile 2 H.264 decoder based on a DM642 DSP is described. In section 2, an overview of the DM642 DSP architecture is outlined. Section 3 explains the decoder software architecture. Section 4 describes the tests performed. Finally, section 5 is devoted to the conclusions. 1 This work was supported by the Spanish Ministry of Science and Technology under grants TEC C02-01 and TIC C02-01 and by the Comunidad de Madrid regional government under grant S-0505/TIC/ Arbitrary slice order and multiple slice groups not supported. 2. DM642 ARCHITECTURE The DM642 is a fixed point video-oriented DSP [6]. The CPU is an VLIW processor with a performance of up to MHz. There are two 16 kb level-1 caches for code (L1P) and data (L1D). A 256 kb internal SRAM can be splitted into a level-2 cache (L2) and an internal data/program memory. The external memory is accessed through a dedicated interface, EMIF, using a 64-bit data interface. The other peripherals are a controller, two video ports, an Ethernet port (EMAC), an output audio interface (McASP) and several general-purpose I/O pins (GPIO). The controller allows moving data between memory and peripherals. transfers can be requested by L1P, L1D, L2, the user and the peripherals. The user programmable requests can be Q (Quick ) and E (Enhanced, more flexible but slower). 3. H.264 DECODER IMPLEMENTATION The decoder implements the Baseline Profile of H.264 video coding standard [2, 10-12] at level 3. The starting point was a standard compliant raw-c decoder fully tested first in a PC environment and moved to the DSP environment afterwards. This initial code was optimized to increase the execution speed in about two orders of magnitude. The decoder reads the H.264 stream from an input buffer and decodes the Network Adaptation Layer (NAL) units in sequence. In Fig. 1, a simplified flow diagram of the decoding process for an NAL unit is shown. After decoding the NAL header, the NAL unit content is identified as a slice header or another syntax element (e.g. an SPS or a PPS, see [2, 10, 11] for details). Virtually all computational load is consumed in the slice decoding process. After processing the slice header a loop is performed to decode every in the slice. First, up to 16 motion vectors are decoded. After each vector is obtained, three Q requests are started to move the luma and chroma reference data from one or more picture buffers in external memory to the REF buffer in internal memory (see Fig. 2). After starting all requests, the Inverse Cosine Transform () coefficients are obtained and stored in the internal memory _COEFFS buffer (Fig. 2). The is computed using _COEFFS buffer as both source and /07/$ IEEE 1491

3 destination. Following, data in REF are used to compute the reference and the result is stored in REC_N, a ping-pong buffer in internal memory (N can be 0 or 1, see Fig. 2). Afterwards, the motion compensation is performed by adding REC_N and _COEFFS and storing the results in REC_N. The deblocking filter is applied to the data stored in REC_N. Finally, three Q requests are started to move luma and chroma from REC_N to a picture buffer in external memory. Fig. 2. Memory usage in the decoder Data movement from reference pictures In H.264, different block sizes are used in the encoding process. Fig. 3-a shows the different block types supported in the standard while Fig. 3-b is an example of how these types can be combined in an. 8x16 8x8 (a) Block sizes (b) An example Fig. 3. Block sizes allowed in H.264. Fig. 1. Simplified diagram of the decoding process. As it was said before, the reference data pointed by the motion vectors are moved from the reference picture buffers to the REF buffer. Actually, there are three REF buffers, one for luminance data (REF_Y) and two for chrominance data (REF_CR and REF_CB). The REF_Y buffer is 1728 bytes length. This room is enough to store all the reference blocks and their borders in the worst case (sixteen 4 4 blocks). The blocks must be moved with their borders because 1/4 pel arithmetic may be further applied to them. The former description applies only for INTER coded s. For INTRA coded s there are neither motion vectors nor reference data. Instead, an prediction is computed and the results are stored directly in the REC_N buffer before adding the results from _COEFFS. More specific details about the decoding process are given in the next sub-sections. Data movement will be explained in detail because it plays a central role in the optimization process Use of memory and data flow Fig. 2 summarizes the DSP memory usage in the decoder for luma (additional chroma buffers, not shown in Fig. 2, are used). The current and reference pictures are stored in external memory because of their size. REF, _COEFFS and REC buffers are located in internal memory (internal SRAM is configured as 192 Kbytes of internal memory plus 64 Kbytes of L2 cache). (a) Block sizes with borders (b) The REF_Y buffer Fig. 4. Contents of the REF_Y buffer. 1492

4 As an example, Fig. 4-a shows, only for luminance, all possible block sizes (according to Fig. 3-a) with their borders, and Fig. 4-b shows the contents of the REF_Y buffer for a for a reference used to predict an like the one in Fig. 3-b. It must be noted that, in Fig. 4-b, the blocks spend more room than their counterparts in Fig. 4-a; this is because all transfers must be 32-bit aligned Deblocking filter related data flow (c) Prepare right border for next filtering. (d) Save unfiltered right border for next INTRA prediction The deblocking filter operation adds more complexity to the decoder data flow. To filter the current, the 4 bottom rows from the top and the 4 rightmost columns from the left (see Fig 5-a) are used. The REC_N buffers are dimensioned to have room for these pels (Fig 5-b). After the unfiltered current has been moved to the REC buffer, the following operations must be performed prior to filter: The 4 rightmost columns of the current REC buffer are moved to the 4 leftmost columns of the REC buffer that will be used in the decoding of the next (Fig 5-c). The rightmost column in current REC buffer is saved in a buffer allocated in internal memory (Fig 5-d). These data will be eventually used to compute the next prediction, if INTRA. The bottom row is also saved in an internal memory buffer (Fig 5-e). This information may be used to compute the INTRA prediction of the bottom (Fig 5-a) so it must be stored in a line (picture-width) size buffer. The 4 top bottom rows of the top are moved from an internal buffer to the current REC buffer (Fig 5-f). After these steps, the current REC buffer is ready so as the current can be filtered. After filtering, two operations must be performed: The filtered must be moved from the current REC buffer to the current picture buffer (Fig 5-g) using Q. The 4 bottom rows of the current REC are not moved to the current picture buffer. Instead, they are moved to an internal memory buffer (Fig 5-h) so as they will be available in the bottom filtering process. Actually, the 4 rightmost pels in each row are not saved until the next filtering operation. The chroma blocks are processed in a similar way using additional buffers. 4 (e) Save unfiltered bottom border for bottom INTRA prediction. (g) Save filtered in current picture buffer. (f) Gets upper border prior to filtering. (h) Save filtered bottom for bottom filtering. Fig. 5. Deblocking filter related data flow Parallelization Fig. 6 shows how data movement and decoding operations have been parallelized. As said before, up to 16 3 Q requests are used to move data from the reference pictures to the REF buffer. The movement of the reference data is first parallelized with the VLC decoding of the remaining vectors, secondly with Q requests and afterwards with the CAVLC decoding of the coefficients and the computing itself. After an has been decoded and filtered completely, it is moved with Q transfers from REC_N buffer to the current picture buffer in external memory. The REC ping-pong buffer allows performing these data movement in parallel with the decoding of the next without waits... Decode Header MVs #X to REF buffer Coeffs to current frame -1 Figs 6-c, 6-f & 6-h -1 #X wait MC Arithmetic Deblock filter Decode Header +1 MVs #X+1 to REF buffer to current frame Figs 6-c, 6-f & 6-h Coeffs +1 X+2 Fig. 6. Parallelization of data movement with the decoding operations (not in scale)... t 4 16x Code optimization (a) Top, left & bottom pels used in the current filtering. (b) Structure of the REC_0 or REC_1 buffer. In order to reach real-time operation, additional optimization steps [13] have been carried out: 1493

5 The higher computational cost functions have been moved to internal memory. Also frequently accessed data have been moved to internal memory. Frequent arithmetic operations have been coded using intrinsic (pseudo-assembler) instructions. The core of the deblocking filter,, CAVLC and MC has been also encoded using intrinsics. 4. TESTS Tests have been carried out using actual DVD movies like Star Wars: episode I and Finding Nemo and a football sequence taken from a digital TV channel. Using them, H.264 test streams were generated with a commercial encoder 3. The test-bench is shown in Fig. 7. The test streams, stored in a file, are read at a picture basis and written in a stream buffer allocated in external memory. The decoder reads the stream from this memory decodes a picture and writes it in a picture buffer. The picture is also written in a file. Fig. 7. Test-bench used to profile the decoder. A PC running Code Composer Studio v3.1 has been used to carry out the simulation profiles. Table 1 contains the profiling results using the sequences mentioned above. Results are given in average clock cycles per frame for the full decoder and for its main functional blocks (CAVLC, +MC, filter and others). The %CPU row shows the percentage of CPU load with a 600 MHz clock for the full decoder. 5. CONCLUSIONS In this paper, the implementation of a optimized Baseline Profile H.264 decoder based on a DM642 digital signal processor has been presented. The three most important contributions presented in this paper to improve the decoder performance are the following: to implement in a novel way the deblocking filter to use only the internal memory, to manually code different parts of the algorithm in parallelized assembly language and to parallelize data movement and algorithm execution using Q. Simulation profiles using different actual (transcoded) DVD and TV test streams state that the 3 All sequences have been generated with an AMK 430 AVC encoder from ATEME. Length: 100 pictures. Format: fps. Averaged bit rate: 2 Mb/s ( Nemo1M has a 1 Mbps avg. bit rate), 5% I, 95% P. All are available online at decoder is able to process 25 fps using about 75% capacity of a DM642@600MHz. Although, there are DM642-based commercial products supporting H.264 decoding [14-16], no comparison has been carried out due to the lack of enough performance information. Table 1. H.264 decoder performance in simulation. # cycles 10 6 Nemo Star Wars Football Nemo1M decoder CAVLC MC filter others %CPU 75.4% 71.7% 72.1% 59.6% 6. REFERENCES [1] ISO Information technology. Coding of audio-visual objects. Part 2: Visual. (May 2004). [2] ISO Information technology. Coding of audio-visual objects. Part 10: Advanced Video Coding (Dec. 2005). [3] Ostermann, J. et al. Video coding with H.264/AVC: tools, performance, and complexity. IEEE Circuits and Systems Magazine, Vol. 4, Issue 1, pp. 7-28, [4] Philips Semiconductors. Nexperia Media Processors. Available online at home/products/media_processors/index.html [5] Analog Devices. Blackfin processors. Available online at: [6] Texas Instruments. C6000 DSPs. Available online at: ly=dsp&sectionid=2&tabid=1941&familyid=1398 [7] Y.-S. Tung et al. "DSP-Based Multi-Format Video Decoding Engine for Media Adapter Applications". IEEE Trans. on Consumer Electronics, Vol. 51, Issue 1, pp , Feb [8] Vaidyanathan Ramadurai; Sanjay Jinturkar; Moudgill, M.; Glossner, J. Implementation of H.264 decoder on Sandblaster DSP. IEEE International Conference on Multimedia and Expo, ICME 2005, 5 pp. Jul [9] Moshe, Y.; Peleg, N. Implementations of H.264/AVC baseline decoder on different digital signal processors. 47th International Symposium ELMAR, pp , Jun [10] I.E.G. Richardson, H.264 and MPEG4 video compression. Ed. Wiley. [11] T. Wiegand, G.J. Sullivan, G. Bjøntegaard and A. Luthra, Overview of the H.264/AVC Video Coding Standard, IEEE Transaction on Circuits and Systems for Video Technology, Vol. 13, no. 7, Jul [12] JVT of ISO/IEC MPEG & ISO VCEC. Context-adaptive VLC (CVLC) coding of coefficients. Proposal JVT-C rd Meeting: Fairfax, USA, 6-10 May [13] F. Pescador et al. A DSP based IP set-top box for home entertainment. IEEE Transactions on Consumer Electronics Volume 52, Issue 1, Feb Page(s): [14] H.264 Software IP Suite for DSP. download.php?file=604. [15] Sentivision. [16] Ittiam Co. dec.htm. 1494