Beyond Macroblocks in Lossy Video Compression

Transcription

1 Beyond Macroblocks in Lossy Video Compression Claudio Cappelli 1 1 Eco Controllo SpA Centro Direzionale, Isola E5, Napoli (NA), Italy claudio.cappelli@ecocontrollo.com Abstract In this paper we describe the performances of a new proprietary lossy video compression technique with respect to those of main competitors. Since our codec is covered by industrial secret, we only sketch the basic principles of the compression algorithms underlying it, while discussing the key advantages with respect to macroblocks based techniques, on which most current codecs rely. 1. Introduction Recent years have witnessed a remarkable growth of digital image processing applications. This is mainly due to several factors, including the development of Internet, digital TV, the increasing prevalence of mobile devices with high multimedia capabilities, and so forth. Although seemingly different, these application domains have a common denominator: the need to transmit still and / or moving images through different types of network infrastructures, wired or wireless that, given the average size of objects to be transmitted, has yielded an exponential growth of bandwidth needs. The problem can be faced by first acting on the increase in bandwidth and on the other hand on the compression of multimedia objects to be transmitted. In the first case, however, we often clash with budget constraints, long realization time (reduced in part by the adoption of wireless network infrastructures), as well as maintenance problems, which have often led to delays in development or simply in the worldwide diffusion of certain applications. Thus, although the development of network links is a key aspect for the success of these multimedia applications, the pursuit of satisfactory targets with adequate times and costs is undoubtedly linked to the development of efficient compression techniques. In the last decade, many video codecs have been created. Among these, MPEG-4 and H.264 have received particular attention [10], and many applications have exploited them. More recently, particular interest is devoted to VP8, a video compression format owned by Google and originally created by On2 Technologies. Codecs have played a central role in competitions among big commercial brands (think about recent contests between Google and Apple, as well as Microsoft commercial strategies to include Windows Media Player within its operating systems), all focusing on the promotion of proprietary codecs as a means to build customer loyalty and to gain market shares. This yields a trend of these brands to highlight slight technological differences with respect to their competitors, claiming the exclusivity of their products to customers, and focusing on innovation features that should be properly advertised. All these codecs rely on the concept of macroblock, a square portion of a video frame whose size depends on the codec and is usually a multiple of 4, and in modern codecs is about 16x16 pixels. Usually, codecs analyze a macroblock trying to guess the macroblock in the next frame that best approximates it [10]. This might have different coordinates due to the dynamics within the scene. If such an approximate search succeeds, the codec might avoid to fully encode the macroblock twice for the two frames. Rather it might store it once for the first video frame, together with a motion vector to reconstruct the position that the macroblock has on the next frame, plus the pixels that have changed. For instance, MPEG-4 might fully encode a macroblock, or just store the information enabling its reconstruction from a macroblock in a previous video frame, or in combination from a previous and a following video frame. This mechanism is particularly sophisticated. For some video contents it might be difficult encoding macroblocks in terms of motion vectors and differences with respect to other

2 macroblocks, which might entail fully encoding of video frames, yielding a tremendous increase in the required bit rate. In this paper we describe the basic principles and performances of a new proprietary codec. The codec has been produced within a research project at Eco Controllo s.p.a., and it is based on a completely new idea. In particular, it does not rely on macroblocks, rather it exploits the grayscale component of images to partition them and to guide the prediction of basic objects in a scene and their motion. In the past, it has already been experimentally shown that this prediction method yields more efficient video coding, allowing to reach a better tradeoff between bit rate and quality of the compressed images [1]. Due to industrial secret, in this paper we only provide some hints on the ideas underlying our compression algorithms. However, we provide further experimental evidence of their potential advantages. In particular, we describe results of recent tests we entrusted to external research institutions, in which the latest version of our codec has been compared to VP8 [3]. The choice of VP8 has been motivated by the increased interest on this codec after Google purchasing its commercial rights from On2 Technologies. In such tests our codec outperformed VP8 both in terms of objective video quality metrics applied to compressed videos and in terms of compression time. The paper is organized as follows. Section 2 provides a sketch description of the ideas underlying our compression algorithms. In Section 3 we provide test results. Finally, Discussion is reported in Section Principles of the Eco Controllo s codec fashion, starting from the basic grayscale component up to fine color tones. Also MPEG somehow takes into consideration the layered nature of the colors in an image, although it exploits this characteristic to derive a better sampling of the image, processing images represented in YUV rather than in the RGB color model [10]. Like MPEG, also Eco Controllo s video codec relies on YUV color model. In particular, it exploits the separation of the luminance component (Y) from the two chrominance components (UV). The former alone is capable of producing grayscale images, whereas the second one measures the color values to be added to the gray pixels in order to derive a colored image. Thus, while it will always be possible to express a picture by only employing the Y component, achieving a grayscale picture, the converse will not be true. In other words, the U and V values only make sense if they are associated to some luminance value expressed by the Y component. This means that the 8 most significant bits of the YUV representation are the ones providing the skeleton of the picture (Figure 1(a)), whereas the other two less significant bytes, corresponding to the U and V components, respectively, add color details (Figure 1(b)). (a) Grayscale skeleton of an image. Although Eco Controllo s codec is covered by industrial secret [2], in this section we try to give an idea of its strength points. Eco Controllo s compression algorithms gain main advantages by using alternative techniques with respect to macroblocks. This is because macroblocks tend to partition images in a way that is independent of their contents, yielding an artificial partitioning method that might be more or less suitable depending on video contents, possibly making motion prediction a hard process. Conversely, Eco Controllo s video compression technique exploits the grayscale component of images to partition them and to guide the prediction of basic objects and their motion. The idea stems from the experimental observation that the colors in a picture tend to be distributed in a layered Figure 1. (b) True color representation of image (a). Grayscale and true color pictures in the YUV model.

3 3. Experimental Comparison Codecs are continuously compared to test their capabilities to preserve proper quality of video sequences while reaching high compression ratios. Most modern codecs have been experimentally compared at the CS MSU GRAPHICS & MEDIA LAB VIDEO GROUP [12]. In this section we describe the results of comparative tests that Eco Controllo has committed to external research organizations, in order to compare their codec with respect to VP8, the codec used on YouTube. Codecs might be tested by means of both subjective or objective metrics [17]. The latter try to simulate the visual quality as it is perceived by humans [4][5][6][10][11][13][14][15][16], whereas in the formers the evaluation is accomplished by a group of individuals, according to a given methodology [9]. The results we describe in this paper have been measured by comparing Eco Controllo s codec and VP8 through the Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity Index (SSIM) objective metrics [17]. In particular, Eco Controllo requested external research organizations to estimate the perceived "fidelity" of the compressed video (by means of the different codecs being compared) with respect to the original video, given a fixed value of bit rate. Moreover, it has been requested to repeat such tests for different frame rates (25 and 50 Hertz), and different values of bit rate, with respect to which compress each selected video, by using each of the codecs being compared. Test video sequences were selected by choosing typical high-definition sequences, many of which had been previously used in the context of similar experiments described in literature, including heterogeneous video sequences commonly used in television programs, hence realized with professional quality. In particular, test cases were selected among the following video test sets: HDTV (720p-50Hz and 25 Hz) SVT High Definition Multi Format Test Set [7] - Video sequences produced by the Swedish television channel SVT, also available on the Video Quality Experts Group (VQEG) web site ftp://vqeg.its.bldrdoc.gov/hdtv/svt_multiformat/. CIF - Test Set: Xiph.org. In particular the Derf collection, available at In order to execute tests, a sample of 20 video sequences (in what follows named 3, 4, 17, parkjoy, ducks, shields, parkrun, and mobcal) in the two different formats (720P@25Hz, 720P@50Hz) were selected. Finally, Tests were accomplished by using Linux OS (Ubuntu) on a Siemens Celsius V830 Workstation, RAM 8 GB, CPU 2 AMD Opteron 240, HD 2 HD SataII 400GB, VID NVIDIA Quadro FX MB. The computation of PSNR and SSIM was performed by using a workbench running on Windows 7. Video sequences named 3, 4,.17, were available in 720P@25Hz, and 720P@50Hz formats, respectively, whereas those named parkjoy, ducks, shields, parkrun, and mobcal were only available in 720P@50Hz format. Each available video sequence was compressed at 1000, 2000, 3000, and 4000Kbps, using both compared codecs, yielding a total of 280 different compressed files. Among these, only those having size ± 5\% than F have been considered, where br: bit rate per second in Kbps (1000 bit/sec) s: video duration in seconds F: file size in KBytes The 280 different compressed video files were successively evaluated through the PSNR and SSIM metrics, by using the MSU Video Quality Measurement Tool rel. 1.4, produced by the Graphics & Media Lab Video Group of Moscow State University [12]. Table 1 and Figure 2 show the SSIM averaged over the above mentioned video resolutions and bitrates. Table 2 and Figure 3 show average PSNR. It can be noticed that Eco Controllo s codec reaches better objective quality values. In particular, Figure 2 shows how the gap between the two codes is broader on the last five test video sequences. This is due to the fact that such video sequences have characteristics that stress video codecs, creating the conditions in which Eco Controllo s codec performs better. We noticed similar trends when using low bitrates, where Eco Controllo s codec degraded performances less than VP8. Such results are not achieved by devoting more time for compression, since Eco Controllo s codec has turned out to execute much faster than VP8 on the selected test video sequences. This is also shown in Table 3 and Figure 4, reporting average time in seconds to compress the 20 video sequences with the two compared codecs.

4 Average SSIM Video Sequence VP8(IVF) EcoControllo 03 0, , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,93480 parkjoy 0, ,79144 ducks 0, ,80566 shields 0, ,92825 parkrun 0, ,84549 mobcal 0, ,94903 Comprehensive Average 0, ,91270 Table 1. Average SSIM comparison between EcoControllo (ECO10) and VP8(IVF). Average PSNR Video Sequence VP8(IVF) EcoControllo 03 30, , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,36191 parkjoy 24, ,75025 ducks 24, ,24344 shields 33, ,29050 parkrun 24, ,58961 mobcal 34, ,41307 Comprehensive Average 33, ,76193 Table 2. Average PSNR comparison between EcoControllo and VP8(IVF). Figure 2. Average SSIM comparison between EcoControllo (ECO10) and VP8(IVF). Figure 3. Average PSNR comparison between EcoControllo (ECO10) and VP8(IVF).

5 Average Compression Time (Seconds) Video Sequence VP8(IVF) Eco , , , , , , , , , , , , ,

6 Technical Report, University of Salerno, November [4] M. P. Eckert and A. P. Bradley, `Perceptual quality metrics applied to still image compression', Signal Processing, vol. 70, pp , Nov [5] A. M. Eskicioglu and P. S. Fisher, `Image quality measures and their performance', IEEE Trans. Commun., vol. 43, pp , Dec [6] B. Girod, `What's wrong with mean-squared error', in Digital Images and Human Vision, A. B. Watson, Ed. Cambridge, MA: MIT Press, pp , quality\_index/demo.html. [15] Z. Wang and A. C. Bovik, `A universal image quality index', IEEE Signal Processing Letters, vol. 9, pp , Mar [16] S. Winkler, `A perceptual distortion metric for digital color video', in Proceedings of SPIE, vol. 3644, pp , [17] Z. Wang, A. C. Bovik, and L. Lu, `Why is image quality assessment so difficult', in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, vol. 4, Orlando, FL, pp , May [7] L. Haglund, `The SVT High Definition Multi Format Test Set', Sveriges Television, 2006: ftp://vqeg.its.bldrdoc.gov/hdtv/svt_multifo rmat/svt_multiformat_v10.pdf. [8] S. Ishihara. `Tests for colour-blindness', Handaya, Tokyo, Hongo Harukicho, [9] ITU-R. Methodology for the subjective assessment of the quality of television pictures, RECOMMENDATION ITU-R BT , 1-48, [10] I.E.G. Richardson, H.264 and MPEG-4 video compression - Video Coding for Next-generation Multimedia, Wiley [11] P. C. Teo and D. J. Heeger, `Perceptual image distortion', in Proceedings of SPIE, vol. 2179, pp , [12] D. Vatolin, D. Kulikov, A. Parshin, MPEG-4 AVC/H.264 Video Codecs Comparison, Report of GRAPHICS & MEDIA LAB VIDEO GROUP at Moscow State University (MSU), rison/index_en.html. [13] Z. Wang, `Rate scalable Foveated image and video communications', Ph.D. dissertation, Dept. Elect. Comput. Eng., Univ. Texas at Austin, Austin, TX, Dec [14] Z. Wang. `Demo Images and Free Software for ``a Universal Image Quality Index'''. Available: