Multidimensional Transcoding for Adaptive Video Streaming Jens Brandt, Lars Wolf Institut für Betriebssystem und Rechnerverbund Technische Universität Braunschweig Germany NOSSDAV 2007, June 4-5 Jens Brandt, Lars Wolf IBR, TU Braunschweig 1
Outline Introduction Multidimensional Video Transcoding Evaluation Conclusions & Future Work Jens Brandt, Lars Wolf IBR, TU Braunschweig 2
Adaptive Media Streaming for Mobile Devices Mobile devices: Smartphones, PDAs, pocket multimedia players 300 600 MHz CPU 64 128 MB RAM Small screen resolutions Great heterogeneity of mobile devices Video streams: High data rates Complex to decode Fixed encoding parameters Jens Brandt, Lars Wolf IBR, TU Braunschweig 3
Video Streaming Video adaptation to support different devices Different versions of video streams Traditional approach: Differently encoded versions of each stream on each server Dynamic approach: Individual adaptation on each server or in the network Jens Brandt, Lars Wolf IBR, TU Braunschweig 4
Adaptation Dimensions Spatial resolution: reducing the resolution of each frame Temporal resolution: reducing the number of frames Detail resolution: increasing the quantization level Bit stream syntax: changing the syntax of the stream Semantic: changing the content of a video... Jens Brandt, Lars Wolf IBR, TU Braunschweig 5
Decoder - Encoder Encoding parameters are defined at encoding time Cascaded Pixel Domain Transcoder (CPDT): Transcoder Decoder Data Manipulation Encoder very flexible, but computational intensive Improvements: Minimizing the needed decoding steps Reusing information from the original data Transcoding = manipulation in the compressed domain Jens Brandt, Lars Wolf IBR, TU Braunschweig 6
Multidimensional Transcoding Different specialized transcoding architectures exist A combined architecture is missing Multidimensional transcoding by combining existing transcoder architectures Focus of this work: resolution reduction in three dimensions Temporal Resolution Spatial Resolution Detail Resolution Jens Brandt, Lars Wolf IBR, TU Braunschweig 7
Transcoding in Three Dimensions Temporal Resolution by B-Frame Skipping P-Frame Skipping Motion Vector Refinement Spatial Resolution by Macro Block Aggregation Motion Vector Refinement Detail Resolution by Requantization Drift-Error Correction Jens Brandt, Lars Wolf IBR, TU Braunschweig 8
Transcoding in Three Dimensions Temporal Resolution by B-Frame Skipping P-Frame Skipping Motion Vector Refinement Spatial Resolution by Macro Block Aggregation Motion Vector Refinement Detail Resolution by Requantization Drift-Error Correction Inverse Motion Compensation needed Jens Brandt, Lars Wolf IBR, TU Braunschweig 8
Transcoder Chain Jens Brandt, Lars Wolf IBR, TU Braunschweig 9
Processing Architecture Pull architecture Passive transcoder modules: T i One active component: Controller Events: e i Jens Brandt, Lars Wolf IBR, TU Braunschweig 10
Implementation Prototype implementation in C++ Transcoding in three dimensions: temporal, spatial, detail Features: B-Frame Skipping P-Frame Skipping Downscaling by a factor of 2 Requantization Static Controller MPEG-4 Simple Profile Jens Brandt, Lars Wolf IBR, TU Braunschweig 11
Implemented Transcoder Chain Data exchanged between transcoders = partially decoded Frames Jens Brandt, Lars Wolf IBR, TU Braunschweig 12
Frame States Jens Brandt, Lars Wolf IBR, TU Braunschweig 13
Testvideos Source: 12 seconds (300 frames, 25 fps, 30 frames GOV length) CIF (352x288) reolution Encoded by XviD Akiyo Foreman Transcoded to 150 frames, 12.5 fps, QCIF Jens Brandt, Lars Wolf IBR, TU Braunschweig 14
Akiyo - Average PSNR Values 31 Akiyo recoded Akiyo transcoded 30 PSNR 29 28 27 26 0 20 40 60 80 100 120 140 160 180 Bit Rate [KBit/s] Jens Brandt, Lars Wolf IBR, TU Braunschweig 15
Foreman - Average PSNR Values 29 28 Foreman recoded Foreman transcoded 27 26 PSNR 25 24 23 22 21 20 0 100 200 300 400 500 600 700 Bit Rate [KBit/s] Jens Brandt, Lars Wolf IBR, TU Braunschweig 16
Runtime Performance Intel Pentium 4 CPU 3.20GHz, 512 MB RAM: Sequence Processing Time Processed Frames Akiyo 4.31 s 34.80 fps Foreman 10.71 s 14.01 fps Transcoded video: 150 frames, 12.5 fps, 12 s Jens Brandt, Lars Wolf IBR, TU Braunschweig 17
Conclusions & Future Work Flexible processing architecture for multidimensional transcoding Three transcoding dimensions for fine grain adaptation Transcoder implementation for this dimensions Promising evaluation results Future Work Optimization to improve runtime Further features of MPEG-4 to reduce bit rate Dynamic rate control mechanisms Quality evaluation by subjective tests Jens Brandt, Lars Wolf IBR, TU Braunschweig 18
Questions? Jens Brandt, Lars Wolf {brandt wolf}@ibr.cs.tu-bs.de Institut für Betriebssystem und Rechnerverbund Technische Universität Braunschweig Germany http://www.ibr.cs.tu-bs.de Jens Brandt, Lars Wolf IBR, TU Braunschweig 19