4K DELIVERY TO THE HOME Pierre Larbier, CTO ATEME, France ABSTRACT The broadcast industry is completing the transition from standard definition to high definition all around the world. But demand for an even enhanced end-user experience with high fidelity content especially for premium events like sports is rising. High fidelity means that the remote viewing experience is as close as possible to watching the event on-site, which covers visual experience as well as audio/sound fidelity. Quad Full High Definition or 4k2k could be the answer, as cameras and TV sets start to be commercially available. But there is still a large uncertainty in other parts of the broadcast chain that include video compression and delivery. Assuming 4k content were available, what should be the bandwidth required to broadcast it? What is the right compression technology? After providing an accurate definition of what 4K means and presenting the challenges for a complete broadcast chain, this paper will evaluate H.264/AVC and HEVC as codec candidates to support early cable or satellite deployments. INTRODUCTION The analogue switch-off, completed in most developed countries in the past few years, and the availability of many digital broadcast means like satellite, IP-TV or Digital Terrestrial Television (DTT) paved the way to even more immersive audio and video formats. But as of today, and for most viewers around the globe, television represents the availability of a large number of HD and SD channels, with an on-going transition to all-hd. As the television history demonstrates, there is a continuous demand for enhanced enduser experience. This is especially true with premium content like TV shows and sports events. This motivates the definition of "higher fidelity" audio and video formats which enable the experience of watching the content as close as possible to the on-site event. The promise of a more immersive viewing experience in the home could be fulfilled by increasing video resolution and frame-rate, and improving sound spatialization. The next natural step after HD would be 4k where video resolution is quadrupled. But while many 4k devices like TV sets and cameras are made available in 2012, no established standard defines accurately what 4k means in the context of broadcast television. Video compression of this new format has still not been demonstrated on current broadcast delivery infrastructures, and very little is known about the bit-rates that could be envisioned for live applications.
This paper presents a 4k format definition that could be used in broadcast television applications. In addition, it outlines the likely benefits, and also the challenges our industry may expect from this new format. Should we assume that the bandwidth required for 4kTV will increase in linear proportion to the pixel count? Should we assume that 4kTV mandates a new more efficient than H.264/AVC compression standard to leverage the existing broadcast infrastructure? While access to 4k content is still limited, our research offers early insights into these questions. WHAT IS 4K ANYWAY? 4k formats There is currently no commonly accepted audio-video format targeting broadcast television applications beyond HD. Consequently, several different specifications, proposed by several standards bodies, research organizations and manufacturers co-exist. A notable research effort, Super Hi-Vision also known as 8k, led by NHK, aims at specifying the ultimate 2D video and associated audio formats that could be deployed as soon as 2020. This is still a work in progress but the current trend is to use progressively scanned pictures of 7680x4320 pixels transmitted at 120Hz. This represents 64 times more pixels per second than 1080i HD. The audio format definition followed the same criteria as the video which led to the specification of a 22.2 audio system. This is 3 times more channels than the 5.1 multi-channel audio system that comes today with most HD transmissions. But many feel that a 64 times increase in the number of pixels processed per second is too big a gap to allow near term deployment. Consequently lower resolution formats may gain an economic viability much sooner. It should be noted that this difficulty has been anticipated by ITU-R and Wood (5) indicates that an intermediate 4k video format is currently being defined as UHDTV1. But as ITU-R work progresses, several optional capabilities like frame frequencies ranging from 24 to 120Hz have been added to the base definition. This leads to a very wide specification that may find difficulties to be adopted as a whole. In the context of the Digital Cinema Initiative (DCI), the cinema industry has developed a parallel work to define suitable formats for digital representation of movie content (1). The highest format, currently in use in an ever increasing number of movie theatres, requires a progressive resolution of 4096x2160 displayed at 24 frames per second. Regarding audio, the DCI format defines 16 audio channels, but only 5.1 and 7.1 multi-channel configurations are specified. The adoption of the 4k DCI video format for television applications may not be straightforward for two main reasons. First, to limit errors along the video processing chain, square pixels and 16/9 display aspect ratios were chosen when designing HD formats. Changing one of those two parameters would significantly complicate interoperability when broadcasting legacy content. Secondly, the display of sharp 24Hz videos on large screens causes annoying jerky artifacts. Modern TV sets and video projectors add temporal interpolation methods to mitigate the perceived impact. But what is the point of increasing the resolution if most of the displayed pixels are interpolated? Another way to reduce
jerkiness is to add motion blur to the transmitted video. But again, there is little gain from increasing the resolution if the video then goes through a low pass filter. To solve the aspect ratio issue, Quad Full High Definition (QFHD), also known as 4k2k or 4kTV, which is defined as 3840x2160 16/9 frames, is most probably a better choice for broadcast television applications. Width and height are exactly twice that of 1080i HD resolution, so pixel and display aspect ratios are preserved. As we will show later, this format provides significant improvements over HD, with the advantage of being relatively easier to deploy than 4k DCI in the forthcoming years. The choice of a preferred frame-rate is a debated topic for many years. Several studies and reports Yamashita et al (2), Salmon et al (3) indicate that frame-rates in the range of 100Hz are needed to avoid most of temporal artifacts. But such high frame-rates raise important technical difficulties all along the production chain, and could consequently jeopardize an early adoption. A way to limit the frame-rate is to use interlaced formats, but this would introduce inefficiencies in the video compression system and add specific artifacts as well as complexity in TV sets. So as a trade-off, progressive scanning at 50Hz or 59.94Hz is most probably a better choice for 4k television systems. Audio format causes much less difficulties in the distribution chain since bitrates are significantly lower than what is required for video. Consequently, the audio format specification provided by DCI is also probably applicable to broadcast television applications. This would limit multi-channel audio to 7.1, which would cap the cost and complexity of the audio rendering system in the home compared to the 22.2 audio mandated with UHDTV. In addition, compression and transport of 7.1 multi-channel audio is already in use in a variety of application, so it does not require new development or standardization effort to be deployed. Consequently, it is expected that 4k for broadcast television applications will be defined as 3840x2160 progressively scanned 16/9 pictures, transmitted at 50Hz or 59.94Hz, with multi-channel audio up to 7.1. This simple definition implies to process exactly 8 times more pixels par second than current 1080i HD systems. 4k benefits The screen size required to benefit from a given resolution has been extensively studied over the years with somewhat contradictory results. Drewery and Salmon (4) experiments have shown that it takes a screen size of 76 inches or more, at a viewing distance of 2.7m, to provide any visual advantage in terms picture sharpness. But in the course of Super Hi- Vision development, NHK studies have shown that human visual detail perception could be much better than anticipated, leading to immersive impression ("sense of realness" and "sense of being there") with much smaller screen sizes. And in (5), it is expected that 4k would provide the same kind of improvement over HD, than what was observed between SD and HD. In addition, demonstrations performed at CES 2012 have shown that 4k resolution outperformed HD, even on 60 inches screens. One key aspect of increasing the resolution is the ability to keep detailed areas while providing a much larger view of the scene. As it is shown on Figure 2, the character outlined in Figure 1 looks very different when displayed at various resolutions on the same screen. One main characteristic of an immersive viewing experience is the ability to provide fine details on every part of a large scene, and it is exactly what 4kTV offers.
Figure 1 View of a large scene 4k Challenges Figure 2 Character detail at various resolutions The two ends of the broadcast chain, cameras and TV sets, were very recently made available but most of other devices like switchers, encoders and set-top boxes are yet to be announced. The first challenge in deploying 4kTV is in baseband: How to acquire, transport and display uncompressed video? A very limited number of 4k cameras suitable for television applications are currently available. Most of them were designed for movie, drama or documentary applications, but could also be used for traditional live television events. As it was shown, 4k involves exactly 8 times more pixels than 1080i HD. Assuming 4:2:2 10-bit pixel format, this means that about 12Gbits/s are needed to transport uncompressed video and audio. As of today, there is no established standard to handle that much data between professional equipment or between set-top boxes and TV sets. For this reason, the limited number of available 4kTV devices work around this bottleneck by aggregating
multiple currently available links like 3G-SDI or HDMI, in proprietary ways. However, standardization bodies are actively involved in the specification of new transport interfaces with higher capacities. 4k consumer TV sets, video projectors and professional monitors are available in 2012. However, since the 4kTV format is not yet standardized, there are a lot of differences between these devices: Some support only 4096x2160 (Digital Cinema format), others only 3840x2160 (4kTV format) Some are limited to 30Hz, other can display video up to 60Hz Some aggregate HDMI links, others make use of SDI links or DisplayPort. Consequently, it is today very difficult to build a 4kTV prototype broadcast chain and it will take at least a year before the standardization effort bears fruit and guarantees compatibility. DOES 4K FIT ON CURRENT DISTRIBUTION ARCHITECTURE? How practical would it be to use H.264/AVC? Deployment of 4k to the home is only possible if the compressed bit-rate is compatible with the existing distribution architecture. The maximum available bit-rate for a single channel on DTT, satellite or cable distribution networks is between 24Mbits/s and 45Mbits/s. HD channels that are broadcast using the H.264/AVC (6) video compression standard are encoded at bitrates ranging from about 5Mbits/s to 10Mbits/s. This enables between 3 and 8 HD channels per satellite transponder or QAM. Since 4kTV requires 8 times more pixels to process and transmit than 1080i, one could think that it would require 8 times more bitrate. If this was the case, it would be almost impossible to fit a single 4kTV channel on a transponder with a video quality compliant to broadcast standards. Fortunately, an equivalent perceived video quality is achieved at a much lower bit-rate than what could be guessed by the pixel count ratio. For instance, good quality is obtained in SD at about 2Mbits/s while HD requires only about 6Mbits/s. The bit-rate ratio is 3 but the pixel count ratio is 5 (in 25Hz interlaced systems). To verify the bit-rate ratios applicable between 4k and HD, we have encoded in H.264/AVC a test sequence acquired natively in 4k at 50Hz from a RED Epic camera. This sequence was downscaled to 1080p50 and 720p50 and we measured the bit-rate to reach the same objective video quality. Figure 3 present the measurement results and Table 1 the computed bit-rate ratios. It is interesting to note that, like in the SD vs HD case, the required bit-rate to obtain the same quality is much lower than what could be anticipated: Less than 20Mbits/s are effectively needed to match the 720p video quality PSNR metric. This suggests that a single channel could be transmitted on a DVB-T transponder and 2 channels can easily fit using 45MBps-QAM modulation, all using just H.264/AVC compression. Additional tests with scanned films and UHDTV sequences, using objective and subjective analysis methods show that source noise may have a significant impact on video quality when using H.264/AVC compression. Signal to noise ratios of upcoming 4k cameras is not known at this stage, but it could be anticipated that de-noising techniques would be sometimes required to keep a "broadcast quality" at bitrates around 20Mbits/s in H.264/AVC.
Nevertheless, 4kTV broadcasting can be performed on most delivery networks, using the current H.264/AVC codec. However, the bit-rate might be too high for massive deployments. In addition, one valuable advantage of using H.264/AVC is the announced availability of chips able to decode 4kTV formats. This is the case for instance of Intel Ivy- Bridge family of processors. Figure 3 Rate-Distortion curves for Roland Garros test sequence Format Bit-rate at identical PSNR Pixels/s multiplier in a 4kTVp50 Bit-rate multiplier for a 4kTVp50 stream 720p50 6.5Mbits/s 9 2.5 1080p50 9Mbits/s 4 1.8 4kTVp50 (native) 16.5Mbits/s 1 1 Table 1 Bit-rates and ratios Next generation video codec The next generation video codec, High Efficiency Video Coding (HEVC) (7), is currently under heavy development and is planned to be released as an ISO/ITU standard in 2013. The target goal is to enable a two-fold reduction in bit-rate compared to H.264/AVC while keeping the same video quality. And initial objective and subjective tests on various 1080p broadcast content have shown that this target was close to being attained. This codec is particularly tailored toward high resolutions with block sizes that could be as large as 64x64, and high frame-rates with advanced motion vector estimation. This makes HEVC a very good candidate for 4kTV applications. Still we thought it would be meaningful to compare HEVC to H.264/AVC when processing native 4k videos.
A short version of the Roland Garros test sequence was encoded using the HEVC reference software and an ATEME H.264/AVC research software encoder. The graphs presented as Figure 4 indicate a bitrate gain of about 35%. Additional measurements done on scanned film sequences exhibit slightly higher gains, leading to the consideration that HEVC handles noisy sources more efficiently than H.264/AVC. Figure 4 HEVC vs H.264/AVC Rate-Distortion graphs The observed gain would allow the broadcasting of 4kTV at video bitrates below 13Mbits/s. This figure is very similar to bitrates used when transmitting HDTV in MPEG-2 and just about twice the value commonly used with H.264/AVC for HDTV. Therefore, HEVC compression seems to enable commercial deployments of 4kTV broadcast on a much larger scale than what could be achieved with current video codecs. CONCLUSION We have outlined a 4k format that could be used in broadcast television applications and could improve significantly the viewing experience: 3840x2160 progressively scanned 50Hz or 59.94Hz Multi-channel audio, up to 7.1 The challenges to implement this format are far from negligible. Only the two ends of the broadcast chain, cameras and TV sets support 4K to date, yet not always in a fully interoperable fashion. At least for the year to come, it is felt that building a complete broadcast chain will require a considerable research effort. Using recently shot 4k test sequences, we have shown that H.264/AVC could be used as a video compression codec on existing broadcast delivery architecture, However, the 3 times increase in bit-rate over currently deployed HD channels could limit the adoption of 4kTV to high-end premium channels. It was also shown that the coding efficiency gains
provided by the upcoming HEVC codec could reduce the required bit-rates by a factor of almost 2 over H.264/AVC, bringing it back in line with current MPEG-2 data rates for HDTV. This could position HEVC as the ideal codec for an adoption of 4kTV as the successor of HDTV. REFERENCES 1. Digital Cinema Initiatives, LLC. Digital Cinema Specifications version 1.2, March, 2008 2. T. Yamashita et al. 2011. "Super Hi-Vision" video parameters for next-generation television. SMPTE Conf Proc 2011. 3. R. A. Salmon et al, 2011. Higher Frame Rates for More Immersive Video and Television. BBC White Paper WHP 209. 4. J.O. Drewery, R.A. Salmon, 2004. Tests of visual acuity to determine the resolution required of a television transmission system. BBC White Paper WHP 092. 5. D. Wood, 2012. The Strategic Impact of Ultra High Definition Television. NAB Broadcast Engineering Conference Proceedings, 2012. 6. ISO/IEC 14496-10:2010 Information technology - Coding of audio-visual objects - Advanced Video Coding 7. Joint Collaborative Team on Video Coding (JCT-VC), 2012. High efficiency video coding (HEVC) text specification draft 6. JCTV-H1003. ACKNOWLEDGEMENTS The author would like to thank his colleagues Jérôme Viéron and Jean-Marc Thiesse for their contributions to this work.