BT Media & Broadcast. Research Paper

Transcription

1 BT Media & Broadcast Research Paper

2 BT Media and Broadcast Research Paper Ultra High Definition Video Formats and Standardisation Version 1.0 April 2015 Mike Nilsson Page 1

3 Contents Page 1 Introduction 6 2 Enhanced Resolution 7 3 High Dynamic Range The Benefit of High Dynamic Range The dynamic range of the human visual system The non-linearity of the human visual system The mapping of linear light to code levels The mapping of pixel code levels back to linear light Black level: how dark should displays be? The mapping of pixel code levels to linear light in the presence of ambient light The current state of high dynamic range capture and display technology Interest and Experience in Hollywood Dolby Cinema 22 4 Wider Colour Gamut The CIE RGB Colour Space The CIE XYZ Colour Space Perceptually Uniform Colour Spaces Colour Television The need for a Wider Colour Gamut Wider Colour Gamut Standards Conversion between Colour Gamuts How large a Colour Gamut is needed? Display Technology for Wider Colour Gamut 31 5 Higher Frame Rate The artistic impact of frame rate Historical choices of frame rate The motion blur jerkiness trade-off Subjective evaluation of moving picture quality 35 Page 2

4 5.5 The impact of lighting frequency on frame rate 38 6 Standardisation ITU-R Parameters for Digital Television High Dynamic Range Colorimetry conversion Higher Frame Rates IN STRICTEST 41 CONFIDENCE 6.2 SMPTE UHD Parameter Values High Dynamic Range Colour Volume Metadata Digital Cinema Colour Equations On-going Activities MPEG High Efficiency Video Coding The Future of Video Coding Standardisation High Dynamic Range and Wide Colour Gamut DVB UHD-1 Phase UHD-1 Phase UHD-2 Eco-design requirements for electronic displays IN STRICTEST 49 CONFIDENCE EBU DTG Blu-ray Disk Association HDMI Forum The Digital Cinema Initiatives (DCI) The Forum for Advanced Media in Europe (FAME) UHD Alliance and UHD Forum 54 7 Abbreviations 55 Page 3

5 BT Media and Broadcast contacts Name Business Area / Role Phone John Ellerton Head of Media Futures [email protected] Jonathan Wing Head of Sales [email protected] BT Research and Innovation contacts Name Business Area / Role Phone Mike Nilsson Research and Innovation [email protected] Steve Appleby Research and Innovation [email protected] Page 4

6 Executive summary BT Media and Broadcast provides services to broadcast and media organisations worldwide, including the international carriage of Ultra High Definition Television signals. We commissioned our colleagues in BT Research and Innovation to examine the current international status of technology and standardisation of Ultra High Definition Television to aid in our own understanding and future product development. This paper is the result, which we were so impressed with, we decided to release as a resource to the professional broadcast community. We hope it is useful do get in touch if you IN have STRICTEST feedback or CONFIDENC comments. UHD televisions are now retailing in significant numbers, and services, such as those offered by Netflix and Amazon, are starting to appear in the market. But while these services offer higher resolution than HD services, further improvement could be made in due course to provide an even better viewing experience. Future television systems should be capable of producing an experience that is IN either STRICTEST closer to real CONFIDENCE life or is capable of more accurately recreating the artistic intent of the storyteller. To this end, increased resolution, wider colour palette, higher frame rate and an improvement in the dynamic range of the images when used together, have the potential to provide viewers with a better visual experience compared to current television applications and provide a viewer IN STRICTEST with a stronger CONFIDENCE sense of being there. Many standardisation organisations around the world have been, and are continuing to be, very active in the area of UHD TV standardisation. DVB has created standards for what it terms UHD-1 phase 1, effectively the parameters of HDTV but with enhanced resolution, similar to the deployments of Netflix and Amazon. DVB is now working on the commercial requirements for two subsequent phases, the first of which is expected to add support for higher dynamic range, wider colour gamut, and higher frame rates, and the second of which is planned to add support for even higher, 8K, resolution. The HEVC compression standard, almost essential to make delivery of UHDTV commercially feasible, has been approved, with the recently approved second version including support for higher bit-depths and enhanced chroma formats. MPEG is currently studying whether HEVC is optimal, as currently standardised, to support higher dynamic range and wider colour gamut, and if not, is expected to launch a standardisation activity during 2015 to address the deficiencies. There is widespread interest in backwards compatibility of future UHDTV services, which may have higher dynamic range, wider colour gamut, and higher frame rates, with first generation UHDTV services with only enhanced resolution. Technology to achieve this is still under development, and the ultimate success or failure of backwards compatibility will depend on how efficiently it could be implemented compared to simple but inefficient simulcasting. There are, at the writing, many problems and few agreed solutions, but there are many lively discussions taking place in the standards community could be the year in which significant advances are made to the technical standardisation of the second phase of UHDTV. This report begins with sections describing the science and technology of UHD television: enhanced resolution, higher dynamic range, wider colour gamut and higher frame rate. This is followed by a section describing the current state of standardisation in various bodies that are essential to the standardisation of UHD television services, ITU-R, SMPTE, MPEG and DVB, while also providing a brief status update on EBU, DTG, Blu-ray Disk Association, HDMI Forum, Digital Cinema Initiatives (DCI), the Forum for Advanced Media in Europe (FAME), the UHD Alliance and the UHD Forum. We consider that while the enhanced resolution of the first phase of deployments of UHDTV services could provide undoubtedly better picture quality than current HDTV services when viewed from an optimal viewing distance, the combination of human visual acuity, screen size, and home viewing distances could make the improvement over HDTV in the home environment less pronounced. We have observed the growing feeling within the standardisation community that to make UHDTV reach the mass market, additional features would be needed beyond the enhanced resolution achieved with the first phase of deployment. In particular, enhancements that that do not depend so critically on viewing distance would be beneficial. Fortunately these features, higher dynamic range, wider colour gamut, and higher frame rates, are becoming technologically IN STRICTEST feasible CONFIDENCE and are generating much interest within the industry. This has perhaps been most noticeable in the film industry. Dolby Cinema, which was announced in December 2014, and which is claimed to deliver high dynamic range with enhanced colour, is planned for theatrical exhibition in The Blu-ray Disc Association is developing a next generation format that will support UHD, a wide colour gamut, and high dynamic range, with the first players and titles being expected before the end of The broadcast TV industry is close behind, with DVB aiming to develop specifications to enable deployment of second phase UHD services in the timeframe. Page 5

7 1 Introduction Ultra High Definition TV has attracted a good deal of attention lately as a result of a push from TV vendors and some content providers, notably Netflix. However, to date, the focus has been solely on increased resolution. Ultra High Definition though, is about more than just increasing the number of pixels, being also about increasing the dynamic range, widening the colour gamut and increasing the frame rate. When viewing TV on a popular size of screen in a typical home environment, it is these additional improvements which may ultimately provide the most benefit to the viewer. The quality of television is therefore set to improve in multiple dimensions over the next few years. There is still heated debate about the standards that will be used, but it seems clear that managing legacy and dealing with multiple dimensions of TV improvements will be a challenge for the broadcast industry. The purpose of this document is to provide a thorough review of the current status IN STRICTEST of UHD standards CONFIDENCE with a view to obtaining a clearer understanding of the challenges and opportunities ahead. Page 6

8 2 Enhanced Resolution Display and capture technology is advancing. Today there are both video cameras and displays capable of resolutions higher than full High Definition TV. These resolutions are frequently referred to as either 4K or Ultra High Definition TV (UHDTV) and have resolution 3840x2160, that is, four times the resolution of full High Definition TV. Figure 1 shows the resolution of UHD relative to the earlier formats of Standard Definition (SD), HD-ready (720p HD), and High Definition (HD). Figure 1. The resolution of UHD relative to earlier, lower, resolutions. UHD televisions are now retailing in significant numbers 1, enabling potentially much better picture quality than current HD televisions. BT worked with the BBC during 2014 to deliver coverage of the 2014 football World Cup Final in UHD resolution live to the BT Tower, using HEVC video compression and MPEG DASH technology, delivering content from BT Wholesale Content Connect over BT Infinity superfast fibre optic broadband through BT Home Hub 5 routers to three set top boxes connected to UHDTVs and directly to two internet-enabled UHDTVs. The picture quality achieved was very impressive. The quality of the UHD signal was undoubtedly much better than HD delivered by Freeview, when viewed near to the screen. The ITU-R describe optimal viewing distance in image heights (H) for various digital image systems 2, recommending 6H for Standard Definition, 4.8H for 720p HD, 3.2H for HD, and 1.6H for UHD. Such optimal viewing distances are often calculated by considering normal human visual acuity to be 20/20, meaning that a letter, such as the letter E, can be identified when top to bottom it subtends an angle IN of STRICTEST 5 arc minutes. CONFIDENCE This corresponds to 1 arc minute per horizontal feature (three lines and two spaces). It also corresponds to one dark to light cycle every 2 arc minutes. One dark to light cycle every 2 arc minutes equals 30 cycles per degree, which can be represented with 60 pixels per degree. This has been taken as standard acuity for TV applications, although clearly some viewers have worse eye sight and some have better eye sight. The screen diagonal size, S, measured in inches, required to achieve a resolvable number of horizontal pixels, R, when viewed from a distance D, measured in centimetres, is given by the following equation, assuming visual acuity of 60 pixels per degree and 16:9 picture aspect ratio. 1 ( ) 2 "Guidelines on metrics to be used when tailoring television programmes to broadcasting applications at various image quality levels, display sizes and aspect ratios", Recommendation ITU-R BT (03/2010), IN STRICTEST CONFIDENCE I Page 7

9 However, there is reason to believe that viewers will not sit as close as 1.6H to IN their STRICTEST television, and CONFIDENCE hence would not achieve the ITU-R optimal viewing experience. A survey 3 of 102 BBC employees in 2004 reported that the median viewing distance for viewing the main TV in the home was 2.7m. The BBC carried out a more extensive survey of the UK population during the summer of , collecting information from 2185 people who have a television in their home, and finding the median viewing distance was 2.63m. While the 2004 survey found a median screen height of 32.5cm, the median height from the 2014 survey was 49cm, which, assuming a 16:9 picture aspect ratio, corresponds to a screen diagonal size of 39.3 inches. The 2014 survey also found that the median relative viewing distance is 5.5 times the screen height (5.5H). The 2014 survey, by asking respondents how they would expect the size of their next television to compare to that of their current one, found that about half the respondents expected to buy a larger screen next time. When asked to estimate the ideal television size for their current home, assuming that money were no object, they responded with a median ideal diagonal size of 48 inches. By assuming that people would need to sit three picture heights (3H) or closer to a UHD screen to get a resolution improvement over HD, the BBC deduced from the 2014 survey that 10.2% of viewers would be able to observe a benefit from UHD with their current television, and 22.9% would if they acquired their ideal television. These two surveys show no significant change in television viewing distances over a period of 10 years. As this is likely to be significantly influenced by the size of rooms in UK homes, it would seem reasonable to expect little change in the coming years. Currently 55 inch and 65 inch diagonals are popular for UHD televisions. There would appear little reason to believe that this would change in the near future, as it is in part determined by the size and layout of rooms in which people watch television. A viewing distance of 1.6H, which the ITU-R consider to be optimal for viewing UHD content, when calculated for a television with 65 inch diagonal, corresponds to a viewing distance of about 1.3m. It seems unlikely given the results of the two BBC surveys that many people would sit this close to their UHDTV. Table 1 shows the resolution of various popular TV formats and the screen size that would be needed for optimal viewing at a distance of 2.63m, the median viewing distance found in the 2014 BBC survey. It is unlikely that viewers would want to, and be able to, acquire screens with diagonal size of 148 inches, so that they could both maintain STRICTEST a viewing CONFIDENCE distance of 2.63m and achieve optimal, as defined by the ITU-R, viewing of UHD content. While some viewers may compromise on viewing distance and screen size to get such a UHD experience, it seems likely that the majority would not, and hence they may not get the full benefit of UHDTV resolution. Format Horizontal Resolution Screen Size (Inches) SD p HD HD 1920 IN STRICTEST 68 CONFIDENCE UHD Table 1. The screen size needed for optimal viewing of popular TV formats from 2.63m. The screen would not need to be so large if the requirement were relaxed from achieving optimal viewing of UHD content to achieving a better experience than watching HD content. Again referring to Table 1, any screen with diagonal size larger than 68 inches would enable more resolution than HD to be observed from 2.63m. However, there is some disagreement with this whole methodology. Spencer 5 has reported results of subjective tests in which viewers assessed content from a distance of 30cm, controlled using a head mount. By using the methodology described above, the 3 Results of a survey on television viewing distance, N. E. Tanton, June A Survey of UK Television Viewing Conditions, Katy C. Noland and Louise H. Truong, January 2015 Page 8

10 viewers should have been limited to seeing benefits as the pixel density increased IN STRICTEST to about 300 pixels CONFIDENCE per inch, and then seeing no further improvement as the pixel density was increased. However, the presented results show improvements at 500 pixels per inch, and yet further improvements at 1000 pixels per inch. Spencer explains these results by stating that the human eye is able to detect differences represented by even finer structures, referencing tests of vernier acuity, which involve distinguishing the relative alignment of parallel lines, that show that humans can distinguish details five to ten times smaller than standard resolution measurements predict. While this is an isolated study, it does suggest that further investigation of the benefits of the increased resolution of UHDTV, when viewed on televisions of desirable size at normal home viewing distances, would be worthwhile. 5 How much higher can mobile display resolution go?, Lee Spencer, Page 9

11 3 High Dynamic Range The human visual system has a significantly larger dynamic range than that supported by current television systems. Display devices that are able to support a much larger dynamic range than conventional televisions systems are starting to appear in the market. They achieve the larger dynamic range by extending the range both towards darker as well as lighter values as a result of fitting a conventional LCD panel with a spatially varying backlight, which could be either a projector or a panel of LEDs which are individually addressable. A key technical challenge for video delivery is how to signal the higher dynamic range while also supporting legacy standard dynamic range displays. Some proprietary technology is already entering the market, and various standardisation bodies, including SMPTE, the EBU and MPEG are considering the issues. This section provides an overview of the benefits of using a higher dynamic range IN STRICTEST for images and CONFIDENCE video, the capability of the human visual system, and the capability of emerging display technology. Additionally, the issues and proposed solutions for integrating higher dynamic range into existing end to end television systems are discussed. 3.1 The Benefit of High Dynamic Range Figure 2 is an example of an image where the use of a higher dynamic range would IN STRICTEST improve the CONFIDENCE viewing experience. With a standard dynamic range, it is not possible to have visible details in both the shaded parts of the image and the parts that are in full sunshine. Figure 2. An image that would benefit from high IN STRICTEST dynamic range. CONFIDEN The use of a high dynamic range is not simply making images brighter, which in itself can be effective, but enabling detail to be seen in both dark and light areas. Figure 3 shows an example of a histogram of the pixel luminance values of an image, represented with both standard and high dynamic range. Many of the pixels have about the same luminance in both cases, and only some pixels, the lowlights and the highlights in the image, have lower or greater luminance in the high dynamic range variant. Luminance is measured in the derived SI units of candela per square metre (cd/m 2 ), also known as the nit (1 nit = 1 cd/m 2 ). Typically displays are limited to a dynamic range of 100:1, and a typical maximum luminance of 100 cd/m 2 due to legacy limitations of CRT display technology and the derived current specification for display interfaces. The human visual system, as described later, is capable of perceiving a much higher dynamic range and higher levels of luminance. Figure 4 illustrates the wide range of luminance values that occur in real world scenes, labelling specific features with approximate values of light level. Page 10

12 Relative Frequency Luminance (cd/m2) Standard Dynamic Range Image High Dynamic IN STRICTEST Range Image CONFIDENCE Figure 3. An example pixel luminance histogram for a standard and high dynamic range image. 4,000 nits 10,000 nits 50,000 nits 100 nits 40 nits 2 nits Figure 4. Examples of approximate light levels in real world scenes. 3.2 The dynamic range of the human visual system The human visual system has a huge dynamic range, being able to cope with starlight at 10-4 cd/m 2 and bright sunshine at 10 5 cd/m 2. Hood and Finkelstein 6 report values ranging from 10-6 to 10 cd/m 2 for scotopic IN STRICTEST light levels CONFIDEN where light transduction is mediated by the rod cells in the eye, and from 0.01 to 10 8 cd/m 2 for the photopic range where the cone cells are active. And in the overlap, called the mesopic range, both rods and cones are involved. At any one time, the human visual system is able to operate over only a fraction of this enormous range. This subset is called the simultaneous or steady-state dynamic range. It shifts to an appropriate light sensitivity due to various mechanical, photochemical and neuronal adaptive processes 7, so that under any lighting conditions the effectiveness of human vision is maximised. 6 From D. C. Hood and M. A. Finkelstein, Visual sensitivity, in Handbook of Perception and Human Performance, K. Boff, L. Kaufman, and J. Thomas, Eds., vol. 1. Wiley, New York, J. A. Ferwerda, Elements of early vision for computer graphics, IEEE Computer IN Graphics STRICTEST and Applications CONFIDENCE 21, 5, 22-33, luthuli.cs.uiuc.edu/~daf/courses/rendering/papers3/ pdf. Page 11

13 The simultaneous dynamic range over which the human visual system is able to IN function STRICTEST can be CONFIDENCE defined as the ratio between the highest and lowest luminance values at which objects can be detected, while being in a state of full adaptation. In a given room that contains a display device, the human visual system will typically be in a steady state of full adaptation. Figure 5, taken from Hood and Finkelstein, shows the overall range of the human visual system compared to the ranges of its approximate steady state as well as the range for typical Low Dynamic Range (LDR) and High Dynamic Range (HDR) displays. While the overall dynamic range of the human visual system and these display devices is known, less is known about the range of the human visual system in the steady-state. Kunkel and Reinhard 8 have reported a sequence of psychophysical experiments, carried out with the aid of a high dynamic range display device, where they determined the simultaneous dynamic range of the human visual system, finding the human visual system to be capable of distinguishing contrasts over a range of 3.7 log units, equivalent to a range of about 1:5000, under specific viewing conditions. They also found that the dynamic range is affected by stimulus duration, the contrast of the stimulus and the background illumination, which they claimed accounts for the different dynamic ranges being reported in the literature. Figure 5. The dynamic range of the human visual system. Jenny Read, neuroscientist at the University of Newcastle, talked at DVB-EBU HDR Workshop in June on how the Human Visual System reacts to high dynamic range screens, stating that encoding schemes must be defined by taking into account human contrast sensitivity. It is also necessary to take into account the human visual system adaptation states, where light to dark adaptation is slow, typically taking between 8 and 30 minutes, and where dark to light adaptation is much quicker, typically taking just a few minutes. Daly et al 10 report studies to find the dynamic range that is preferred by human observers. They used a Dual Modulation Research Display, as shown in Figure 6, and referenced by Hammer 11, which is capable of supporting a dynamic range from cd/m 2 to 20,000 cd/m 2, that is, a contrast ratio of 5,000,000:1; and which supported the DCI P3 colour gamut. The experiment used several realistic and synthetic stimuli in a dark viewing environment. They found for diffuse reflective images a dynamic range between 0.1 and 650 cd/m 2 matched the average preferences, but to satisfy 90% of the population, a dynamic range from to about 3000 cd/m 2 is needed. They claim that since a display should be able to produce values brighter than the diffuse white maximum, as in specular highlights and emissive sources, they conclude that the average preferred maximum luminance for highlight reproduction satisfying 50% of viewers is about 2500 cd/m 2, increasing to marginally over cd/m 2 to satisfy 90%. They conclude that there would be a benefit from more capable displays, as the preferred luminances found in their study exceed even the best of consumer displays today. 8 T. Kunkel and E. Reinhard, A reassessment of the simultaneous dynamic range of the human visual system, Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization, ISBN , pp , July DVB-EBU HDR Workshop, IRT, Munich, Germany, 17 June 2014, 10 S. Daly, T. Kunkel, S. Farrell & Xing Sun: Viewer Preferences for Shadow, Diffuse, Specular, and Emissive Luminance Limits of High Dynamic Range Displays. SID Display Week Hammer, High-Dynamic-Range Displays, Page 12

14 Figure 6. The experimental set up used by Daly et al. to assess preferred dynamic range. Hanhart, Korshunov, and Ebrahimi 12 report a set of subjective experiments to investigate the added value of higher dynamic range. Seven test video sequences at four different peak luminance levels were assessed using the full paired comparison methodology. Pairs of the same sequence with different peak luminance levels were displayed side-by-side on a Dolby Research HDR RGB backlight dual modulation display (aka Pulsar ), which is capable of reliably displaying STRICTEST video content CONFIDENCE at 4000 cd/m 2 peak luminance. Their results, as shown in Figure 7, show that the preference of an average viewer increases logarithmically with the increase in the maximum luminance level at which the content is displayed, with 4000 cd/m 2 being the most attractive option of those tested. Figure 7. Results of subjective evaluation of HDR Video using pair comparison by Hanhart, Korshunov, and Ebrahimi. 3.3 The non-linearity of the human visual system Ernst Heinrich Weber studied of the human response to a physical stimulus in a quantitative fashion, finding, in what is now known as Weber's law, that the just-noticeable difference between two stimuli is proportional to the magnitude of the stimuli, that is, an increment is judged relative to the previous amount. Gustav Theodor Fechner used Weber's findings to construct a psychophysical scale in which he described the relationship between the physical magnitude of a stimulus and its (subjectively) perceived intensity. 12 Subjective Evaluation of Higher Dynamic Range Video, Philippe Hanhart, Pavel Korshunov, and Touradj Ebrahimi, SPIE Optical Engineering + Applications, Applications of Digital Image Processing XXXVII. Page 13

15 Fechner's law states that subjective sensation is proportional to the logarithm of IN the STRICTEST stimulus intensity. CONFIDENCE Fechner scaling has been found to apply to the human perception of brightness, at moderate and high brightness, with perceived brightness being proportional to the logarithm of the actual intensity 13. At lower levels of brightness, a more accurate description is given by the de Vries-Rose law which states that the perception of brightness is proportional to the square root of the actual intensity. 3.4 The mapping of linear light to code levels Current television systems typically support content with a range of brightness from about 0.1 cd/m 2 to about 100 cd/m 2. Consequently, it is the de Vries-Rose characteristic, described above, that has been designed into television systems to date, where a function known as gamma or gamma correction, is used to map brightness (linear light) to a scale in which each increment corresponds to a constant perceptual change. This function is known as the Opto-Electrical IN STRICTEST Transfer CONFIDENCE Function (OETF). In analogue television systems this characteristic made the whole range of intensities equally sensitive to noise, whereas in digital television systems it enabled the number of bits needed to represent each sample to be minimised 14. This characteristic is shown in Figure 8, where mappings, as specified in ITU-R Recommendation BT.709, to both 8 and 10 bit code levels are shown. The use of 10 bit codes, as increasingly used in video production, gives little benefit over 8 bits if BT.709 is still used, as the extra bits are the least significant bits, which are often discarded for display on 8 bit devices, and which at best could reduce the minimum black level, that is, make the blacks even blacker. While using BT.709, the extra two bits cannot increase the brightness, even though this would often be more desirable than increasing the maximum darkness. ITU-R Recommendation BT.2020, a more recent specification than BT.709, defines higher frame rates and a wider colour space, but the same OETF as BT.709. These and other ITU-R Recommendations are discussed in further later in this research paper. The dynamic range of a video signal can be increased by using an alternative Opto-Electrical Transfer Function (OETF) to that specified in BT.709. Digital Imaging and Communications in Medicine (DICOM) is a standard for handling, storing, printing, and transmitting information in medical imaging. This includes in Part an Opto-Electrical Transfer Function STRICTEST from linear CONFIDENCE light to 10 bit code levels, also shown in Figure 8, and valid between 0.05 and 4000 cd/m2 whereby each 1-bit luminance increment is equally visible according to Barten 16. Jenny Read stated at the DVB-EBU HDR Workshop in September 2014 that the film industry has used log-curves for many years that are quite near the human visual system and are much more efficient than the BT.709 Transfer Function used for TV today. A logarithmic OETF is used to map 14 stops of linear light, that is, a dynamic range of about 16,000:1, to a 10 bit signal, and is satisfactory to capture the Cineon format. Miller et al 17 of Dolby Laboratories have proposed a new OETF to provide higher dynamic range for video and movie production and distribution, which has recently been standardised by the Society of Motion Picture Engineers as SMPTE ST 2084:2014, High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays, which is described later in this research paper. This is also based on the work of Barten, and is also shown in Figure 8. But there is no industry consensus on the relevance of the work of Barten. The BBC in its white paper argues that Miller s proposal links the camera OETF to the absolute brightness of the display and that this has potentially far reaching consequences. 13 Weber Fechner law, 14 C. A. Poynton, Digital Video and HDTV: Algorithms and Interfaces, Electronics & Electrical, Morgan Kaufmann series in computer graphics and geometric modelling, ISBN , Barten, P.G.J., Physical model for the Contrast Sensitivity of the human eye. Proc. SPIE 1666, (1992); Barten, P.G.J., Spatiotemporal model for the Contrast Sensitivity of the human eye and its temporal aspects. Proc. SPIE (1993); and P. G. J. Barten, Formula for the contrast sensitivity of the human eye, Proc. SPIE-IS&T Vol. 5294: , Jan Scott Miller, Mahdi Nezamabadi and Scott Daly. Perceptual Signal Coding for More Efficient Usage of Bit Codes. SMPTE Motion Imaging Journal : doi: /j BBC White Paper WHP 283, Non-linear Opto-Electrical Transfer Functions for IN High STRICTEST Dynamic Range CONFIDENCE Television, T. Borer, July Page 14

16 Linear Luminance (cd/m2) The BBC argues that previously the photographic, movie and television industries IN STRICTEST have all always CONFIDENCE worked with relative, rather than absolute, luminance levels. The BBC argues that changing to absolute luminance levels will require significant changes to the way television is produced and viewed; and that it is not clear that the large dynamic range of 10 7 is needed for image display, when the simultaneous dynamic range of the eye is about They have developed their own proposal for an OETF, as shown in Figure 8, and which is imagined to be consistent with their submissions to ITU-R WP6C. There is on-going discussion of high dynamic range in standardisation bodies. The formal standardisation process for high dynamic range signal and exchange format, including the consideration of suitable Opto-Electrical Transfer Functions, is on-going in ITU-R WP6C (RG-24) and in SMPTE 10-E as described later in this report. MPEG is currently investigating the effectiveness of existing compression technologies with high dynamic range video, and the means to support standard and high dynamic range video within a single coded representation BT.709 with 8 bit code levels BT.709 with 10 bit code levels DICOM Part 14 Perceptual Quantiser, SMPTE ST 2084:2014 BBC HDR Proposal - 10 bit code levels Code Levels 3.5 The mapping of pixel code levels back to linear light Figure 8. The mapping of linear light to code levels. While Cathode Ray Tubes (CRT) were the only or most popular display device for IN television, STRICTEST there CONFIDENCE was no need to specify an inverse to the OETF, known as the Electro-Optical Transfer Function (EOTF), as the inverse function was provided implicitly from the physics of cathode ray tubes. Fortunately, this function was well matched to the human visual system. It was only in 2011, by which time the consumer market for CRTs had almost completely disappeared, that a standard for an EOTF, ITU-R Recommendation BT , was finally agreed. This effectively documents the characteristics of CRT displays. The EOTF is not necessarily best defined as the inverse of the OETF: the combination of the OETF and the EOTF yields the total transfer function, sometimes known as the end-to-end gamma or system gamma. Some believe that this total transfer function should be a power law function with the exponent value dependent on the lighting condition, for example, with values of 1.0, 1.25, and 1.5 being appropriate for bright, dim, and dark surrounding environments 19 IN. The fundamental basis for interpreting any visual signal is the transfer function, the description of how to convert the signal, that is, the digital code values, to optical energy. It is therefore this EOTF and not the OETF IN STRICTEST that truly defines CONFIDENC the intent of visual signal code values. The vast majority of content is colour graded (either live in the camera, or during post production) according to artistic preference while viewing on a reference standard display. 19 Report ITU-R BT , (03/2014), The present state of ultra-high definition IN television. STRICTEST CONFIDENCE Page 15

17 The EOTF, when applied to code values with a given bit depth, 8, 10 or 12 bits etc., IN STRICTEST should ideally CONFIDENCE avoid the viewing of discontinuities in tone reproduction, and hence the EOTF and bit depth should be matched to the contrast sensitivity function of the human visual system. A well respected model for the contrast sensitivity function of the human visual system was developed by Peter Barten 16, and has been referenced by many electronic imaging studies and standards. This complex model, based on physics, optics, and some experimentally determined parameters, has been shown to align well with many visual experiments spanning several decades of research. Dolby Laboratories have used the Barten model directly to compute an optimized perceptual EOTF 17. This function is defined in Figure 9. This is defined in terms of absolute luminance levels viewed on the display screen, not in terms of absolute luminance at capture. This Perceptual Quantiser curve has nearly a square root behaviour (slope = -1/2) at the darkest light levels, consistent with the Rose-DeVries law, and then rolls off to a constant zero slope for the highest light levels, consistent with the log behaviour of Weber s law. Between those extreme luminance regions, it exhibits varying slopes, IN STRICTEST and throughout CONFIDENCE the mid luminance levels it exhibits a slope similar to the gamma non-linearities of BT ( ) Figure 9. The EOTF derived from the perceptual quantiser function, as proposed Dolby. Dolby Laboratories reported 17 the results of subjective tests with real images with peak luminance up to 600cd/m 2, comparing BT.1886 scaled for a peak luminance of 1000cd/m 2 with the perceptual quantiser based EOTF scaled to 1000cd/m 2, and scaled to 10,000cd/m 2. They reported finding that with both variants of the perceptual quantiser, 10 bits of bit depth were sufficient to avoid visible quantisation steps on all eight test images, whereas the use of BT.1886 required less bits for light (white) images, typically one bit less of bit depth, but needed more bits for the dark (black) images, including needing more than 12 bits of bit depth for one image. These results are consistent with the known limitations of BT.1886 that it has greater precision for lighter regions than darker regions, and effectively wastes code values at the light end, and does not have enough at the dark end. It is generally thought that it would be difficult to operate legacy infrastructures with bit depth greater than 12 bits, and most live production and broadcast environments still operate at the 10 bit level. Hence it is commonly agreed that applying BT.1886 to a higher dynamic range is not feasible as the bit depth required would be too high. However, there is no consensus that the perceptual quantiser proposed by Dolby Laboratories is the best solution: there are concerns about its being defined in terms of absolute light levels, and other functions have been proposed, as are discussed elsewhere in this report. 3.6 Black level: how dark should displays be? How dark or black a region of a display can appear depends on two factors: the minimum emission from the display and the amount of ambient light that is reflected. The effective display black level, L black, can be calculated, as in the equation below, as the sum of the display minimum light emission, L min, known as dark current in the days of CRT, IN STRICTEST and meaning CONFIDEN the lowest level of luminance that comes out of the display; and a product of the display screen reflectivity, R display, and the ambient light level, E ambient. The impact therefore of higher levels of ambient light is to raise the minimum black level, and consequently to reduce the dynamic range of the image, as the maximum intensity is mostly unchanged with ambient light. ( ) Mantiuk et al. 20 have reported an experiment to determine the highest luminance level that cannot be discriminated from absolute black as the surrounding luminance is varied. They showed viewers a patch in the centre of a screen with absolute black on one side of the patch and non-zero luminance on the other, and asked viewers to choose the side that was brighter, or choose randomly 20 Mantiuk, R. and Daly, S. and Kerofsky, L. (2010), The luminance of pure black: exploring the effect of surround in the context of electronic displays. In: Proc. of Human Vision and Electronic Imaging XXI, IS&T/SPIE's IN STRICTEST Symposium CONFIDENCE on Electronic Imaging. Page 16

18 Black Level (Logarithm to the base 10 of values measured in cd/m2) if they looked the same. Different values of non-zero luminance and of surrounding IN STRICTEST luminance were CONFIDENCE tested. Two viewing distances were used, 1.4m and 4.7m, so that the size of the square patch corresponded to 6.1 and 1.8 visual degrees. They converted these results of just detectable differences from absolute black so they could be plotted as a function of ambient light rather than the luminance of surrounding pixels. The results are shown in Figure 10, with the experimental results labelled as HVS Small Patch and HVS Larger Patch. It can be seen that as the ambient illumination is increased, the lowest level of black that can be distinguished from absolute black increases. Also shown in Figure 10 is the black level of a black diffuse surface with reflectance of 3%, such as black velvet, and the performance of three displays: a CRT with minimum light emission of 1cd/m 2 and 3% reflectance; a conventional CCFL (cold cathode fluorescent lamp) backlight LCD with minimum light emission of 0.8cd/m 2 and 1% reflectance; and a modern LED-backlight LCD with spatially uniform back-light dimming with minimum light emission of cd/m 2 and 1% reflectance.1.3/2.4 The CRT appears grey compared to the diffuse black (velvet) for ambient light below 300lux (about 2.5 on the horizontal axis of Figure 10), a level of brightness found in an office or a very well lit room in a home 21. For the CCFL-LCD, this threshold is 100lux (2.0 in Figure 10), typical of a room in a home. This is because the display effective black level is higher than the luminance of a diffuse black surface, due to a combination of reflectance and the minimum light emissions of the displays. The experimental results indicate that the eye can appreciate even deeper black than a diffuse black surface, and that of the considered display technologies, only the LED-LCD display can satisfy the demands of the human visual system, and only at levels below about 1.6lux (0.2 in Figure 10, where the LED-LCD curve crosses the HVS IN Larger STRICTEST Patch curve), CONFIDENCE an indoor illumination level that could be considered near pitch black. The problem is not the minimum light emissions which are very low, but the reflectance of ambient light from the screen. It appears unlikely that very low reflectance coatings will be possible, but this is unlikely to matter, as a reflectance of about 1% is likely to acceptable to almost all viewers as there are not many objects in the real-world that would have lower reflectivity and thus appear darker than a display CRT CCFL-LCD LED-LCD Diffuse Black HVS Small Patch HVS Larger Patch Deep Twilight Dark Room Home Office Ambient Luminance (Logarithm to the base 10 of values measured in Lux) Figure 10. Comparison of the black levels of displays and human detection capability. 3.7 The mapping of pixel code levels to linear light in the presence of ambient light Although standards including BT.1886 and IEC (srgb 22 ) specify the mapping of pixel code values to light levels, the actual perceived light level on a screen also depends on the ambient light and the reflectivity of the screen Page 17

19 Pixel Luminance (Logarithm to the base 10 of values measured in cd/m2) Mantiuk et al. 23 have developed a display model that combines the standardised IN gamma STRICTEST mapping CONFIDENCE with a factor to include the reflections of ambient light. This display model, shown in the equation below, includes the maximum brightness and dynamic range of the display device, and the viewing conditions, the amount of the ambient light that is reflected from the screen. Such reflected light increases the luminance of the darkest pixels shown on the display, thus reducing the available dynamic range. ( ) ( ) ( ) ( ) They claim that most CRT and LCD displays can be modelled with this equation, where the displayed luminance, L d, is calculated as a function of the luma (pixel value), L, and the display gamma, γ, the peak display luminance, L max, the display black level, which is the luminance of a black pixel displayed in a perfectly dark room, L black, the display screen reflectivity, R display, and the ambient light level, E ambient. Figure 11 shows the mapping of pixel code levels to linear light in the presence of different levels of ambient light using this equation from Mantiuk et al. for a display with peak display luminance 80cd/m 2, display black level of 1cd/m 2, reflectivity of 1% and gamma of 2.2. It can be seen that the effective dynamic range of a display gets compressed due to screen reflections, making lower pixel values almost indistinguishable. The dynamic range of nearly 2600:1 of srgb is reduced IN to STRICTEST only 75:1 in a CONFIDENCE dimly lit room, and to only 6:1 in sun light srgb : Dynamic Range 2584:1 Dim Room (20 lux) : Dynamic Range 75:1-1.0 Light Room (150 lux) : Dynamic Range 54:1 Sun light (5000 lux) : Dynamic Range 6: Luma pixel value (0.005 to 1.000) Figure 11. The mapping of pixel code levels to linear light in the presence of ambient light. Mantiuk et al. addressed this issue with adaptive tone mapping. As ambient light increases, the image gets brighter to avoid dark tones, which are the most affected by the display reflections. For the outdoors illumination, many of the bright pixels are clipped to the maximum value. An example of the actual result of their tone mapping, and how it differs for different ambient light conditions, is shown in Figure R. Mantiuk, S. Daly and L. Kerofsky, Display Adaptive Tone Mapping, ACM Transactions IN STRICTEST on Graphics, CONFIDENCE Vol. 27, No. 3, Article 68, Publication date: August Page 18

20 Figure 12. An image tone-mapped for three different ambient IN STRICTEST illumination conditions. CONFIDENCE Kunkel and Daly 24 stated that current displays are very thin and highly reflective perpendicular to the display: they are similar to mirrors! The earlier LCD displays had a matte, diffuse effect, with relatively low reflectivity, but the matte caused some image blur. Glossy LCDs eliminate the blurring effect, but at the expense of more reflections, making it important to position the screen and the viewer to minimise light sources affecting the experience. While this is controllable to some extent in the home environment, it is not so for outdoor use and signage. They state that as screens get brighter and brighter, the reflection of the viewer, illuminated by the light from the display, becomes more of an issue, but screen manufacturers may be reluctant to address this issue in the short term because such a TV looks good when turned off: the current design popular for marketing. Sharp have developed technology to minimize screen reflection, which they refer STRICTEST to as moth CONFIDENCE eye 25. This technology, showcased in October 2012 at CEATEC, Japan's largest consumer electronics show, involves applying an anti-reflecting coating to LCD panels based on technology similar to the nanostructure of a moth s eyes: the surface of a moth's eyes is covered with bumps and valleys that absorb oncoming light, enhancing night vision. Unlike conventional anti-reflection technology, Sharp s claimed its new LCD offers more vivid colour images and higher contrast. It demonstrated 60, 70 and 80 inch moth eye panels at CEATEC based on its Aquos large-screen TVs. Sharp said its panel technology is ready for deployment in commercial products for indoor use. But Kunkel and Daly comment that although moth-eye displays are diffuse, with even lower reflectivity than the earlier matte screens, they currently suffer from not being scratch resistant. Thicker displays were demonstrated at CES Kunkel and Daly claim these allow better audio because larger speakers can be included, and that they also allow for better backlight modulation. They also comment on curved screens, stating that curvature may help with screen surface side effects, and may enhance immersion, and that curved displays may be good for mobile use as they may allow the user to more easily avoid reflections from external light sources. 3.8 The current state of high dynamic range capture and display technology Modern digital motion imaging sensors can originate linear video signals having dynamic ranges up to about 14 stops, that is, ranges up to about 16,000:1. This dynamic range is similar to the simultaneous dynamic range of the human visual system 18. Such cameras include the Arri Alexa and Amira, various models by Canon including the EOS C300 and EOS C500, the Red Epic, and various models by Sony including the A7S and the F65. Cinema5D have reported results of tests carried out on a selection of cameras 26. They used the DSC labs XYLA-21, a high quality LEDbacklit transmissive chart that displays 21 stops of dynamic range: each vertical IN bar STRICTEST in the chart represents CONFIDENCE one stop of light. The chart is filmed with each camera in turn in a completely dark room using the same very sharp Zeiss 50mm CP2 T/2.1 makro lens with interchangeable mount adjusted for the camera bayonet. Each camera was set to its native ISO and the F-stop of the lens was 24 Timo Kunkel and Scott Daly, Dolby Labs, Inc., SMPTE Monthly Webcast: Lessons in Light: From Reality via Display to the Eye Page 19

21 adjusted accordingly. The Intra frames were extracted from the recorded video IN and STRICTEST tested with CONFIDENCE software from IMATEST. The results are reproduced in Figure 13. It can be seen that the Arri and Sony cameras produced a usable dynamic range of about 14 stops, and the Canon cameras about stops. Figure 13 also shows that these cameras use a variety of transfer functions to map linear light to digital code values. The RED Epic with a Mysterium-X sensor is claimed to support 13.5 stops of dynamic range, and up to 18 stops when using RED HDRx 27. The Panasonic VariCam 35 is claimed to support 14+ stops of dynamic range 28 IN. STRICTEST CONFIDENC Dynamic range is considered more important to some camera manufacturers than increased spatial resolution. For example, Arri has stated 29 that it will not move to 4K until it can do so without compromising the dynamic range on its range of cameras, including the Alexa and the newly introduced Amira. They believe that they have the best dynamic range on the market today, and that there is interest from the creative community to create better pixels, not just more pixels. IN STRICTEST The Alexa is CONFIDENCE among the most widely-used digital cinematography cameras. Figure 13. Camera dynamic range measurements by Cinema5D. Miller et al 17 of Dolby Laboratories report that typical displays today are now achieving IN STRICTEST peak levels CONFIDENC of 500cd/m 2 or more, and that there are several commercial examples above 1000cd/m 2. As an example, the Samsung OL46B - OL Series 46" Outdoor High Brightness High Definition Display 30 is stated to have a peak brightness of 1500cd/m 2. However, typical televisions appear to have peak brightness levels up to about 350cd/m 2. For example the Toshiba 48" LED-backlit LCD HDTV, 48L1435DB 31, has a brightness of 300cd/m 2 ; and the Sharp 50" LED-backlit IN STRICTEST LCD HDTV, CONFIDENCE LC 50LE651K 32, has a brightness of 350cd/m 2. MPEG plans to use two high dynamic range displays in analysis of the submissions IN STRICTEST to its Call for CONFIDEN Evidence of new tools that may improve the performance of HEVC when used to encode high dynamic range and wide colour gamut video. Subjective testing will be done using a SIM2 monitor in two locations and a Pulsar monitor in another location GGH.html?refs= &q=brightness&src=16#usedstock 32 Page 20

22 The sim2 33 is a 47inch HD display with 2202 individually controllable white LED IN backlights, STRICTEST peak brightness CONFIDENCE of 4000cd/m2 and BT.709 colour gamut. It supports two input modes: HDR Yu v where 12 bits are assigned for Y in a log representation, and 10 bits are assigned for u and v using linear encoding, with 4:2:2 sampling; and DVI plus mode, in which the input comprises an LED signal (2202 values) and the LCD signal (1920x1078, 8 bit RGB values). This display is known to have some issues: when using the HDR Yu v input mode: chroma subsampling and different sample locations are issues. And when using DVI plus mode, direct knowledge of the display and the ability to accurately compute both the backlight and the RGB sample values is needed. The Dolby Pulsar has been used for demonstration and experimental purposes 12. It is a 42inch HD display with peak brightness of 4000cd/m 2, DCI-P3 colour gamut, and 12 bits per colour sample 34. It is likely that higher dynamic range televisions will not reach the market as a step change, but that over time, televisions with increasing peak brightness will become available. As stated above, televisions are already available that support peak brightness levels up to about 350cd/m 2. It is possible that this could steadily increase to 1000 cd/m 2 and beyond. But a change in broadcast or storage media format will be necessary for these screens to benefit from true higher dynamic range rather than just increased brightness. 3.9 Interest and Experience in Hollywood There s a growing number in Hollywood who want high dynamic range, which, as it is independent of the number of pixels in a frame, can be used with any picture resolution. There is a growing number of industry veterans that say, given a choice of high dynamic range or ultra high definition, it is high dynamic range that creates the more noticeable improvement, and hence it is this option that is generating a lot of interest 29. Emmanuel Chivo Lubezki, director of photography of the movie Gravity, is reported to have stated, I think every cinematographer will have an interest in high dynamic range, and that he would explore this potential for all his future projects 31. A short film, entitled Emma was shot in high dynamic range by director Howard Lukk and cinematographer Daryn Okada. The first four minutes of the 13 minute film was shown at the SMPTE 2014 Symposium, October An Alexa camera was used in Open Gate mode 36, capturing at a resolution of 3414x2198, and capturing into the ARRIRAW format using the ACES colour space. It was reported that one of the biggest problems on set was monitoring: they were shooting a high dynamic range movie and using a standard dynamic range monitor. Consequently they had to rely on a light meter to estimate light drop-off in certain shots. Shooting in high dynamic range is reported to have added no extra time, but spending more time with the make-up artist would be beneficial. With high dynamic range it is possible to see so much more, not only in the actors faces but also details in the background, which might not be desirable. With high dynamic range it was reported IN STRICTEST to be more CONFIDENC difficult to hide things and hence they spent more time on masks to get the right compositions. On the subject of whether it was possible to convert previously shot projects into high dynamic range, it was reported: If you shot on a very good dynamic range element, meaning film, you can probably pull out a high dynamic range out of it. If you re trying to go back and pull it out of a video camera from 10 years ago it will be impossible Better Pixels: Color Volume and Quantization Errors, Robin Atkins, Dolby Labs Alexa XT Open Gate. Record it all: Open Gate sensor mode. Page 21

23 It was reported that the following would help with future high dynamic range productions: Finding a high dynamic range display to use on screen for monitoring Locking down a transfer function for the high dynamic range master such as ACES or OCES Settling aspect ratio, since scope on Ultra-HD has to be downsized and letterboxed (otherwise pan and scan) Settling on a standard light level 3.10 Dolby Cinema Dolby Cinema was announced 37 in December It will feature the Dolby Vision projection system, which is claimed to be able to deliver high dynamic range with enhanced colour: its amazing contrast, high brightness, and colour range closely matches what the human eye can see. They claim that the blacks are truly black, colours are vibrant, and the contrast ratio far exceeds that of any other image technology on the market today. The first Dolby Vision projectors and titles for theatrical exhibition are expected in Early Dolby Cinema locations will be equipped with high-brightness laser projection technology available today with 4K, high-framerate 2D and 3D capabilities. When Dolby Vision content becomes available, the IN Dolby STRICTEST Cinema laser CONFIDENCE projection systems will be replaced with Dolby Vision projection systems Page 22

24 4 Wider Colour Gamut The human visual system is sensitive to electromagnetic radiation in the approximate range of wavelengths from 400nm (blueviolet) to 700nm (red), using three types of cone cell that have different spectral response. 4.1 The CIE RGB Colour Space In the 1920s, W. David Wright and John Guild independently conducted a series IN of STRICTEST experiments on CONFIDENC human colour vision. The experiments used a circular screen, of size two degrees to the observers, showing on one side a single wavelength test colour and on the other side a viewer adjustable mix of three primary colours, red, green and blue. The viewers were asked to adjust the mixture of primary colours until they considered a match with the single wavelength test colour to have been achieved. This was not possible for all test colours, where instead a variable amount of one IN of STRICTEST the primaries CONFIDENCE was added to the test colour, and a match with the remaining two primaries was carried out. The CIE RGB colour matching functions were consequently defined using these IN experimental STRICTEST results, CONFIDENCE using three monochromatic primaries at standardized wavelengths of 700nm (red), 546.1nm (green) and 435.8nm (blue). The primaries with wavelengths 546.1nm and 435.8nm were chosen because they are easily reproducible monochromatic lines of a mercury vapour discharge. The 700 nm wavelength, which in 1931 was difficult to reproduce as a monochromatic IN STRICTEST beam, was chosen CONFIDENCE because the eye's perception of colour is rather unchanging at this wavelength, and therefore small errors in wavelength of this primary would have little effect on the results. The colour matching functions are the amounts of primaries needed to match the monochromatic test colour. These colour matching functions, known as the "1931 CIE standard observer", are shown in Figure 14. The wavelengths at which one of the colour matching functions goes negative correspond to the test cases where a variable amount of one of the primaries was added to the test colour, and a match with the remaining two primaries was carried out: the test colour is effectively being matched by a positive contribution from two primaries and a negative contribution from the other R(λ) G(λ) B(λ) Wavelength (nm) Figure 14. The CIE 1931 RGB Colour Matching Functions. The RGB tristimulus values, (R, G, B) for a colour with spectral power distribution, IN STRICTEST I(λ), can be calculated CONFIDENC as the integral of the product of the colour matching function (R(λ ), G(λ ), B(λ)) and the spectral power distribution, I(λ ), as shown below. Note that many different spectral power distributions correspond to the same RGB tristimulus values, and also appear to the human visual system to be the same colour. ( ) ( ) ( ) ( ) ( ) ( ) Page 23

25 The chromaticity of a spectral power distribution, (r,g), can be calculated by normalising the RGB tristimulus values: This results in the CIE rg chromaticity space as shown in Figure 15. The curve shown represents the colours of single wavelength. Colours within this locus can only be made by mixing light of different wavelengths together, including those colours that lie along the line g=0. Points outside of the locus are not real colours and cannot be observed. 2.0 Green 1.5 g The CIE XYZ Colour Space Figure 15. The CIE rg Chromaticity Diagram. Red The CIE decided 38 to map the RGB colour space onto a new derived colour space, XYZ, with derived colour matching functions, and derived chromaticity space, with the benefits that the derived colour space would satisfy the following constraints: r The colour matching functions would be positive or zero at all wavelengths; and The Y colour matching function would be exactly equal to the photopic IN luminous STRICTEST efficiency CONFIDENCE function V(λ) defined by the CIE in 1926 for the "CIE standard photopic observer ; The RGB colour space is mapped onto the XYZ colour space using the following IN linear STRICTEST transformation, CONFIDEN which has been standardised by the CIE 39 : Blue [ ] ( ) [ ] 38 How the CIE 1931 Color-Matching Functions Were Derived from Wright Guild Data, Hugh S. Fairman, Michael H. Brill, Henry Hemmendinger, Matching%20Functions%20Were%20Derived%20from%20Wright-Guild%20Data.pdf 39 Publication CIE No. 15.2, Colorimetry, Second Edition, Central Bureau of the IN Commission STRICTEST Internationale CONFIDENCE de l Éclairage, Vienna, Austria, Page 24

26 As described above for the RGB colour space, and shown in the equation below, IN the STRICTEST chromaticity CONFIDENCE of a spectral power distribution, (x,y), can be calculated by normalising the XYZ tristimulus values. And when this is done, by the requirement of positive values of X, Y and Z, the gamut of all colours lies inside the triangle [1,0], [0,0], [0,1]. The CIE 1931 xy chromaticity diagram, taken from Wikipedia 40, is shown in Figure 16, together with the RGB primaries: 700nm (red), 546.1nm (green) and 435.8nm (blue). All colours within this triangle defined by IN the STRICTEST three primaries CONFIDENC can be made by mixing the three primaries in the appropriate non-zero combination. Colours outside of this triangle can be made by mixing the primaries, but only if negative quantities are allowed. This is why in the Guild and Wright experiments, some single wavelength test colours had to be colour matched by mixing the test colour with one primary, and matching against a mixture of the other two primaries. Both the XYZ and the xyy colour spaces can be used in practice. The latter consists IN STRICTEST of the pure chromaticity CONFIDENCE values x and y, and the luminous value Y. 4.3 Perceptually Uniform Colour Spaces Figure 16. The CIE 1931 xy chromaticity diagram showing IN STRICTEST the CIE RGB primaries. CONFIDENCE The CIE 1931 XYZ colour space is not perceptually uniform, that is, a constant magnitude change in the tristimulus values does not correspond to a constant perceptual change. The CIE has developed colour spaces that are closer to being perceptually uniform. These include the Lab and Luv colour spaces standardised by the CIE in Both of these colour spaces are derived from the master XYZ colour space, with dimension L for lightness and a and b or u and v for the colour dimensions. Both share the same definition of lightness, L, with L=0 indicating black, L=100 indicating diffuse white, and higher values indicating specular highlights, but they have different definitions for the colour dimensions. It is believed that IN Lab STRICTEST is the more widely CONFIDENCE used, and that both were standardised as the standardisation committee could not reach agreement on a single one Page 25

27 4.4 Colour Television Colour television has taken advantage of the above observations on the mixing of primary colours: by choosing three primary colours, and mixing them appropriately, all colours within the triangle on a chromaticity diagram defined by the three primary colours can be created. Cathode ray tube displays, the primary means of watching colour television until relatively recently, use three different phosphors which emit red, green, and blue light respectively, when excited by one of three beams of electrons. LCD displays achieve colour by applying three colour filters, red, green, and blue, to the backlight. In both cases of CRT and LCD displays, the colour primaries are not single wavelength sources, but have quite spread spectra, as shown in Figure 17, where the CRT image is from Wikipedia 41 and the right image is recreated from Chen at al Wavelength (nm) Figure 17. The spectra of display primary colours, typical CRT on the left, typical LCD with colour filters on the right. The spectra of the colour primaries of display devices limit the range (gamut) of STRICTEST colours that can CONFIDENCE be shown on the display. The ITU-R Recommendation BT.709, and the earlier BT.601, defined a set of colour primaries, in terms of the CIE 1931 x and y chromaticity diagram, to be used in television systems. These colour primaries, and the colour space defined by them, are shown in the CIE 1931 xy chromaticity diagram of Figure 18, taken from Wikipedia 43. This system is still in use today, although as described fully below, there is significant interest in extending display specifications and consequently producing devices to support wider colour spaces. It can be seen clearly in Figure 18 that this colour space leaves a large set of colours that could not be displayed. Care should be taken when quantifying this, because, as pointed out above, CIE 1931 is not a perceptually uniform colour space Quantum-Dot Displays: Giving LCDs a Competitive Edge through Color, Jian Chen, Veeral Hardev, and Jeff Yurek Page 26

28 Figure 18. The primary colours of the BT.709 colour space, and its primary colours on the CIE 1931 xy chromaticity diagram. 4.5 The need for a Wider Colour Gamut Current television systems do not support the wide range of colours that the human eye can perceive. Future television and other video distribution environments are hoped to give a viewing experience that is closer to a real life experience, to provide a user with a stronger sense of being there. This requires supporting both significantly higher dynamic ranges and wider colour gamuts than supported today. As stated above, Figure 18 shows the primary colours of the BT.709 colour space. STRICTEST These are joined CONFIDENCE to form a triangle, which is often referred to as the bounds of the colour space. But this is only true for low values of luminance. For higher values of luminance, the range of colours is reduced, such that for the maximum relative luminance of 1.0, the chromaticity gamut is only a single point, being the white point of the colour space on the chromaticity plane 44. Hence in order to describe the full range of colours that can be represented in a colour space, for example that described in BT.709, it is necessary to consider a 3D colour volume. Assuming this 3D colour volume to be drawn with the x and y chromaticity coordinates in the horizontal plane, and the luminance, Y, vertically, then the horizontal cross section of the 3D colour volume is the triangle shown on Figure 18 up to a relative luminance of , at which point IN the STRICTEST blue primary CONFIDENC cannot add to the relative luminance. In order to increase relative luminance of blue beyond this level, it is necessary to desaturate it by adding red or green or a mixture of the two. Hence from upwards, the cross section becomes a quadrilateral, with two vertices directly above the red and green primaries, and with two vertices inward from the blue primary, to an increasing extent as relative luminance is increased. At relative luminance of , the red primary similarly cannot add to the relative luminance, and hence to increase relative luminance of red beyond this level, it is necessary to add blue or green or a mixture of the two. The cross section consequently becomes pentagonal, but only until relative luminance reaches = , at which the blue plus red vertex meets the red plus blue vertex, and the luminance of either can only be increased by adding green, and the cross section returns to being a quadrilateral. At a relative luminance of , the green primary can no longer add to the relative luminance on its own, and the cross section again becomes pentagonal. As relative luminance increases further, the cross section becomes quadrilateral yet again, and finally triangular at relative luminance of , reaching an apex at relative luminance of , and the D65 white point given by chromaticity coordinates (x=0.3127, y=3290). This is shown on the CIE 1931 xy chromaticity diagram in Figure 19, where in addition to the showing the spectral locus, labelled as CIE 1931, the cross section of the BT.709 3D colour volume is shown for various IN values STRICTEST of luminance, CONFIDENCE showing that the cross sectional area reduces as the relative luminance increases Page 27

29 Y y CIE 1931 Y = Y = Y = IN STRICTEST Y = CONFIDENC Y = Y = Y = x Figure 19. The horizontal cross section of the BT.709 3D colour volume STRICTEST for various CONFIDENCE values of luminance. Figure 20 shows another view of the 3D colour volume, this time showing a vertical cross section, with chromaticity coordinate, x, plotted horizontally and the relative luminance, Y, plotted vertically, for chromaticity y = The colours shown on the figure are illustrative, and clearly not precise. But it is can be seen, particularly on the right hand side, that as the relative luminance is increased, the range of colours that can be represented is reduced Out of Gamut Colour x Figure 20. A vertical cross section of the BT.709 3D colour IN STRICTEST volume for y = CONFIDENCE Page 28

30 This illustrates the need for both a wider colour gamut and a higher dynamic range, IN STRICTEST as stated by CONFIDENCE Kunkel and Daly 24, who quote an example of a volcano at night, such as that illustrated in Figure Emissive colour, such as the molten lava of Figure 21, can be both very bright and very saturated, and outside of the 3D colour volume, as shown on Figure 20. Without a change to colour gamut or dynamic range, the orange of the volcano would have to be moved into the colour volume. This could be accommodated by making it darker, effectively moving it down in Figure 20 so that it moves into the colour volume or by making it less saturated, effectively moving it horizontally into the colour volume, or some combination of both. When using a higher dynamic range and a wider colour gamut, the colour volume increases in all three dimensions, allowing such a point to be represented: just having one of these aspects may not be enough. The shape of the colour volume is such that at high luminance, it is not possible to have high saturation: while a display may be able to display white with 1000cd/m 2, it may not be able to display green with 1000cd/m 2 because of the way the display is built. Combining a wider colour gamut with a higher dynamic range will create a larger colour volume which can then transport more colours. 4.6 Wider Colour Gamut Standards Figure 21. Emissive colour can be both bright and saturated. The colour space defined in BT.709 has been widely adopted, and is common to other standards, including BT.1361, IEC (srgb or sycc), IEC , and SMPTE RP 177 (1993) Annex B. The DCI P3 colour space is used by digital cinema. It is a part of the SMPTE Recommended Practice RP for Digital Cinema Quality Reference Projector and Environment, first released by the SMPTE in 2007, and most recently updated in It defines the Reference Projector and its controlled environment, along with the acceptable tolerances around critical image parameters for Review Room and Theatre applications. Although the colour gamut is specified in terms of XYZ, a Digital Cinema projector is required to support a gamut of at least DCI-P3. The DCI-P3 colour space uses the same blue primary as BT.709, but it uses different green and red primaries. The red primary of DCI- P3 is monochromatic 615 nm, a slightly deeper hue of red than BT.709. The green IN STRICTEST primary is a slightly CONFIDEN more yellowish hue of green but is more saturated. In CIE 1931 (x,y) coordinates, the DCI-P3 primaries are red (0.680, 0.320), green (0.265, 0.690) and blue (0.15, 0.060). Additional colour spaces with wider gamut have been defined with the aim of supporting wider colour gamuts in the future. The ITU-R has specified a colour space in BT.2020 using single wavelength primaries. In addition, specifications have been defined that cover the full visible gamut. For example, SMPTE specified in ST usage of the CIE 1931 XYZ colour space, and the Academy Color Encoding System (ACES) defined the ACES colour space, which is similar to IN XYZ, STRICTEST but slightly CONFIDENCE smaller while still encompassing the whole spectral locus. In CIE 1931 (x,y) coordinates, the ACES primaries are red ( , ), green ( , ) and blue ( , ). Figure 22 shows the BT.709, DCI-P3, BT.2020, ACES and XYZ colour gamuts on the CIE 1931 xy chromaticity diagram Page 29

31 4.7 Conversion between Colour Gamuts As colour gamuts are specified relative to CIE standards, conversion between colour gamuts is possible. When converting from one colour gamut to another, where the target gamut fits fully within the originating gamut, is straightforward, being essentially a mathematical transformation. For example, as described later in this report, the ITU-R is understood to be developing a new Recommendation, to specify a method of colorimetry conversion from BT.709 to BT Such a conversion could be needed for example when HDTV programme content is included within UHDTV programmes. However, other conversions, for example from BT.2020 to BT.709, are more problematic IN STRICTEST as there CONFIDENC may be colours in the originating colour space that cannot be represented in the target colour space. These out of gamut colours cannot be represented accurately, and so some loss in quality must be tolerated. The problem is that while out of gamut colours need to be changed to bring them into the gamut, colours that are not outside the gamut do not need to be changed, and the larger any change to them is, the more degradation occurs, but if they are not changed, they will no longer have an appropriate relationship to colours that are moved into the gamut. A high quality gamut mapping algorithm should minimise the change of colours in the original representation that are within the gamut of the target colour space, minimise the perceived changes of hue, minimise IN STRICTEST changes to brightness, CONFIDENCE contrast and saturation, avoid loss of spatial detail, and avoid creating visible discontinuities. Frohlich et al 46 report that colour gamut conversion is a problem today in the cinema industry where newer digital cinema projectors are capable of a wider colour space than older projectors, and where content is now being produced for these newer projectors, but where the issue of supporting older projectors with a narrower colour space remains. Current practice consists of simply clipping each colour component (RGB) to fit within the target colour space, but this is reported to result in hue shifts and loss of detail. Frohlich et al take the common view that a perceptually uniform colour space is best suited for performing such a conversion: a perceptually uniform colour space is one in which equal (Euclidean) distances in the colour space correspond to equal perceived colour differences. But rather than using the popular CIE L*a*b* uniform colour space, they prefer one defined by Ebner and Fairchild 47 known as IPT. They report the results of subjective tests comparing algorithms that they refer to as PCLIP and WmindE. PCLIP, short for Post-CLIP is assumed to be implemented in projectors today, and consists of simple clipping of each colour component (RGB) in the projector s native colour space. This is reported to result in hue shifts and loss of detail. The tested alternative, WmindE, which is short for weighted minimum delta E, selects the in-gamut colour with the minimum colorimetric distance for each out-of-gamut colour, but where a different weight is used for each of the components, lightness, chroma and hue. They report that in their subjective tests, WmindE in the IPT colour space was found to offer better detail preservation than PCLIP while retaining hue; and that WmindE was preferred over PCLIP by 80% of the viewers. They claim that all of the algorithms that they tested except PCLIP are too complex to render in real-time in current projectors, but that they could be implemented in IN a STRICTEST 3D lookup table. CONFIDENC 4.8 How large a Colour Gamut is needed? While the chromaticity diagrams referred to above indicate the full gamut of colours that the human visual system can perceive, it is difficult if not impossible to produce displays that show the full gamut. Particularly, if displays are limited to three primary colours, the largest gamut that could be created is one defined by the largest triangle that could be placed within the chromaticity diagram. But there is still a question of how large the gamut needs to be, to provide a realistic IN STRICTEST viewing experience. CONFIDEN The work of Pointer 48 addresses this question. A maximum gamut for real surface colours, known as Pointer s Gamut, was derived from the analysis of the colour coordinates of 4089 samples of real surfaces. Figure 22 shows Pointer s Gamut of real colours on the 1931 CIE xy chromaticity diagram, and the standardised colour spaces, BT.709, DCI P3, BT.2020, ACES and XYZ. It can be seen that while BT.709 and even DCI P3 do not cover the whole of Pointer s Gamut, 46 Jan Frohlich, Andreas Schilling, and Bernd Eberhardt, "Gamut Mapping for Digital Cinema," SMPTE Conf. Proc. vol. 2013, no. 10, pp (2013). 47 F. Ebner and M. Fairchild, Development and Testing of a Colour Space(IPT) with Improved Hue Uniformity, in the proceedings of the Sixth Color Imaing Conference: color Science, Systems, and Applications, page 8-13, The Gamut of Real Surface Colours, M. R. Pointer, Color Research & Application, IN STRICTEST Volume 5, Issue CONFIDENCE 3, pages , Autumn (Fall) 1980 Page 30

32 y BT.2020 does in fact cover 99.9% of it 49. Hence, although there are many colours IN that STRICTEST could not be CONFIDENCE described in BT.2020, there are very few naturally occurring colours that could not be described. So BT.2020 would appear to be a sufficient colour space for television systems to display realistic images CIE 1931 Pointer's Gamut BT.709 and srgb DCI P3 BT.2020 ACES XYZ x Figure 22. Comparison of Pointer s Gamut of real colour and STRICTEST standardised CONFIDENCE colour spaces. 4.9 Display Technology for Wider Colour Gamut LCD displays produce images by selectively filtering a backlight. Many individual LCD shutters, arranged in a grid, open and close to allow a metered amount of the light through, each shutter being paired with a coloured filter to remove all but the red, green or blue portion of the light from the original white source. The shade of colour is controlled by changing the relative intensity of the light passing through the LCD shutters 50. The colour gamut that can be produced by a conventional CCFL (cold cathode fluorescent lamp) backlight LCD is dependent on the spectrum produced by the backlight and the bandwidth of the filters used to produce the primaries. The gamut can me made larger by using narrower band filters on the backlight. However this also lowers screen luminance by decreasing the proportion of the backlight that passes through the filters. Increasing IN STRICTEST the luminance CONFIDENCE of the cold cathode to counter this effect tends to shorten the life of the device and often results in lighting irregularities 51. Wide gamut cold cathode fluorescent lamps are reported to be under development IN STRICTEST by many manufacturers, CONFIDEN but not yet widely adopted because of the shorter lifetime and lower optical efficiency 52. The use of white LED backlighting is now popular. Typically, white light is produced by using blue LEDs with YAG (yttrium aluminium garnet) phosphor. Such white light typically has a spectral peak in blue, from the LED, and a broad peak around yellow from the phosphor. After red, green and blue colour filtering, with relatively wide band filters to maintain a reasonable amount of brightness in the red and green primaries, the result is usually a smaller colour gamut than CCFL backlit displays. However, such displays can Five-Primary-Color LCDs, Hui-Chuan Cheng, Ilan Ben-David, and Shin-Tson Wu, IN STRICTEST Journal of display CONFIDENCE technology, Vol. 6, No. 1, January 2010, Page 31

33 achieve high dynamic range, alternatively referred to as high contrast, by dimming IN STRICTEST the LEDs in the CONFIDENCE dark areas of the image being displayed. RGB LED backlighting can also be used. Small groups of pixels in the LCD panel are backlit using triads of controllable red, green and blue LEDs. This increases the display s colour gamut and the bit depth, as well as providing deep black levels due to localised dimming of the backlight 53. The Dolby Professional Reference Monitor PRM-4200 is an example, using dual modulation technology, where the backlight unit consists of approximately 1,500 RGB LED triads, modulated on a frame-by-frame basis, that directly illuminate the LCD panel 54. In 2013, it was reported 55 that this was the only commercially available monitor capable of displaying the DCI P3 colour gamut. However since then other products have been announced. Another means to widen the gamut is to use more than three colour primaries. This has been demonstrated in projection displays and some direct-view LCDs. However, the display brightness is reduced and cost is increased because of the increased number of colour pixels and fabrication complicities 52. Quantum dots is another backlight technology, where blue LEDs are used, but instead of using a phosphor to produce white light, a layer of quantum dots is used. These act in a similar way to a phosphor, absorbing the blue light and emitting light with a different spectrum, which is determined by the size of the nanostructures. This allows narrow bandwidth light at the desired colour primary frequencies to be produced. Jeff Yurek of Nanosys, one of a number of organisations developing quantum dot technology, has reported that it is practical to produce a display that covers over 97% of BT.2020 using quantum dot technology, IN STRICTEST and has claimed CONFIDENCE to demonstrate over 91% BT.2020 just by modifying an off-the-shelf, standard LCD TV set with a specially tuned sheet of Quantum Dot Enhancement Film (QDEF) 56. The first TV manufacturer shipping TVs of this kind was Sony in 2013, using the trade mark Triluminos, and incorporating QD Vision s quantum dot technology Color IQTM 57. Quantum dot technology is also used in the Kindle Fire HDX Samsung Electronics, LG Electronics, the Chinese TCL Corporation and Sony showed quantum dot enhanced LED-backlighting of LCD TVs at the Consumer Electronics Show Samsung introduced an extensive range of SUHD TVs at the Samsung Forum in Monaco in February These UHD displays use quantum dot technology, referred to by Samsung as nano-crystal technology, STRICTEST to achieve CONFIDENCE a wide colour gamut, assumed to be consistent with DCI P3 due to their announcement of collaboration with 20th Century Fox who re-mastered multiple scenes from its film, Life of Pi, specifically for the SUHD TV. Another technology that promises to be able to achieve wide colour gamut is organic light-emitting diode (OLED) technology. Unlike LCD technology, OLED does not require backlighting to function. Light is produced by a process known as electrophosphorescence when current flows through an emissive layer of organic molecules, such as polyaniline 61. The colour of light emitted depends on the exact organic makeup of the molecules. As there is no backlight, an OLED can display deep black levels and can consequently achieve a higher contrast ratio than an LCD. Active Matrix OLED technology (AMOLED) is used in a number of smartphones, including leading models produced by Samsung. Sony announced the BVM-X monitor at the Hollywood Post Alliance Tech Retreat, 9-13 February Sony claims that this is its first OLED master monitor to combine UHD resolution, high dynamic range and IN STRICTEST wide colour gamut CONFIDENCE display. Sony claims that this 53 Color Correction Handbook: Professional Techniques for Video and Cinema, Alexis Van Hurkman Page 32

34 30-inch monitor has the ability to display ITU-R BT.709 and DCI-P3 colour gamuts IN more STRICTEST accurately CONFIDENCE than any previous Sony Trimaster display, and in addition, can display 80% of the ITU-R BT.2020 colour gamut. The Mitsubishi launched the LaserVue Series of DLP TVs in that were powered by lasers and which they claim was able to achieve 91.4% of the BT.2020 colour gamut. They also claim that of as of 2014, no other commercially available display has been able to achieve such a wide colour gamut. However, the displays, which retailed around $6000 were discontinued in December 2012 due to the inability to compete with the shrinking size and prices of LCD TVs. Displays that rely on a laser backlight system are understood to be very power-hungry and consequently impractical, and considering the European Commission eco-design directive, which is discussed further later in this research paper, unlikely to be legal in Europe due to low power efficiency. NHK laboratories are believed to have developed a laser projector, which is believed to be the only system currently capable of achieving the BT.2020 colour space, but it is likely to be very demanding of power and hence impractical for home use Page 33

35 5 Higher Frame Rate Motion reproduction is an important feature of television: to make a sequence of still images appear to be natural movement. When the frame rate is too low, artefacts are perceived by the viewer, which can be classified into flicker, motion blur, and jerkiness (stroboscopic effect). Flicker is a phenomenon in which the whole screen or a large area of the screen flickers due to low frame frequency. Motion blur is due to the accumulation mechanism of the capturing devices. It also happens IN when STRICTEST moving images CONFIDENC are tracked on the screen of a hold-type display. Non-smooth reproduced motion at low frame frequencies is called jerkiness. The use of the shutter during acquisition can produce a multi-exposure-like image even at high frame frequencies. This phenomenon is sometimes called jerkiness as well. Motion blur and jerkiness depend on the object speed. The object speed (arc-degree/second) generally tends to become faster as the IN field STRICTEST of view of the CONFIDENCE system becomes wider. For example, given the same shooting angle, an object on an HDTV screen will move faster than the one on an SDTV screen, assuming that screen sizes and viewing distances are such that the extra detail of HDTV can be observed, that is, the HD screen fills a larger part of the viewer s field of view than the SD screen. Consequently high definition IN STRICTEST TV systems have CONFIDENCE stricter motion portrayal requirements and UHDTV systems higher still. SDTV, HDTV, and current UHD systems are specified at frame rates up to 60 frames per second. However, it is claimed that the UHDTV viewing experience could be improved by the use of higher frame rates. 5.1 The artistic impact of frame rate The choice of frame rate may sometimes be determined by whether the scene contains normal or fast movement, but it is more frequently due to the director of photography s selection of a programme look : low frame rates are often used to create a film like look, regardless of the speed of movement or action in the scene. High frame rates are often used to obtain a more contemporary look with increased motion clarity, and/or to obtain sharper slow motion images. The cinema industry still favours 24 frames per second. Daly et al. 65 comment on STRICTEST this, stating CONFIDENCE that in addition to the historical association of motion judder with cinema, the motion blur associated with this frame rate allows the viewer s attention to be directed to parts of the scene in a similar way to the use of a shallow depth of field; motion blur is useful for hiding cinema craft (e.g., fake beards); and aids the suspension of disbelief, as realistic imaging could hinder the viewer s imagination. 5.2 Historical choices of frame rate In the early days of television, it was considered sufficient 66 for the frame rate to be high enough merely to exceed the threshold for apparent motion, above which a sequence of recorded images appear to the eye as containing moving objects rather than being a succession of still photographs. Another factor taken into consideration was that the frame rate should be high enough that flicker was imperceptible on contemporary televisions. However, priority was not given to the elimination of motion artefacts such as smearing and jerkiness, possibly as contemporary technology may have limited the benefits of a higher frame rate anyway The motion blur jerkiness trade-off Current television frame rates cause problems for motion portrayal. Stationary IN objects STRICTEST are sharp, CONFIDEN provided they are in focus, but objects that move with respect to the camera smear due to the integration time of the camera s sensor. Shuttering the camera to shorten the integration time reduces the smearing, but the motion breaks up into a succession of still images, causing jerkiness. The perceptual difference between moving and stationary subjects is increased with the increasingly sharper images due to new television systems with successively higher spatial resolutions, so long as the temporal resolution remains unchanged. The problems smearing, jerkiness or a combination of the two, are more noticeable with larger displays where the eye tends to follow the motion across the scene. 65 "A Psychophysical Study Exploring Judder Using Fundamental Signals and Complex Imagery", Scott Daly, Ning Xu, Jim Crenshaw, and Vickrant Zunjarrao, SMPTE Annual Conference, Zworykin, V. K. and Morton, G. A., Television: The Electronics of Image Transmission. Wiley. New York. 67 BBC White Paper WHP 209: Higher Frame rates for more Immersive Video and IN STRICTEST Television, R.A. CONFIDENCE Salmon, Mike Armstrong, Stephen Jolly. Page 34

36 Figure 23, copied from the BBC White Paper , shows the problem in terms IN of STRICTEST the movement CONFIDENCE of a ball across a plain background. In the top illustration, the trajectory of the ball is shown as if captured by a video camera with a very short shutter: each frame would show the ball frozen in time, and the motion would appear jerky when the video sequence was replayed. In the middle illustration, the effect of a half-open shutter is depicted: camera integration smears the motion of the ball out over the background, removing any spatial detail and making it partially transparent. These effects would be clearly visible in the final video sequence. The bottom image shows the effect of doubling the frame rate: both the smearing and jerkiness are reduced. A substantial further increase in frame rate would still be required in this example to eliminate their effects. Figure 23. The effects of frame rate and shuttering on motion portrayal. Without a change in frame rate and shutter speed, it takes only a small amount of motion, estimated by the BBC as three pixels per displayed image, to lose the benefits of HD over SD resolution. The problem becomes more severe as resolution increases to UHD. Just as shuttering in the camera reduces the extent of smearing, a sample-and-hold IN STRICTEST characteristic CONFIDENC in the final display increases it in a directly comparable fashion. This smearing arises with motion that is tracked by the viewer, where the viewer s eye is following the object across the screen, but where within each displayed image the object remains stationary for duration of the frame or field. LCD televisions have this characteristic, which is one reason why these displays have a reputation for representing fast-moving material, such as sport, poorly. LCD television manufacturers have added functionality to perform a motion-compensated frame rate increase, which ameliorates the problem to some extent at the cost of introducing IN STRICTEST other artefacts CONFIDENCE when the motion becomes too hard to predict, and during cuts and cross-fades. 5.4 Subjective evaluation of moving picture quality Emoto et al 68 have reported the results of experiments using a high-speed HD camera and projector that operate up to 240 frames per second, using typical television content including sports scenes. The content was presented at 60, 120 and 240 frames per second and viewers were asked to score on the five grade quality scale. The content was presented on a 100inch display, and the 60 viewers observed from 3.7m from the screen, that is, about three screen heights away. The results, as illustrated in Figure 24, show that the scores increase as the frame frequency increases, while the extent of the IN STRICTEST improvement CONFIDENCE depends on the content. The increase in scores when going from 60 to 120 frames per second range from 0.14 to 1.04, while the average increases across all of the content are 0.46 from 60 to 120 frames per second, and 0.23 between 120 and 240 frames per second M. Emoto and M. Sugawara, Flicker perceptions for wide-field-of-view and hold-type IN STRICTEST image presentations, CONFIDENCE IDW'09, pp , 2009 Page 35

37 Figure 24. Results of subjective evaluations of picture quality IN STRICTEST at higher frame CONFIDENC rates. The Broadcast Technology Future s Group (BTF) 69, a collaboration of the EBU, IRT, BBC, Rai, and NHK, looked at benefits of higher frame rates. This work used five test clips (train, bike, runner, football, and carousel, and shown in Figure 25), filmed at 240 fps with 100% shutter, and created lower frame rate versions from these, with different simulated shutter settings. The lowest frame rate was 30 fps. A 55 inch 1080p display was used as a higher resolution display would IN STRICTEST not support the CONFIDENCE higher frame rates. 26 viewers observed the content at 60, 120 and 240 frames per second at a distance of 3H, the point at which the 1080 HD pixels can just be discerned. The BTF reported a statistically significant improvement for all the test sequences when the frame rate was increased from 60 fps to 120 fps and when it was increased from 120 fps to 240 fps. The results averaged over the five test clips are shown in Figure 26. The difference between 60 fps and 120 fps was one grade, and the difference between 60 fps and 240 fps was nearly two grades. The BTF stated that there was some perceptible improvement in using a shorter (50%) shutter, but the results were not conclusive Page 36

38 Figure 25. BTF Test Material. Figure 26. BTF Average Experimental Results. Kuroki et al. 70 report results on the subjective impact of frame rate, where motion images from a high-speed camera and computer graphics were shown on a high-speed display. The camera captured at 1000 frames per second, and the resulting images were processed to simulate various frame rates and shutter speeds. A 480Hz CRT display IN STRICTEST was used to CONFIDENC present the content. Viewers were asked to evaluate the difference in quality between motion images at various frame rates. Their results show that a frame rate of 120 frames per second provides good improvement compared to 60 frames per second, and that the maximum improvement beyond which evaluation is saturated is found at about 240 frames per second for representative standard-resolution natural images. Nolan described 71 a theoretical approach to frame rates and the human visual system, combining sampling theory and the human contrast sensitivity function model, for viewing content at 1.5 picture heights from the display, for tracked and non-tracked motion. When using a 100% shutter, Nolan reported that to match the temporal resolution IN STRICTEST to the spatial CONFIDEN resolution, assuming classical sampling, for UHD content, the frame rate required was 140 frames per second for non-tracked motion and 700 frames per second for tracked motion. But it was also noted that an acceptable balance of motion blur and strobing is likely to be possible at a lower frame rate, and that there is still a need to understand the right balance between the visibility of blur (tracked) and strobing (nontracked) for a 100 fps frames per second system. 70 A psychophysical study of improvements in motion-image quality by using high frame rates, Y. Kuroki, T. Nishi, S. Kobayashi, H. Oyaizu, and S. Yoshimura, Journal of The Society for Information Display, Volume 15, Issue 1, pages 61 68, January "The Human Visual System and High Frame Rate Television", Katy C. Noland, UHDTV: Voices & Choices, 25th November 2013 Page 37

39 The BBC demonstrated higher frame rate technology during the Commonwealth IN Games, STRICTEST 2014, at CONFIDENCE the Glasgow Science Centre 72, to show the potential benefits of watching sport at higher frame rates. Content was captured using a For-A FT-One camera that is designed for use in slow-motion action replays: it is a 4K resolution camera, capable of recording a 10 second shot at 900 frames per second into its internal memory uncompressed. Content was captured and stored at 400 frames per second, and converted to HD resolution at 50 and 100 frames per second for display using two ceiling mounted Barco F85 projectors. A large portion of the public were reported to be able to see the benefits of higher frame rate straight away and were very impressed; others needed some prompting on where to look and eventually could appreciate the difference, while others couldn t see the differences at all, even IN with STRICTEST lots of prompting. CONFIDENC However, the latter were reported to be a small minority, about 5% of the visitors. 5.5 The impact of lighting frequency on frame rate The BBC have reported to the ITU-R the results of experiments on capturing content at 120 frames per second when illuminated with lighting at 50Hz. The results of these tests have led to the conclusion that 120 frames per second acquisition within 50 Hz territories would give rise to significant visual artefacts 19. While it would be desirable to have a worldwide agreement on a single higher frame rate for UHDTV, this work demonstrated that within 50 Hz territories there would be significant interaction between the camera settings and the different lighting types that are currently in common use. Many locations are equipped with legacy lighting systems that are not under the control of the broadcaster, and many events use lighting primarily for the benefit of the audience at the location rather than for the television coverage. These lights are suitable for 25/50 fps and 29.97/59.94 fps systems, but have been found to have shortcomings when using higher frame rates that are not related to the mains supply frequency in use at the acquisition location. NHK reported results on the same topic, reporting on measuring the flickering characteristics of lighting equipment for programme production as well as the waveform of the captured image at 120 fps, finding that the flickering frequency of the light must be equal to or higher than the frame frequency in order not to generate the flicker artefact by the flickering lights with any temporal characteristics. They also reported the trend of lighting equipment which is shifting STRICTEST to higher CONFIDENCE flickering frequency than the power frequency 19. In the latest version of BT.2020, table 2, Picture temporal characteristics, does not include the frame rate of 100 frames per second, but, perhaps due to lack of consensus within the ITU-R on the inclusion of this higher frame rate, instead states in a footnote to the table The additional frame rate of 100 Hz is used in a number of 50 Hz countries. 72 Higher frame rate television for future broadcasts at the Commonwealth Games But I already have a 100Hz TV?, Page 38

40 6 Standardisation A number of standardisation bodies are involved in the specification of digital television standards. Figure 27 shows a simplified view of this from a UK perspective for distribution of television to end users. The DTG publishes and maintains the technical specifications for Freeview, known as the D-Book. This primarily references DVB standards, and TS in particular. This DVB standard makes use of media coding and multiplexing from ISO/IEC, notably the so-called MPEG Transport Stream format and video compression codecs, MPEG-2, H.264/AVC and HEVC. This DVB standard also makes use of ITU-R Recommendations on TV signal formats, including BT.709, BT.1886, and BT.2020 which define transfer functions, IN colour STRICTEST primaries CONFIDENC and frame rates. Other standardisation bodies are also important but not shown. SMPTE produces standards that are of primary interest for the film industry and for contribution networks, being those networks used for moving content between production locations up to the point of final encoding for distribution to end users. DTG UK Digital TV Standards (DTG D-Book) DVB Digital TV Standards (TS ) MPEG Audio-visual Compression (ISO/IEC , ISO/IEC , etc.) ITU-R TV Signal Formats IN STRICTEST (ITU-R BT.709, CONFIDENC ITU-R BT.1886, etc.) 6.1 ITU-R Figure 27. The relationship between standards bodies involved in UK digital television standards. ITU-R SG6 WP6C is a Working Party within the International Telecommunication Union Radiocommunication Sector (ITU-R) concerned with signal formats for the making and exchange of television and radio programmes, and ways to evaluate picture and sound quality. WP6C currently has responsibility for standardisation of UHDTV, including high dynamic range, higher frame rates and wider colour gamut. Page 39

41 6.1.1 Parameters for Digital Television ITU-R Recommendation BT , Studio encoding parameters of digital television for standard 4:3 and wide-screen 16:9 aspect ratios, includes the definition of colour and opto-electronic transfer (OETF) characteristic for digital television. In CIE 1931 (x,y) coordinates, the colour primaries for 625 line television systems IN are STRICTEST red (0.640, CONFIDENC 0.330), green (0.290, 0.600), and blue (0.150, 0.060), and for 525 line television systems are red (0.630, 0.340), green (0.310, 0.595), and blue (0.155, 0.070). The reference white is D65 (0.3127, ), for both systems. The opto-electronic transfer function, commonly known as gamma, is defined as below as a combination of a linear segment near black and a power law segment for the majority of the range. { ITU-R Recommendation BT , Parameter values for the HDTV standards for production and international programme exchange, defines the image format parameters and values for HDTV. It defines the same transfer function as BT.601, above, and defines similar colour primaries, which again in CIE 1931 (x,y) coordinates, are red IN STRICTEST (0.640, 0.330), CONFIDENCE green (0.300, 0.600), and blue (0.150, 0.060). The reference white is D65 again. ITU-R Recommendation BT , Reference electro-optical transfer function for flat panel displays used in HDTV studio production, specifies the reference electro-optical transfer function (EOTF) that the displays used in HDTV programme production should follow in order to facilitate consistent picture presentation. The reference EOTF is specified as a simple equation, with exponent function, as shown below, based on measured characteristics of the Cathode Ray Tube (CRT). The parameters a and b correspond to the legacy user controls, with a representing the gain or contrast control, and b representing the black level lift or brightness control. ( [( ) ]) It was only in 2011, by which time the consumer market for CRTs had almost completely disappeared, that this standard for an EOTF was finally agreed. It effectively documents the characteristics of CRT displays. It does not specify a mathematical inverse of the OETF specified in BT.601 and BT.709, which by taking into account the linear portion, is approximately a power law function with exponent 0.5. The combined transfer function, sometimes known as the end-to-end gamma or system gamma, is approximately a power law function with exponent 1.2, which has been stated to have been designed to compensate for the subjective effects of viewing pictures in a dark surround at relatively low brightness 18. ITU-R Recommendation BT , Parameter values for ultra-high definition television systems for production and international programme exchange, defines the image format parameters and values for UHDTV. It specifies two resolutions, 3840x2160 and 7680x4320, frame rates up to 120 fps, and a wider colour gamut than BT.601 and BT.709. In CIE 1931 (x,y) coordinates, the colour primaries are red (0.708, 0.292), green (0.170, 0.797), and blue (0.131, 0.046). The reference white is D65 again. BT.2020 specifies the same transfer function as the earlier BT.601 and BT.709. However, it also states: that if it is shown that an alternative electro-optical transfer function will provide significant benefits without also imposing significant disadvantages, then this Recommendation should be extended to enable use with an improved EOTF. 73 ITU-R Recommendation BT.601, Studio encoding parameters of digital television IN STRICTEST for standard CONFIDENCE 4:3 and wide screen 16:9 aspect ratios ITU-R Recommendation BT.709, Parameter values for the HDTV standards for production and international programme exchange ITU-R Recommendation BT.1886, Reference electro-optical transfer function for flat panel displays used in HDTV studio production ITU-R Recommendation BT.2020, Parameter values for ultra-high definition IN television STRICTEST systems CONFIDENCE for production and international programme exchange. Page 40

42 6.1.2 High Dynamic Range ITU-R WP6C is understood to be considering this issue of alternative non-linear transfer functions to support high dynamic range. Proposals have been received from Philips, Technicolor, Dolby, and the BBC 77. It has been reported that ITU-R WP6C is planning to perform quality evaluations that will allow the proposals to be ranked for effectiveness 78. The differences between these proposals are believed to include the dynamic range that needs to be supported, whether the specification should be written in terms of absolute or relative brightness levels, and the importance of some degree of compatibility with the transfer function of BT.709. The BBC, in its white paper , has argued that the use of absolute luminance levels would require significant changes to the way television is produced and viewed, and that the very large dynamic range supported by the SMPTE standard ST , imagined to be consistent with the Dolby/USA proposal into ITU-R WP6C, is not needed for image display. The BBC white paper, which is imagined to be consistent with BBC submissions to ITU-R WP6C, describes a transfer function that is broadly compatible with BT.709, and which is claimed to facilitate compression and video processing using systems designed for BT 709, and which allows the display of pictures on existing displays with a quality IN at STRICTEST least sufficient CONFIDENCE for monitoring purposes. ITU-R WP6C held a meeting February No public statements appear to have been made about the progress or finalisation of this work. The next meeting is scheduled for July Colorimetry conversion ITU-R WP6C is understood to be considering the development of a Recommendation on colorimetry conversion from BT.709 to BT Unlike a conversion in the reverse direction, this should be relatively straightforward to specify, as all colours that could be represented with BT.709 colorimetry could be specified with BT.2020 colorimetry. Colour conversion from BT.2020 to BT.709 is more problematic, as there may be colours in the originating BT.2020 representation than cannot be represented in BT.709, while there is a desire to avoid changing STRICTEST colours that can CONFIDENCE be represented in BT.709. BT.2020 specifies a colour gamut for UHDTV that is extended with respect to that specified in BT.709 for HDTV. There is therefore a need to tailor the colour gamut of UHDTV content to the HDTV colour gamut, in situations where content produced in UHDTV is needed for HDTV production or broadcasting. Colour gamut mapping is not trivial. Unless due care is taken in colour space conversion, there is a concern that material that takes advantage of the extended UHDTV gamut could look inferior on an HDTV display compared to material originated directly in HDTV using BT.709. ITU-R WP6C is understood to have created a Rapporteur Group in March 2014 to consider the subject of colour gamut tailoring 80. No public statements appear to have been made about the progress of this work Higher Frame Rates Following the identification by the United Kingdom of issues of flicker when cameras capturing at 120 frames per second are used under 50 Hz lighting 19, ITU-R Recommendation BT.2020 was revised in March 2014 IN STRICTEST to add support CONFIDEN for 100 frames per second and 120/1.001 frames per second, but only as a footnote to the table of frames rates, presumably due to opposition to inclusion at all. The footnote, rather than mandating support for these frame rates, simple states: The additional frame rate of 100 Hz is used in a number of 50 Hz countries and The additional frame rate of 120/1.001 Hz is used in a number of 60 Hz countries, while it is still under study in a number of other countries. 77 Contributions to ITU-R WP6C, "Programme Production and Quality Assessment" Report on the meeting of Working Party 6C (Geneva, November 2014), Annex 06 - Preliminary draft new Recommendation ITU-R BT.[709to2020] - Colorimetry conversion from Recommendation ITU-R BT.709 to Recommendation ITU-R BT Report on the meeting of Working Party 6C (Geneva, March 2014), Annex IN STRICTEST 05 - Appointment CONFIDENCE of a Rapporteur Group - Colour gamut tailoring. Page 41

43 The meeting of ITU-R WP6C in February 2015 received at least one contribution IN proposing STRICTEST to move CONFIDENCE the 100 frames per second value into the main table SMPTE The Society of Motion Picture and Television Engineers (SMPTE) is an international professional association, based in the United States of America, of engineers working in the motion imaging industries. SMPTE has produced over 600 Standards, Recommended Practices and Engineering Guidelines for television production, filmmaking, digital cinema, audio recording, information technology, and medical imaging UHD Parameter Values The SMPTE standard ST , Ultra High Definition Television Image Parameter Values for Program Production, defines a family of progressive image sample structures for Ultra High Definition Television, with resolution 3840 x 2160 (termed UHDTV1) and 7680 x 4320 (termed UHDTV2), with an aspect ratio of 16x9, and with RʹGʹBʹ IN or STRICTEST YʹCʹBCʹR colour CONFIDENCE encoding and digital representation 83. The most recent version was approved in October Colorimetry is defined consistently with BT In addition, for UHDTV1 at frame rates up to 60 frames per second, BT.709 colorimetry can be used. The transfer function ( gamma ) is consistent with BT.2020 IN STRICTEST and BT.709. CONFIDENCE The 2013 revision of the standard listed, for both resolutions, various frame rates up to 60 frames per second, but also 120 frames per second. This was intended as a single worldwide higher frame rate for UHDTV. However, following publication, requests were received to also include 100 and 120/1.001 frames per second. Consequently the 2014 revision also lists the frame rates 100 and 120/1.001 frames per second for both resolutions, although it does state It is not necessary for an implementation to support all formats to be compliant with this standard. It would appear that there is still no consensus among SMPTE members on which of the higher frame rates should be supported. The UHDTV Ecosystem Study Group Report produced by SMPTE in March , and hence before the latest revision of ST , states SMPTE TC members were divided on whether to include 120 fps, 120/1.001, STRICTEST or both CONFIDENCE of these frame rates for UHDTV, and reports the arguments made for and against 120/1.001 and 120 frames per second, where the major factor in favour of the fractional frame rate is interoperability with the massive library of existing content, and where the major factor in favour of the integer frame rate is simplicity, particularly in respect to time related labels. No argument is reported against inclusion of 100 frames per second, which as described earlier in this report, was found to achieve better performance with 50Hz lighting than 120 frames per second High Dynamic Range SMPTE published its first HDR standard, SMPTE ST 2084, High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays, in September It specifies an electro-optical transfer function (EOTF) characterising high dynamic range reference displays used primarily for mastering non-broadcast content. It also specifies IN STRICTEST an Inverse-EOTF CONFIDENCE derived from the EOTF. The EOTF has a high luminance range capability from 0 to 10000cd/m 2, and as it is referenced to absolute luminance, the display is assumed to be operating in a specified reference viewing environment. It is intended IN STRICTEST to enable the CONFIDEN creation of video images with an increased luminance range, and not the creation of video images with overall higher luminance levels. 81 Document 389, Soliciting inclusion of a 100 Hz frame rate among those listed in Table 2 of Recommendation ITU-R BT Page 42

44 The EOTF is derived from the contrast sensitivity function of the human visual system IN STRICTEST that was developed CONFIDENCE by Peter Barten 16. It is claimed to allow the mapping of digital code values containing as few as 10 bits to the large, absolute luminance range from 0 to 10000cd/m 2. It is shown in the equation below. ( [( ) ] ) N denotes non-linear colour value L denotes linear colour value Colour Volume Metadata SMPTE published SMPTE ST 2086, Mastering Display Color Volume Metadata Supporting High Luminance and Wide Color Gamut Images in October This standard specifies the metadata items to specify the colour volume, that is, the colour primaries, white point, and luminance range, of the display that was used in mastering video IN STRICTEST content. The metadata CONFIDENCE is specified as a set of values independent of any specific digital representation. It is applicable to three-color additive display systems, such as RGB displays Digital Cinema SMPTE has produced a number of standards for Digital Cinema: the ST 428, ST 429, ST 430, ST 431, ST 432, and ST 433 suites, along with the ST 2048 suite. These document the image formats and workflows that are optimised for theatrical presentation and not television reproduction 84. The SMPTE standard ST 428-1, D-Cinema Distribution Master - Image Characteristics, approved in 2006, describes the image characteristics of the Digital Cinema Distribution Master (DCDM) 87. It describes STRICTEST three operational CONFIDENCE levels, being 2048x1080 at 24 and 48 frames per second, and 4096x2160 at 24 frames per second. CIE XYZ colorimetry is specified. The transfer function is a power law function (similar to gamma of BT.709), with exponent 1/2.6, and peak luminance of 52.37cd/m 2. The SMPTE Recommended Practice RP for Digital Cinema Quality Reference Projector and Environment, first released by the SMPTE in 2007, and most recently updated in 2011 as RP 431-2: It defines the Reference Projector and its controlled environment, along with the acceptable tolerances around critical image parameters for Review Room and Theatre applications. The transfer function is a power law with exponent 2.6. The colour gamut is specified in terms of XYZ. A projector is required to support a gamut of at least DCI-P3, specified by the primaries red (0.680, 0.320), IN green STRICTEST (0.265, 0.690) CONFIDENC and blue (0.15, 0.060), but a projector may have a larger gamut. A maximum luminance of 48cd/m 2 shall be supported. The SMPTE standard ST , " and Digital Cinematography Production Image Formats FS/709", approved in 2011, defines a family of progressive sample structures of and images for Digital Cinema content creation 89. The standard also defines what it refers to as Free Scale-Gamut (FS-Gamut), allowing images having arbitrary chromaticity to be conveyed. However, images can also be conveyed using BT.709 colorimetry, as suggested by the 709 in the title of the standard. In CIE 1931 (x,y) coordinates, the colour primaries are red ( , ), green IN STRICTEST ( , ), CONFIDEN and blue ( , ), and the reference white is D65. It has been suggested that the primaries relate to the Sony wide gamut delivered by the F23, F35, and F65 cameras 90. The standard also defines a transfer function which it refers to as Free Scale-Log (FS-Log) curve. This has a logarithm characteristic over the middle part of the range, and optionally over the whole range, and is specified by four parameters and an exposure "Free Scale Gamut, Free Scale Log (FS-Gamut, FS-Log)", Charles Poynton. Gamut-Free-Scale-Log.html Page 43

45 parameter. The top and bottom of the curve can optionally be specified as a non-linear IN STRICTEST function, CONFIDENCE defined as a smooth curve through three indicated points. The standards states that the FS-Log curve has an affinity with the sensitivity of human eye and can specify a much wider dynamic range than the curve defined in BT.709. This standard also defines the Color VANC, which is ancillary data that conveys the parameter values of the user-defined colour space and Log curve. SMPTE ST states that the Color VANC data, when transported through a single-link or a multiple-link HD-SDI shall be transmitted in link A or the first link of a multi-link interface transport once per frame. The recommended mapping location of the Color VANC packet is in the vertical ancillary data space of line 18 of the Y data stream Colour Equations The SMPTE Recommended Practice RP 177:1993, Derivation of Basic Television Color Equations, defines the numerical procedures for deriving basic colour equations for colour television and other systems using IN additive STRICTEST display CONFIDENCE devices. In Annex B, an example of these calculations is given for the D65 white point and the colour primaries, as also specified in BT On-going Activities SMPTE has a Study Group that is compiling a report on the High Dynamic Range IN Ecosystem. STRICTEST It also CONFIDENCE has a new project defining Dynamic Metadata for Color Transforms of High Dynamic Range and Wide Colour Gamut Images 92. The Study Group on High Dynamic Range Ecosystem aims to identify the specific parameters that constitute High Dynamic Range. The Study Group will identify and report on requirements for new SMPTE Standards, Recommended Practices, and Engineering Guidelines; identify necessary revisions of current SMPTE Engineering Documents, and provide recommendations for the prioritisation of work within SMPTE 93. High dynamic range technology proposals have been reported to have been received from the BBC, Philips, Technicolor and NHK. The Drafting Group on Dynamic Metadata for Color Transforms of HDR and WCG Images plans to develop standards for specifying the semantics and representation of content-dependent metadata needed for STRICTEST colour volume CONFIDENCE transformation of high dynamic range and wide colour gamut imagery to smaller colour volumes (e.g. BT.709 or Digital Cinema) in mastering applications 94. It is stated that content dependent metadata is required to ensure creative intent is maintained: if colour mapping were performed without this metadata, the resulting out of gamut content could suffer severely in visual quality due to clipping. It is reported that although the science of colour volume mapping has advanced significantly recently, there is no single method to map adequately between different colour volumes. Instead, a colour transformation with content-dependent parameters could be used. For example, a very dark scene should be mapped differently from a very bright scene when transforming from a large colour volume to a more restricted colour volume, but it is claimed that this could only IN be STRICTEST achieved effectively CONFIDENC when this transformation is guided by metadata. The Study Group on Integer and Fractional Frame Rate Conversion is considering the issue of whether it would be practical to achieve real time high quality conversion between integer and fractional frame rates so that the use of fractional frame rates above 60 frames per second could be discontinued. It was reported in December that a Request For Information had been agreed and that it would be sent to organisations that could comment on the performance that could be achieved with available IN conversion STRICTEST technology. CONFIDEN It was hoped that technology demonstrations could be provided in the March 2015 timeframe Standards Quarterly Report December 2014, E Study Group on HDR Ecosystem E Dynamic Metadata for Color Transforms of HDR and WCG Images. Page 44

46 6.3 MPEG The Moving Picture Experts Group (MPEG) is a working group of ISO/IEC, specifically ISO/IEC JTC 1/SC 29/WG 11, with the mission to develop standards for coded representation of digital audio and video and related data. MPEG, working jointly with the ITU-T, has developed the well-known video compression standards MPEG-2 (officially known as ISO/IEC and ITU-T Recommendation H.262), H.264/AVC (Advanced Video Coding, officially known as ISO/IEC and ITU-T Recommendation H.264), and most recently HEVC (High Efficiency Video Coding, officially known as ISO/IEC and ITU-T Recommendation H.265) High Efficiency Video Coding HEVC, High Efficiency Video Coding, is the latest compression standard to be developed IN STRICTEST jointly by CONFIDENCE MPEG and ITU-T, in the Joint Collaborative Team on Video Coding (JCT-VC). It enables the bit rate for video services to be approximately halved compared to the previous standard, H.264/AVC, while achieving the same video quality. The first version of this standard was approved in January 2013, where three profiles were specified: Main Profile, Main 10 Profile, and Main Still Picture Profile. It is expected that Main Profile will be used for mass market video services that historically require only 8 bits of precision in their processing, whereas Main 10 IN Profile, STRICTEST which CONFIDENCE supports up to 10 bits of processing precision, is expected to be used where additional accuracy is needed, notably for encoding video that has a high dynamic range. The second version of this standard was approved in July 2014, where three amendments were added to the original specification. The first, known as the Range Extensions amendment, added support for colour sampling formats beyond 4:2:0 and up to 16 bits of processing precision. The second, known as Multiview HEVC (MV-HEVC), added support for providing efficient representation of video content with multiple camera views and optional depth map information, such as for 3D stereoscopic and auto stereoscopic video applications. The third, known as Scalable HEVC (SHVC), added support for embedded bitstream scalability in which different levels of encoding quality are efficiently supported by adding or removing layered subsets of encoded data. The second version of the HEVC compression standard includes features to support STRICTEST high dynamic CONFIDENCE range and wide colour gamut. In the Video Usability Information (VUI), a code point was added to indicate the colour primaries CIE 1931 XYZ, as specified in SMPTE ST 428-1, adding to a number of existing colour primary code points, including those indicating BT.709 and BT And code points were added to indicate the transfer characteristics specified in SMPTE ST 2084 and SMPTE ST 428-1, adding to a number of existing code points including those indicating BT.709 and BT HEVC version one specifies a tone mapping Supplemental Enhancement Information (SEI) message, which allows multiple mappings from the encoded dynamic range to a different, typically smaller, dynamic range, for display on devices that support a different dynamic range to that of the encoded representation. The mappings are defined as one dimensional look up tables, which apply in either the luma or RGB colour space domain, and which should be applied to the luma component or to each RGB component. A related SEI message, knee function information, was added in the second version of the HEVC compression standard. This allows one or more mappings from the dynamic range of the encoded stream to the dynamic range of the display to be specified, allowing the dynamic range of the encoded stream to be compressed or decompressed, that is, supporting both use cases of displaying high dynamic range content appropriately on a standard dynamic range display and displaying standard dynamic range content appropriately on a high dynamic range display. The mapping is specified as a piecewise linear function, where the pieces join at socalled knee-points, mapping normalised encoded values to normalised display values, and also signalling, in units of cd/m 2, the peak luminance level of the encoded representation and the peak luminance level of the representation for display. A mastering display colour volume SEI message was also added in the second version of the HEVC compression standard. This allows the colour volume (the colour primaries, white point, and luminance range) of the mastering display for the video content to be indicated. This information is intended to be adequate for purposes corresponding to the use of SMPTE ST However, it does not provide information on the colour transformations that would be appropriate IN STRICTEST to preserve CONFIDENCE creative intent on displays with colour volumes different from that of the described mastering display. This functionality is however supported by the colour remapping information SEI message which was also added in the second version of the HEVC compression standard. This allows one or more mappings from the colour space of the encoded stream to the colour space of the display to be specified, so as to preserve the artistic intent, that is, it specifies mappings from the colour graded encoded content to content suitable for the display, and approximating the fidelity of the content had it been specifically colour graded for the display. For example, it could specify a mapping from BT.2020 to BT.709 to enable high fidelity reproduction of content, mastered for BT.2020, on the BT.709 capable display. The colour remapping IN STRICTEST model is composed CONFIDENCE of a first piecewise linear Page 45

47 function applied to each colour component, followed by a three by-three matrix IN applied STRICTEST to the three CONFIDENCE colour components, followed by a second piecewise linear function applied to each colour component The Future of Video Coding Standardisation A brainstorming session of the Future of Video Coding Standardisation was held during the meeting of MPEG in October 2014 to explore use cases, requirements and potential timelines for the development of future video coding standards. Guest speakers included Roger Bolton of Ericsson, Harald Alvestrand of Google, Zhong Luo of Huawei, Anne Aaron of Netflix, Stéphane Pateux of Orange, Paul Torres of Qualcomm, and JeongHoon Park of Samsung, who each provided presentations discussing the needs of applications in their industry segments 95. A range of opinions was expressed by the panellists and other participants from IN industry STRICTEST and academia, CONFIDENCE who shared a common view that further increases in compression efficiency remain a fundamental need. MPEG set up two ad-hoc groups to continue the study of future application requirements and to begin to establish a roadmap for future video coding standardisation. The presentations at the brainstorming session are available on the MPEG web IN site STRICTEST 96. CONFIDENCE High Dynamic Range and Wide Colour Gamut MPEG is currently in the process of determining whether extensions to the HEVC standard would provide significant benefits for the compression of video with high dynamic range and wide colour gamut, compared to the performance that could be achieved with the current version of the standard. To achieve this, immediately following its meeting in February 2015, MPEG issued a Call for Evidence of new tools that may improve the performance of HEVC when used to encode high dynamic range and wide colour gamut video. The results of the Call for Evidence will be analysed to determine if MPEG should launch a new standardisation effort to extend the current capabilities of HEVC 97. In order to allow the various technology proposals to be compared, MPEG has received test content with high dynamic range and/or wide colour gamut from a number of companies. This content was submitted in one of the following formats: OpenEXR file format, linear light, and BT.709 colour gamut; and TIFF files, 12 bit Perceptual Quantiser (SMPTE ST 2084) non-linear samples, and DCI-P3 colour gamut. The test content is RGB with 4:4:4 sampling, and resolution 1920x1080 progressive. The test content is defined with respect to the BT.2020 container, and in addition three of the test sequences submitted with BT.709 colour gamut are additionally defined with respect to the BT.709 container. While it is expected that high dynamic range content would be delivered using the BT.2020 colour gamut in commercial services, IN STRICTEST the additional CONFIDENC use of the BT.709 container for some test sequences would allow the impact of colours at the edge of the colour gamut to be assessed. While it could have been possible to generate artificial content with colours near the edge of the BT.2020 colour gamut, there are no available screens that could display the processed content, and hence to investigate edge of colour gamut issues, the BT.709 container had to be used. The MPEG developed software tools known as HDRConvert have been used to IN convert STRICTEST the test content CONFIDENCE into the format that was used to generate what MPEG calls the anchors, that is, the base case against which proposals will be compared and against which an improved performance will be necessary for standardisation work to be performed. This format is YCbCr, 4:2:0, 10 bit Perceptual Quantiser, and two colour containers (BT.2020 and BT.709), encoded with various IN STRICTEST fixed quantisation CONFIDEN settings with HEVC using the Main 10 Profile and the Scalable Main 10 Profile. The timeline for the work is as follows. The Call for Evidence and the anchors were made available during February Expressions of interest in responding to the call must be registered by the end of March. Encoded content must be submitted by 26 May, with documents registered and submitted by 8 June. Evaluation of the submissions, including subjective assessment, will be carried out from 8-26 June. 95 MPEG Press Release, 24 October Presentations of the Brainstorming Session of the Future of Video Coding Standardization MPEG Press Release, 20 February Page 46

48 The Call for Evidence defines the following test conditions related to the different IN STRICTEST forms of scalability CONFIDENCE between standard and high dynamic range video coding Compression efficiency improvement over HEVC Main 10 Profile. This case is not concerned with compatibility with standard dynamic range equipment, and is simply concerned with how efficiently high dynamic range video could be encoded, and specifically whether a significant gain could be achieved over what could be done with the currently defined HEVC Main 10 Profile. 2a and 2b. Backward compatibility with legacy standard dynamic range decoders. In this case, the encoder produces two streams: a compressed high dynamic range stream and a standard dynamic IN STRICTEST range stream CONFIDENC that could be decoded by a Main 10 decoder. The former stream could consist simply of metadata to enhance the latter, or could also or alternatively include enhancement data. The aim is to quantify the benefits relative to simulcast of standard and high dynamic range bitstreams, and relative to the use of SHVC with Colour Gamut Scalability using the Scalable Main 10 profile. There are two variants of this test case. In case 2a, the encoder is provided with the source in both high and standard dynamic range formats, and is provided with an encoded standard dynamic range bitstream; and in case 2b, the encoder is provided only with a video source with high dynamic range and the encoder is responsible for generating and encoding a standard dynamic range stream. 2c and 2d. Backward compatibility with legacy standard dynamic range displays. This case is concerned with new, high dynamic range capable decoders, being connected to legacy displays. In this case, the encoder produces two streams: a compressed high dynamic range stream and a metadata stream to help a new receiver to create standard dynamic range video content from the high dynamic range stream. The primary aim is not to test or standardise the tone mapping algorithms, but to determine the type of metadata that is needed and the bit rate that would be needed for it. There are two variants of this test case. In case 2c, the encoder is provided with both high and standard dynamic range video content and in case 2d it is provided only with high dynamic range video content. In both cases the encoder must generate the metadata to enable the high dynamic range to standard dynamic range conversion to be performed in a receiver by a tone mapping process using that metadata. 3. Optimisation of the existing Main 10 Profile and/or Scalable Main 10 Profile. This case is concerned with nonnormative changes to the HEVC standard (Main 10 Profile and/or Scalable Main 10 Profile) that would provide better coding efficiency than the anchors used in cases 1 or 2a. Such changes include, but are not limited to, improved pre- or post-processing, better colour sub-sampling, the use of different colour STRICTEST domains for CONFIDENCE encoding, and the use of constant luminance. Submissions received for cases 1 and 2a will be subject to both objective performance assessment and subjective testing. Submissions received for the other cases will be classified as Technology Under Consideration where testing and next steps will depend on the submissions received: submissions will be viewed by experts to investigate the relationship between visual quality and bit rate. Objective performance assessments are to be made using the three objective metrics IN STRICTEST defined in CONFIDENC the Call for Evidence. Subjective testing will be done using a SIM2 monitor in two locations and a Pulsar monitor in another location. Cropped versions of the reference and the compressed video will be presented side by side on a single monitor, with viewers asked to assess the visual impairment. The action MPEG will eventually take on this work will depend on the performance IN STRICTEST of the technology CONFIDENCE submitted. A first possibility is that no standardisation activity is required at all, if efficient high dynamic range and wide colour gamut coding could be provided without any normative changes. A second possibility is that minor changes to the HEVC standard would be required to optimise the performance of the existing Main 10 Profile for high dynamic range and wide colour IN STRICTEST gamut content, CONFIDEN for example, by enabling the carriage of additional signalling. A third possibility would involve normative changes to the HEVC standard that would result in a new profile; such normative changes would probably need to be justified by a significant decrease in the bit-rate required to give a comparable visual quality to that of the Main 10 profile. It is also possible that more than one solution will be developed. 6.4 DVB The Digital Video Broadcasting Project is an industry-led consortium of over 200 broadcasters, manufacturers, network operators, software developers and regulators from around the world committed to designing open technical standards for the delivery of digital television. 98 Call for Evidence (CfE) for HDR and WCG Video Coding. Page 47

49 ITU-R recommendation BT specifies two sets of picture spatial characteristics, IN STRICTEST with resolutions CONFIDENCE 3840x2160 and 7680x4320. The DVB is currently working on standards for UHDTV, relating to this specification. The DVB concluded that initial client implementations would not be able to support the full set of parameters of BT.2020 at the lower of the two resolutions, and hence split their standardisation work into three phases. The first of these three phases, known as UHD-1 Phase 1, supports a subset of the full set of parameters of BT.2020 at the lower of the two resolutions, the second, known as UHD-1 Phase 2, is planned to support the full set of parameters of BT.2020 at the lower of the two resolutions, and the third, known as UHD-2, is expected to support the full set of parameters of BT.2020 at the higher of the two resolutions. This is illustrated in Figure 28, where it should be noted that IN while STRICTEST UHD-1 Phase CONFIDENC 1 is fully defined, the descriptions of the subsequent phases are tentative. UHD x4320 UHD-1 Phase /7 3840x2160, Up to 120 fps 4:2:0 10 bit BT Wide Colour Gamut Higher Dynamic Range More Advanced Audio UHD-1 Phase x2160 Up to 60 fps 4:2:0 10 bit BT.709 and BT.2020 Existing Audio Toolbox Backward Compatibility? Figure 28. The parameters proposed for each of the three phases of UHD in DVB UHD-1 Phase 1 The commercial requirements for UHD-1 Phase 1 were prepared in autumn IN STRICTEST The specification CONFIDENCE was completed in July 2014, when the DVB Steering Board approved a new version of TS , the DVB specification for the use of Video and Audio Coding in Broadcasting Applications based on the MPEG-2 Transport Stream. The new specification has been passed to ETSI for publication as TS v2.1.1 which can be downloaded from the DVB website 99. The main change in this, UHD-1 Phase 1, revision of TS is the addition of High Efficiency Video Coding (HEVC), for which DVB has defined five decoder conformance points, four at HD resolution (the four combinations of the 25 and 30 frames per second families and Main and Main 10 profile), and one at UHD resolution (all frame rates with Main 10 profile). The conformance point at UHD resolution is specified as follows. HEVC video encoding shall be used, as specified in ITU-T Recommendation H.265 IN / STRICTEST ISO/IEC , CONFIDENCE using the Main 10 profile at levels up to and including 5.1, unless the HEVC bitstream is an HEVC temporal video sub-bitstream. The aim in the latter case is to enable temporal scalability with a subsequent version of the specification. In addition to the spatial resolutions that HDTV HEVC receivers shall support, UHDTV HEVC receivers shall be able to support the spatial resolutions, 3840x2160, 2880x2160, 3200x1800, and 2560x1440, with display aspect ratio of 16:9, displaying received 99 Page 48

50 pictures full screen size. These resolutions are progressively scanned at any of the IN STRICTEST following frame CONFIDENCE rates: 24/1.001, 24, 25, 30/1.001, 30, 50, 60/1.001 and 60 frames per second. Receivers shall be capable of decoding bitstreams that use BT.709 or BT.2020 non-constant luminance colorimetry. The specification acknowledges the problem of correctly handling BT.2020 colorimetry in the short term, and recommends that any conversion to BT.709 is not done badly, specifically identifying two methods not to use: receivers should not impose a hard limit such that all colours outside of the gamut of the display are placed on the outer boundary of the display gamut; and receivers should not linearly scale the wider gamut of the bitstream to fit within the gamut of the display. The first commercial service compliant with UHD-1 Phase 1 is believed to be that offered by DirecTV 100 Error! Bookmark not efined.. This requires an Internet-connected Genie HD DVR and a DIRECTV 4K Ready TV, being a UHD TV that supports RVU technology and has UHD decoding capability. The service allows a selection of UHD movies to be viewed instantly UHD-1 Phase 2 The DVB is already working on UHD-1 Phase 2, with the Commercial Module discussing IN STRICTEST the relevant CONFIDENCE parameters. While UHD-1 Phase 1 focused on resolution, UHD-1 Phase 2 will bring higher frame rates, high dynamic range, a wider colour space as well as immersive audio 102. A large number of interested parties are participating in this process and in June IN 2014 STRICTEST over 90 people CONFIDENCE came together, in a combined EBU/DVB workshop, to discuss the relevant parameters for High Dynamic Range. Although UHD-1 Phase 2 is still very much a work in progress, it is expected to support frame rates up to 120 frames per second, at least 10 bits per sample, a bitstream format specified relative to the BT.2020 colour primaries, although display technology may only be capable of displaying a subset of the colours that could be signalled, and more advanced audio. It is planned to finish the specification in time to allow services to be launched in the 2017 / 2018 time frame 103. The main topics of discussion for UHD-1 Phase 2 include the degree of compatibility with DVB UHD-1 Phase 1; the details of how high dynamic range would be signalled and supported, how sub-titles are supported, advanced audio, content protection, and the impact of other organisations and specifications, for example the Blu-Ray Disc STRICTEST Association 100. CONFIDENCE DVB has been in communication with Digital Europe on the likely capabilities of televisions in the 2016/17/18 timeframe, and in particular the capabilities to support higher dynamic range. This may be a chicken and egg situation with DVB wanting to know the likely capabilities of televisions in the future so that they could draft suitable standards, and the television industry, represented by Digital Europe, wanting to know the likely content of future standards in order to plan the development of next generation televisions. There appears to be some desire within DVB for UHD-1 Phase 1 receivers to be IN able STRICTEST to access UHD-1 CONFIDENC Phase 2 services. While this should be technologically feasible, with HEVC compression supporting scalable modes of operation, the cost of this in terms of increased bit rate over non-scalable solutions is not yet known. Hence whether scalability is worth supporting at all and whether both compatible and non-compatible solutions for UHD-1 Phase 2 need to be specified are questions that are still open UHD-2 UHD-2 is a placeholder for a possible future specification with a resolution of 7680x4320, IN STRICTEST with a CONFIDEN tentative timeframe of 2020 or later. DVB has not discussed UHD-2 yet, at least in part because DVB knows of no plans to make UHD-2 consumer displays available. 100 UHDTV: Television a point, David Wood, Consultant, EBU Technology and Innovation DVB Scene Issue DVB Developments and Future Standards Trends, Peter Siebert, DVB Project IN STRICTEST Office. CONFIDENCE Page 49

51 Power (W) However, Japanese broadcaster NHK has confirmed that it wants to push for 8k IN broadcasting STRICTEST as CONFIDENCE soon as possible and will begin testing a service by NHK wants to offer an 8k broadcasting service by 2020, the year of the Tokyo Olympics and Paralympics Eco-design requirements for electronic displays There is concern within the DVB and within Digital Europe on whether a new draft European Commission regulation 105 implementing the eco-design directive 106 with regard to eco-design requirements IN STRICTEST for electronic displays CONFIDENC would allow manufacturers to improve luminescence and move towards higher dynamic range. Figure 29 shows an interpretation of the draft regulation, plotting the power constraint against the screen size, measured as the screen diagonal in inches, for 12, 36 and 60 months after the publication of the Regulation in the Official Journal of the European Union. The draft regulation states that power measurements shall be made using a dynamic IN STRICTEST broadcast-content CONFIDENCE video signal representing typical broadcast content for electronic displays, with the measurement being the average power consumed over ten consecutive minutes after the display has been switched on sufficiently long, typically an hour, to achieve stable operation. DVB has asked Digital Europe for their opinion on the consequences of this draft regulation. Digital Europe is believed to be continuing to discuss internally the possible impact of these energy limitations on the feasibility of higher dynamic range displays. DVB is considering whether specifications should be designed to limit or optimise IN STRICTEST the display energy CONFIDENCE consumption. For example, should the statistical distribution of luminance be constrained? Months 36 Months 60 Months Screen Diagonal (Inches) 6.5 EBU Figure 29. The proposed future power constraints IN STRICTEST on displays. CONFIDENCE The European Broadcasting Union is an alliance of public service media entities. The EBU's technical activities include provision of technical information to Members via conferences and workshops, as well as in written form (such as the EBU Technical Review, and the EBU tech-i magazine) Possible requirements for electronic displays Directive 2009/125/EC of the European Parliament and of the Council of 21 October 2009 establishing a framework for the setting of ecodesign requirements for energy-related products. Page 50

52 The EBU Technical Committee, which comprises 13 members, and which coordinates IN STRICTEST and manages CONFIDENCE the EBU's technical work, produced a policy statement on UHDTV in July This stated the opinion that the EBU Technical Committee believes that the current focus of the consumer electronics industry to provide only an increased resolution ( 4k ) and ignoring other enhancements, of for example, DVB UHD-1 Phase 2, is not a sufficiently large step for the introduction of successful new broadcasting services. The report notes that an enhanced, 1080p-based HD service that includes a certain combination of UHDTV parameters, higher frame rate, higher dynamic range, wider colorimetry and advanced sound system audio, but which does not include higher resolution, is not yet standardised. The report states that such a 1080p-based HD format could be an appealing option for some broadcasters and should be taken into account in the standardisation and investigation IN STRICTEST process. CONFIDENC The EBU proposes that an enhanced 1080p format be developed for broadcasting. 6.6 DTG The Digital TV Group (DTG) publishes and maintains the technical specifications for Freeview, known as the D-Book. The DTG also runs DTG Testing, the ISO17025 accredited digital TV test centre which provides benchmark test and conformance services for the Freeview and Freesat services and international platforms. The DTG established the UK UHD Forum in August 2013 to bring together the entire UK UHD ecosystem of content owners, platform operators, broadcasters, CE makers and silicon vendors, to try and agree what is meant by UHD beyond the number of pixels 108. The aim is to define a conformance specification for 4K and higher definitions, also IN considering STRICTEST high dynamic CONFIDENCE range, wider colour gamut and immersive 3D audio. It is co-chaired by Chris Johns, chief engineer for broadcast strategy at Sky and Andy Quested, head of technology for HD and UHDTV at the BBC 109. The DTG has run two 4k plugfests to date, and is planning another. A first 4k plugfest was held in October Testing focussed on the 4k HDMI interoperability of 12 UHD displays from ten different manufacturers, all of which displays were available in retail at the time, two set-top boxes and a reference set top box design from a silicon vendor, with HDMI test tools supplied by Rohde & Schwarz and Quantum Data. Participants tested different combinations of outputs and 4k resolutions, frame rate and bit depth to build a picture of which HDMI features were supported. It also inaugurated the DTG's 4k receiver and set top box collection, known as the '4k Zoo' 110. A second 4k plugfest was held in January 2015 to investigate HEVC UHD interoperability. Testing was carried out on a range of 4k receivers, integrated televisions and set top boxes, representing the available 4k receiver market of 2013 and 2014, as well as new 2015 models and prototypes from leading silicon vendors. Broadcasters, satellite operators, test labs and test equipment providers contributed UHD HEVC test signals encoded in main and main 10 profiles, covering a variety of resolutions, frame rates, bit depths and colour space. These were delivered to the 4k receivers via USB, modulated as satellite and DTT signals, and using MPEG- DASH 111. This testing showed that there was still limited support for the HEVC compression standard. While all of the 2015 models that were tested supported UHD at 50 and 60 frames per second with HEVC encoding, only 60% of the older models did so. Additionally, only half the models tested were found to support MPEG-DASH adaptive bit-rate streaming 112. The DTG plan another plugfest in April 2015 with the aim of testing HDCP 2.2 copy protection, looking at whether all legacy sources will work with HDCP 2.2 displays and what experience viewers will get with HDCP 2.2 into lower version displays. 6.7 Blu-ray Disk Association The Blu-Ray Disc Association (BDA) is developing the next generation optical disc package media format, launched as Ultra HD Blu- Ray at CES This new Blu-Ray format will support UHD with support for High Dynamic Range. It has been reported that the 107 Tech Report EBU Policy Statement on UHDTV UK will not go it alone with UHD DTG DTG UK UHD Forum welcomes formation of US UHD Alliance K HDMI plugfest inaugurates DTG Testing's Ultra HD testing zoo DTG's 4k Plugfests turn spotlight on HEVC streams Ultra HD technology alignment now close to tipping point for services. Page 51

53 technical specification will be released to licensees in mid-2015, and that the first IN STRICTEST titles are expected CONFIDENCE to be on the market before the end of the year 113. It has been reported 114 that the new disk format will support resolutions up to 3840 x 2160, frame rates up to 60 frames per second, 10-bit colour depth, BT.2020 colour space 115, and HEVC (H.265) video compression technology, on disks from 66GB (dual layer) up to 100GB (triple layer). It has been reported 116 that at least three HDR technologies will be supported, including what has been reported to be an open HDR specification, which is believed to be SMPTE ST 2084, and which is believed to have also been reported as Dolby Vision, as well as optional support for proposals originating from Philips and Technicolor. Display Daily reports 117 that the BDA recommends that frame average light levels should not exceed 400 cd/m 2 and only specular highlights are allowed to go above 1000 cd/m 2. It is also reported 118 that SMPTE ST , a standard that describes the metadata items to specify the colour volume (the colour primaries, white point, and luminance range) of the display that was used in mastering video content, would be used. 6.8 HDMI Forum The HDMI Forum is a trade association comprised of over 80 companies, including manufacturers of consumer electronics, personal computers, mobile devices, cables and components. The HDMI Forum, and previously, the HDMI Founders, have developed versions of the High-Definition Multimedia Interface, a proprietary audio/video interface for transferring uncompressed video data and compressed or uncompressed digital audio data from an HDMI-compliant source device, such as a set top box, Blu-ray player, camera or tablet, to a compatible digital television, monitor, video projector, or digital audio device 119. Version 2.0 of the HDMI specification was released in September It offers a significant increase in bandwidth, up to 18GBit/s, allowing support of 4K resolution at up to 60 frames per second with 24 bits per pixel 120. This limits the number of bits to 8 bits per sample if 4:4:4 sampling is used, but allows 10 or 12 bits per sample if 4:2:2 or 4:2:0 sampling is used. The signalling of device capabilities and data formats for HDMI is specified in the Consumer Electronics Association (CEA) standard, CEA-861-F, A DTV Profile for Uncompressed High Speed Digital Interfaces 121. Support for UHD resolution, 4:2:0 sampling, and BT.2020 colorimetry was added in the F version of this standard in August : previously only 4:4:4 and 4:2:2 sampling were supported, and resolutions up to 1920x1080. This was extended further in January 2015 with the publication of CEA-861.3, HDR Static Metadata Extensions, which adds support for high dynamic range by extending the metadata definitions. This now allows the signalling of traditional gamma with a maximum luminance of 100cd/m 2, as for standard dynamic range, traditional gamma with a maximum luminance of that of the sink (typically display) device, and SMPTE ST It also allows signalling the SMPTE ST 2086 Mastering Display Metadata, which, similarly to the HEVC mastering display colour volume SEI message, as described IN STRICTEST earlier, allows CONFIDENC the colour volume (the colour primaries, white point, and luminance range) of the mastering display for the video content to be signalled. It also allows signalling of Maximum Content Light Level (MaxCLL) and Maximum Frame-Average Light Level, as defined in an annex to the standard HDMI Forum releases version 2.0 of the HDMI specification CEA Technology and Standards Activities, in SMPTE Motion Imaging Journal, IN September STRICTEST CONFIDENCE Page 52

54 It is stated in the scope section of the standard that it is anticipated that the data IN STRICTEST structures will CONFIDENCE be extended to include additional EOTF and HDR metadata capabilities in future versions. In April 2015, the HDMI Forum announced the completion and release of Version 2.0a of the HDMI Specification 123. This update enables transmission of HDR formats by making reference to CEA The Digital Cinema Initiatives (DCI) In July 2014, the Digital Cinema Initiatives (DCI) issued a statement 124 about three IN STRICTEST topics that they CONFIDENC were currently working on, including new projection light sources, as quoted below. New projection light sources, such as laser, supporting wider colour space and higher dynamic range have emerged that hold the promise of an improved cinematic presentation. DCI is creating additional specifications in order to take advantage of these capabilities, while retaining one of its core objectives: to ensure that distribution IN packages STRICTEST are interoperable CONFIDENCE on all systems. DCI seeks to ensure that next-generation Digital Cinema Packages (DCPs) will be accurately reproduced on any system with this capability. The current DCI specification 125, which references numerous SMPTE documents, specifies the following: The Material exchange Format (MXF) shall be used Resolution is 4096x2160 at 24 frames per second, or 2048x1080 at 24 IN or STRICTEST 48 frames per CONFIDENCE second X Y Z colour space, with gamma of 2.6, and 12 bits per non-linear colour sample Images to be encoded according to ISO :2004/AMD1, i.e. JPEG-2000 The entire image shall be encoded as a single tile Each compressed frame of a 2K distribution shall have exactly 3 tile parts, one for each colour component Each compressed frame of a 4K distribution shall have exactly 6 tile parts, the first 3 representing one 2K colour component, and the next 3 containing additional data to decompress each of the three 4K colour components A 2K decoder shall decode the 2K data from a 4K data stream and ignore the rest Bytes per image constraints limit the bit rate to 200MBit/s for 2K and 250MBit/s for 4K 6.10 The Forum for Advanced Media in Europe (FAME) The Forum for Advanced Media in Europe (FAME) is co-chaired by the Digital Interoperability Forum (DIF) and the European Broadcasting Union (EBU). It provides a venue for device manufacturers, platform operators, software and middleware vendors, applications developers and broadcasters engaged in advanced media, and their relevant industry associations, to share knowledge and experience that will support the successful, industry-led introduction of these services in Europe 126. FAME aims to reach a voluntary consensus on issues concerning interoperability and standardization related to advanced media experiences. Although FAME does not develop standards by itself, it provides a venue to examine the feasibility of establishing guidelines that could encourage the technical availability of content and applications. The first FAME UHDTV Roadmap workshop was held in Lucca, Italy, on 5 and 6 June 2014, and was attended by over 65 delegates, representing professional and consumer industries, network operators, and private and public broadcasters 127. Presentations from broadcasters NHK from Japan (who are focussed on the 2020 Olympics in Tokyo) and Kobeta from South Korea highlighted the possibility that Europe could lag behind in introducing UHD TV IN if significant STRICTEST progress CONFIDEN is not made soon. It was concluded that Europe had a 12 month window of opportunity in which to agree the technical requirements and implementation scenarios for what is called UHD 1 Phase 2. The aim of this second phase of UHDTV development is to offer a comprehensive set of advanced parameters: Higher Frame Rate (HFR), Higher Dynamic Range (HDR), higher resolution (3840x2160 pixels) and an extended colour space. It is regarded as a major UHD TV WORKSHOP, LUCCA FORUM months left to lay out UHDTV 'Roadmap'. Page 53

55 step beyond the HDTV services that the majority of European broadcasters are IN currently STRICTEST transitioning CONFIDENCE to; and an advance on the 4k technology that is already being sold in shops but which concentrates 'only' on an increase in resolution. A further meeting was planned for Wednesday 12 November in Geneva UHD Alliance and UHD Forum A group of organisations led by Harmonic started discussions on UHDTV, taking an end to end viewpoint, in December 2014, using the name Ultra HD Alliance 129. The plan was to discuss with different UHD forums IN STRICTEST during December, CONFIDENC before getting together a core group of organisations during CES in January 2015, and making a public launch at NAB in April It was claimed that the UHD market is confused, being pulled in different directions by different interests, with some standards already existing and others in development. It was claimed that an industry wide focus was needed. The scope of the Alliance was stated to be for the use and promotion of UHD for: contribution and distribution; IN STRICTEST production and CONFIDENCE post-production; playout; distribution on managed networks; OTT distribution for live and VOD; and CPEs. The Alliance was stated to consist of the following working groups: Specifications (to collaborate with standards bodies, and to describe and point to all relevant standards); Interoperability (to define the interoperability points at the system level, to organise plug fests, and to publish the results); Business Model (to publish generic business models); Forecast; Applications; and Promotion. However, during CES in January 2015, another group of companies, led by Samsung, IN STRICTEST announced CONFIDENCE a UHD Alliance 130. Consequently, the former organisation renamed itself the Ultra HD Forum. The UHD Alliance claims to be a global coalition of leading film studios, TV brands, content distributors, post-production and technology companies that aims to create a unified criterion for premium UHD platforms, from devices to content including next generation features like as 4K resolution, High Dynamic Range, Wide Colour Gamut, High Frame Rate and Immersive Audio. The group is reported to be composed of DIRECTV, Dolby Laboratories, LG Electronics Inc., Netflix, Panasonic Corporation, Samsung Electronics Co., Ltd., Sharp Corporation, Sony Visual Product Inc., Technicolor, The Walt Disney Studios, 20th Century Fox and Warner Bros. Entertainment. The UHD Alliance launch press announcement mentions establishing new standards STRICTEST to support CONFIDENCE innovation in video technologies including 4K and higher resolutions, high dynamic range, wider colour gamut and immersive 3D audio, while also suggesting UHD branding by stating that premium Ultra-HD content and devices will be clearly identified so consumers can easily recognize them in store. Thierry Fautier of Harmonic 131 suggests that the UHD Alliance is focussing on a single blocking factor at the moment, that is, the specification of high dynamic range, wide colour gamut and immersive audio, but will have a broader marketing and evangelisation remit. He states that the UHD Forum, a group of about 40 companies led by Harmonic, is considering the end-to-end ecosystem impact. In the longer term he thinks that there is no reason why the two entities might not merge, but currently that it seems most efficient to have the two bodies with the different focuses. He also stated that discussions are on-going to ensure that the two groups (UHD Alliance and UHD Forum) work closely together. In April 2015, just before the NAB (National Association of Broadcasters) Show in Las Vegas, the UHD Alliance issued a call for companies to join the UHD Alliance as contributing members to define the "Next-Generation Entertainment Experience" Shaping the UHDTV roadmap for Europe Ultra HD Alliance. hd_forum.pdf Page 54

56 7 Abbreviations ACES AMOLED ARRIRAW AVC BBC BDA BT BTF CCFL CEA CEATEC CES CIE CPE CRT DASH DCI DCP DICOM DIF DLP DTG DTT DVB DVI EBU EOTF ETSI FAME HD HDCP HDMI HDR HDTV HEVC HFR HVS IEC IRT ISO ITU-R ITU-T JPEG LCD LDR Academy Color Encoding System Active Matrix Organic Light Emitting Diode A data format produced by the ARRI ALEXA camera Advanced Video Coding British Broadcasting Corporation Blu-Ray Disc Association British Telecommunications plc Broadcast Technology Future s Group Cold Cathode Fluorescent Lamp Consumer Electronics Association Combined Exhibition of Advanced Technologies Consumer Electronics Show International Commission on Illumination (or Commission Internationale de l Eclairage) Customer Premises Equipment Cathode Ray Tube Dynamic Adaptive Streaming over HTTP Digital Cinema Initiatives Digital Cinema Package Digital Imaging and Communications in Medicine Digital Interoperability Forum Digital Light Processing (Video Projector) Digital TV Group Digital Terrestrial Television Digital Video Broadcasting Project Digital Visual Interface European Broadcasting Union Electro-Optical Transfer Function European Telecommunications Standards Institute Forum for Advanced Media in Europe High Definition High-bandwidth Digital Content Protection High-Definition Multimedia Interface High Dynamic Range High Definition Television High Efficiency Video Coding Higher Frame Rate Human Visual System International Electrotechnical Commission Institut für Rundfunktechnik International Organization for Standardization International Telecommunication Union Radiocommunication Sector International Telecommunication Union Telecommunication Standardization Sector Joint Photographic Experts Group Liquid Crystal Display Low Dynamic Range Page 55

57 LED MPEG MV-HEVC MXF NAB NHK OCES OETF OLED OpenEXR OTT RGB RVU SD SDTV SEI SHVC SMPTE TIFF TV UHD UHDTV USB VANC VOD VUI WCG YAG Light Emitting Diode Moving Picture Experts Group Multiview HEVC Material Exchange Format National Association of Broadcasters Japan's national public broadcasting organisation Output Color Encoding Specification Opto-Electrical Transfer Function Organic Light Emitting Diode A high dynamic-range image file format Over The Top (content delivery) Red Green Blue Remote Viewing (pronounced "R-view") Standard Definition Standard Definition Television Supplemental Enhancement Information message Scalable HEVC Society of Motion Picture and Television Engineers Tagged Image File Format Television Ultra High Definition Ultra High Definition Television Universal Serial Bus Vertical Ancillary data Video On Demand Video Usability Information Wider Colour Gamut Yttrium Aluminium Garnet Page 56

58 About BT Media & Broadcast BT Media & Broadcast (M&B) is the digital media and broadcast services arm of BT Wholesale. All of the UK s digital terrestrial TV is underpinned by M&B s specialist media network, whilst it is also responsible for delivering outside broadcast services from over 150 sports and news locations around the UK. M&B s global media network links every major location around the world where broadcast or film content is created or distributed. About BT Research and Innovation BT Research and Innovation provides world-class technical expertise to the rest of the BT business. We enable them to create new services and find new ways of doing things to improve efficiency and reduce costs IN STRICTEST for BT. Our open CONFIDENCE innovation model gives us access to thinking from outside BT from small, start-up companies, to some of the best universities around the world. We have teams working with customers, partners and universities in the US, Asia, Europe and the Middle East. BT is one of the largest investors in research and development (R&D) in the UK and, over the last 10 years, is the second largest investor in R&D in the telecoms sector in the world. Since 1990, we have had over 10,000 patents granted and we are active in a number of standards bodies. If you want to reproduce or redistribute any of the material in this publication, you should first get BT s permission in writing. This research paper contains a summary of industry analysis carried out in Q BT cannot guarantee the completeness or accuracy of the information in this research paper and shall not be responsible for errors or inaccuracies. You should not rely on any opinion, research or analysis provided by BT as being a recommendation or advice. Under no circumstances shall BT be liable for any reliance by you on any information in this research paper. Offices worldwide British Telecommunications plc 2015 Registered office: 81 Newgate Street, London EC1A 7AJ Registered in England No: Page 57