8K Live Television Coverage of Global Sports Events in Brazil

Transcription

1 8K Live Television Coverage of Global Sports Events in Brazil Michael Stanton 1, Leandro Ciuffo 1, Shinichi Sakaida 2, Tatsuya Fujii 3, Hiroyuki Kimiyama 3, Junichi Nakagawa 4, Hisao Uose 5 1 Rede Nacional de Ensino e Pesquisa (RNP), Brazil 2 Science and Technology Research Laboratories, Japan Broadcasting Corporation (NHK), Japan 3 Network Innovation Laboratories, Nippon Telegraph and Telephone Corporation (NTT), Japan 4 Service Evolution Laboratories, Nippon Telegraph and Telephone Corporation (NTT), Japan 5 NTT Advanced Technology Corporation (NTT-AT), Japan s: michael@rnp.br leandro.ciuffo@rnp.br sakaida.s-gq@nhk.or.jp fujii.tatsuya@lab.ntt.co.jp kimiyama.hiroyuki@lab.ntt.co.jp nakagawa.junichi@lab.ntt.co.jp hisao.uose@jeans.ocn.ne.jp Paper type Case study Abstract R&E networks played an important role in helping Japanese television to transmit live 8K resolution TV images of the 2014 FIFA World Cup TM to Japan. The huge distance between Brazil and Japan, half a world apart, set new challenges for streaming digital video spanning multiple domains over long-distance networks. The project to stream the FIFA World Cup TM in 8K was a joint collaboration between NHK (Japanese Public Broadcasting Corporation), TV Globo (Brazilian TV broadcaster), NTT (Japanese Telecommunications Corporation) and RNP (Brazilian NREN), in cooperation with FIFA and CBPF (Brazilian Centre for Physics Research). The paper describes the audio and video technologies used in 2014, as well as the extensive preparations for efficient and effectively lossless media transmission over the intermediate networks. The high-compression H.264 codec developed by NHK required 280 Mbps for compressed SHV media transmission. In addition, the transmitted media were encrypted (for security), and then protected against packet loss by use of FireFort-LDGM FEC, a new forward error correction (FEC) code based on lowdensity generator matrix (LDGM), developed by NTT. The scheme used in 2014 provided for complete media error recovery, even after the loss of up to 20,000 consecutive UDP packets, and required a total bandwidth of less than 360 Mbps. Keywords 8K video, Super-Hi Vision, video capture, lossless media transmission, FIFA World Cup TM 1. Introduction Whilst for most observers the next generation of Ultra-High Definition (UHD) television is generally thought to use the display resolution of 3840H x 2160V, usually known as 4K, the Japan Broadcasting Corporation (NHK) has already been demonstrating use of their Super-Hi Vision (SHV) media, using the four times higher display resolution of 7680H x 4320V, usually referred to as 8K. These demonstrations to show the benefits of SHV have included the visualisation of cultural and scientific content from several countries, as well as globally known sporting events such as the 2012 Summer Olympic Games in London and the 2014 FIFA World Cup TM in Brazil. In these latter two cases, NHK captured and edited audio and 8K video content and transmitted the resulting SHV media for visualisation at public viewing sites both in the host country and in Japan. This transmission was carried out through a collaboration with the academic research networks in and between intermediate countries. In 2014, the Brazilian (RNP) and Latin American (RedCLARA) networks were used to reach Internet2. In this case, the total distance to Japan was greater, around 23,000 km endto-end, more than half the circumference of the Earth. The whole system functioned in real time, with the public viewing sites showing live coverage of the football games from Brazil. Altogether, nine matches were selected by NHK for live streaming in 8K, starting with the first Japan game (Japan vs. Ivory Coast in Recife on June 14th) and concluding with the final between Argentina and Germany in Rio de Janeiro on July 13th. The selection of games took into consideration the logistics of moving the NHK outside broadcast (OB) vehicles containing the 8K capture and editing equipment between the hosting cities in Brazil. Table 1 displays the games that were streamed live, and Figure 1 shows the

2 location of the football stadia. Table 1. FIFA World Cup TM matches live streamed in 8K Date Match Venue 14/June Ivory Coast 2-1 Japan Recife 16/June Ghana 1-2 USA Natal 19/June Japan 0-0 Greece Natal 23/June Cameroon 1-4 Brazil Brasilia 28/June Brazil 1-1 Chile Belo Horizonte 30/June France 2-0 Nigeria Brasilia 05/July Argentina 1-0 Belgium Brasilia 08/July Brazil 1-7 Germany Belo Horizonte 13/July Germany 1-0 Argentina Rio de Janeiro Figure 1: Location of the 2014 FIFA World Cup TM venues (Available at The remainder of this paper is organised as follows: Section 2 briefly presents some technical details of the SHV architecture. Section 3 illustrates the application of SHV at 2014 FIFA World Cup TM, explaining the network challenges. Finally, Section 4 concludes the paper. 2. Super Hi-Vision The next-generation broadcasting system developed by NHK, named Super Hi-Vision (SHV), reproduces a strong sensation of presence through the use of ultra-high-resolution 33-megapixel 8K video (7680 pixels horizontally by 4320 pixels vertically) and maximum 120-Hz frame rate with progressive scanning, and 22.2 multichannel audio (22.2ch audio), providing 3-D surround sound. The final goal of research and development of SHV is to deliver highly realistic image and sound to viewers homes. When SHV becomes applicable as a broadcasting system, we will be able to use it for many purposes, such as archival and medical use. NHK has developed SHV cameras, displays, recorders, and audio equipment. The first 8K SHV camera was developed in 2002, utilizing four 8 megapixel sensors, its frame rate was 60 Hz, and its weight was

3 around 80kg. 8K image signal captured with this camera has a bit rate of 24Gbps. In 2014, an 8K camera weighs 20kg. NHK has also developed smaller 8K cameras, weighing 5kg and 2kg with image quality almost equal to that of a camera with four 8 megapixel sensors. On the other hand, NHK developed cameras to improve image quality. One is a 33 megapixel (full resolution) camera. An 8K image signal captured with this camera has a bit rate of 72Gbps. Another uses a 120Hz frame rate. NHK also developed a high sensitivity and low noise camera for shooting objects in a theatre, such as for concerts, opera, ballet, plays and so on, where unwanted noise might interfere with the performance. NHK has also developed 8K image display equipment. One is a projector, usually utilized for Public Viewing (PV) events. The others are display panels, including an 85 inch LCD display and a 145 inch PDP (plasma) display. For audio capture, a spherical one point microphone for capturing 22.2ch audio has been developed. This makes it easy to capture directional ambient sounds, using 24 microphone units distributing in 3 layers in the sphere. A 22.2ch audio mixing desk was also developed, which includes 3D panning software. As it is sometimes difficult to monitor 22.2ch sound outdoors or in limited spaces, NHK developed a monitoring system which has the capability to hear 22.2ch sound by headphone. This utilises a binaural technique (using a head-related transfer function which is derived from transfer functions between loudspeaker and ear). Several SHV programmes have been produced using these devices, and demonstrations of the programmes have attracted many visitors at events such as the 2005 World Exposition in Aichi, Japan, where SHV was first demonstrated on a 600 inch screen, as well as NAB (National Association of Broadcasters) in Las Vegas, USA and IBC (International Broadcast Conference) in Amsterdam, Netherlands, etc. NHK is now preparing for the experimental broadcasts of 8K-SHV in 2016 and actual broadcasting in 2018 in Japan. 8K-SHV video should be compressed with high image quality and low bit-rate for real-time transmission using narrow bandwidth media. To achieve this, a new high-efficiency compression coding system is necessary. Therefore, NHK has developed 8K with 60Hz frame rate codec systems that are based on MPEG-4 AVC (Advanced Video Coding)/H.264 video coding standards. At a live transmission, such as a big sports event, stability is the most important issue, so the codec system is implemented as hardware devices. Since this kind of event involves one-way distribution, system delay is not so serious a problem. The same video without compression would require almost 24 Gbps to be streamed. The current state of the art of digital television media prevents 8K signals from being transmitted over long distances. This is why this project depended on the technological support of NTT Innovation Labs (Nippon Telegraph and Telephone Company), RNP (Brazil s R&E network) and other research networks in Latin America, the USA and Japan. 3. Application of SHV during the 2014 FIFA World Cup TM 3.1 Media capture The signal of the nine matches were generated by three 8K cameras, two high-speed 4K cameras and one 4K camera for location shooting (upscaled from 4K to 8K). The signals from the high-speed cameras were recorded on video servers and played back in slow motion. The 8K outside broadcast van travelled together with an audio truck for the recording of 22.2ch surround sound on a console, a transmission truck and an equipment truck. The total distance of the itinerary followed by the four trucks used was more than 4,200 km. 3.2 Transmission As shown in Figure 2, all matches were streamed first to the FIFA s International Broadcast Centre (IBC), regardless of the location of the stadium. The IBC was located in the Riocentro a big conference and exhibition centre in Rio de Janeiro and the FIFA communications network that interconnected all stadia to the IBC was provided by the Brazilian telecommunications company, Telebras.

4 Figure 2: Schematic network diagram in Brazil From the IBC, the 8K signal was streamed to the RNP Point of Presence (PoP) in Rio de Janeiro, using RNP s own metropolitan network, Redecomep. From there, it was streamed to Tokyo using four of the five redundant international routes that had been configured, as shown in the map (Figure 3). The five routes consisted of heterogeneous networks operated by both academic and commercial providers shown below. Route 1: Rio São Paulo Miami Seattle Tokyo (via RNP, Internet2, NTT GEMnet2) Route 2: Rio São Paulo Miami Seattle Tokyo (via RNP, RedCLARA, SINET4, NTT GEMnet2) Route 3: Rio São Paulo Seattle Tokyo (via RNP, NTT VLink, NTT GEMnet2) Route 4: Rio Fortaleza Miami Seattle Tokyo (via RNP, Internet2, NTT GEMnet2) Route 5: Rio Fortaleza Miami New York Tokyo (via RNP, SINET4, NTT GEMnet2)

5 Figure 3: International transmission paths It is important to utilise the full capability of these redundant paths which are not completely independent from each other in terms of statistical congestions and disruptions. Each path was paired with another having the lowest correlation. in order to create a robust transmission link for Transport Streams as shown in Figure 4. Figure 4: Three logical TS transmission links To create a secure communications path suitable for 8K video transmission over multiple IP networks, NTT combined the LDGM-FEC forward error correction codes together with the multi-path transmission scheme. As mentioned in the next section, this combination of added redundancies in both the time and space domains greatly enhances the reliability of this very long distance transmission at a relatively small cost.

6 3.3 Application Level Secure Transmission Technology The H.264 codec developed by NHK was able to provide compression rates of over 100:1, requiring around 280 Mbps for compressed 8K/SHV media transmission. This low bandwidth demand made it possible to use shared IP networks for end-to-end flows, rather than requiring dedicated L2 circuits. In addition, the transmitted media were encrypted (for security), and then protected against packet loss in the shared networks by use of FireFort-LDGM FEC, a special forward error correction (FEC) code based on lowdensity generator matrix (LDGM), developed by NTT. For the 8K public viewing, we selected UDP-based streaming, that was suitable for transmitting 8K videos to many locations. IP packet ARQ (Automatic Repeat Request), like TCP, is another way to carry out reliable data transfer without redundant transmission, but in this study we focused on the availability of UDP-based streaming subject to the large network delays expected with international streaming. The measured delay-time was about 160 msec from Brazil to Japan. The scheme used in 2014 provided complete media error recovery, without any IP packet loss even after the loss of up to 20,000 consecutive UDP packets. The burst loss of 20,000 packets corresponds to the disruption of about 600 msec for 360 Mbps streams. This tolerance was gained by using the FEC system parameters of a blocksize of IP packets with 20% redundancy. Furthermore, in the case that a single flow could not be protected by FEC, a parallel transmission on a redundant path provided a second level of protection. In all, five alternative paths were made available to provide a choice of two paths during any given transmission (see below). This combination was designed because the more robust FEC requires a larger transmission delay to calculate the redundant data, but the above-mentioned FEC parameter already caused the significantly large delay time of more than 15 seconds. Eventually, the combination of the FEC and the parallel streaming was quite effective in this study. During the entire experiment of over 40 days, we could continuously protect the 8K streams against long-term disruption caused by the malfunctions of network nodes or scheduled maintenances in every network domains. As shown in Figure 5, the FEC function, AES 128-bit encryption/decryption and the multipath transmission function were implemented into a plain, small-scale PC system, called the Secure IP terminator. This system was designed so that it could be applied to every UDP/RTP-based video transmission system, just by inserting transmit and receive systems at the edges of the networks. Figure 5: Block diagram of secure IP transmission system by FireFort-LDGM FEC 3.4 Operation and Monitoring Because we used a combination of heterogeneous network resources it was essential to monitor the conditions of all network elements closely during the transmission. For locating network failure and quality degradation we deployed a number of network testers not only at the source and destination of the traffic, but also within the network, especially at the borders (interconnect points) of the different networks. By applying active probing traffic streams in a mesh manner connecting all those network testers, we could identify the malfunctioning elements in the network and isolate the problem in a short period of time. This monitoring system was created using the Openview and Nagios platforms (Figure 6).

7 Figure 6: Network monitoring during the public viewings Figure 7: Public Viewing (PV) sites at CBPF, Rio de Janeiro, Brazil (left) and Osaka, Japan (right). 3.5 Viewing Sites The games in 8K were streamed live to 7 viewing sites, 4 in Japan, in the cities of Tokyo, Yokohama, Osaka and Tokushima, and 3 in Rio de Janeiro, at the IBC, the Sofitel hotel (FIFA s main hotel) and the auditorium of the Brazilian Centre for Physics Research (CBPF). Two of these sites are shown in Figure 7. The CBPF site hosted viewing sessions organized in cooperation with the Brazilian broadcaster TV Globo. Selected audiences of students, researchers, professors, authorities and representatives from press and industry were invited to attend both pre-recorded and live sessions. In addition to the 8K video, the invited audiences enjoyed the Super Hi-Vision s 22.2 channel 3D sound system. In order to fill the intervals between the live transmissions caused by the physical relocation of the outside broadcast vehicles between the stadia, 30 non-live public screening sessions were organized to demonstrate SHV technology. Each session, planned to last 30 minutes, displayed a selection of assorted SHV contents, including the Rio de Janeiro carnival parades, a Japanese fashion show and FIFA World CupTM highlights. Altogether, more than 800 viewers attended the technological demonstrations at the CBPF venue in Rio. For Japan sites, at the NHK broadcasting centre at Shibuya, Tokyo, the IP streams were received from Rio de Janeiro and retransmitted to four cities. Two of the received streams were decoded at NHK and sent to the Tokyo (Toyosu) and Yokohama sites as 24Gbps signals using DWDM (Dense Wavelength Division Multiplex) system with eight 3Gbps SDI waves over dark fibre, since the distances from Shibuya to Toyosu and Yokohama are sufficiently small. For Toyosu, since the distance is 21km, we did not use any optical amplifiers. On the other hand, for Yokohama, the distance is over 120km long, so we inserted several optical amplifiers among the route to prevent any losses. For Osaka and Tokushima sites, the distance are very long and we transmitted the compressed TS signals as IP streams using two redundant routes. None

8 of the transmissions had any errors during live relay. A total of 9022 people visited the four PV sites (1918 for live events), and the public viewing was successful providing the viewers a sense of reality and immersive sensation. 4. Conclusion SHV has been developed with presence as its strongest feature. The experience of the transmissions from Brazil in 2014 have once again shown that an extremely strong sense of presence could be delivered by SHV video and audio and that high levels of emotion could be imparted to viewers, giving them a sense that they were actually present in the stadium during a FIFA World Cup TM game. The production style, without voice-overs (announcements or commentary) and mainly using wide camera angles and long (slow) cut ratios, received many comments of surprise and admiration of the new possibilities it presents for TV broadcasting. To conclude this paper, we summarize some data from the SHV demonstrations in 2014: 9 matches streamed live in 8K to 7 viewing sites in Japan and Rio 275 inches (6,30m x 3,63m) was the size of the screen in the main PV site in Rio 22.2 audio channels distributed in 33 loudspeakers 5 redundant network routes configured between Rio and Tokyo to stream the video flows 360 Mb/s in IP with FEC (280 Mb/s in TS) was the required bandwidth 24 Gb/s would be necessary to stream uncompressed 8K images at 60fps 8K resolution (7.680 x pixels) is 16 times Full-HD, or 4 times 4K References [1] Shishikui, Y. et al., High-Performance Video Codec for Super Hi-Vision. Proc IEEE, vol. 101, no. 1, p [2] Fujii, T. et al., Digital Cinema and Super-High-Definition Content Distribution on Optical High-Speed Networks: Proceedings of IEEE, vol. 101, no. 1, p [3] Tonomura, Y. et al., Layered LDGM Codes for Scalable Video Streaming over Packet Erasure Channels: 2008 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS2008), Swissôtel Le Concorde, Bangkok, Thailand, [4] Shimomura, M. et al. GEMnet2 R&D Testbed Network. NTT Technical Review, Vol. 11 No. 1, p. 1 [5] Stanton, M. et al., RNP: A brief look at the Brazilian NREN: In: TNC Living the network life, Selected Papers - ISBN [Accessed 28 November 2014] Biographies Michael Stanton is Director of Research and Development at RNP, the Brazilian NREN. After a PhD in mathematics at Cambridge University in 1971, he has taught at several universities in Brazil, most recently as professor of computer networking at the Federal Fluminense University (UFF) in Niterói, Rio de Janeiro state. Between 1986 and 1993, he helped to kick-start research and education networking in Brazil, including the setting-up and running of both a regional network in Rio de Janeiro state (Rede-Rio) and RNP. He returned to RNP in 2001, with responsibility for R&D and RNP involvement in new networking and large-scale collaboration projects. Leandro N. Ciuffo is a Manager in the Directorate of R&D at RNP, in charge of interacting with cultural

9 and scientific communities concerning new approaches to advanced network use. Since 2011 he also coordinates the RNP's R&D Programme on Advanced Applications for Remote Visualization, which includes streaming of ultra-high-definition media. From 2006 to 2009 he worked in Grid Computing and e-science projects at INFN-Catania (Italy), being responsible for dissemination and user support activities. Leandro holds a M.Sc. in Computer Science from the Federal Fluminense University (UFF), in Brazil. Shinichi Sakaida is a Senior Research Engineer of the Advanced Television Systems Research Division at the Japan Broadcasting Corporation (NHK) Tokyo, Japan. He received the B.S., M.S., and Ph.D. degrees in electrical engineering from Waseda University, Tokyo, Japan, in 1989, 1991, and 2005, respectively. He joined NHK in 1991, where he has been with the Science and Technology Research Laboratories. He is engaged in the research of video coding of UHDTV system, and super-resolution and reconstruction of compressed video signals. Dr. Sakaida has been actively involved in standardisation works at the ISO/IEC Moving Picture Experts Group (MPEG). Tatsuya Fujii is a Senior Research Engineer, and Supervisor at NTT Network Innovation Laboratories. He received the B.S., M.S., and Ph.D. degrees in electrical engineering from the University of Tokyo, Tokyo, Japan, in 1986, 1988, and 1991, respectively. He joined Nippon Telegraph and Telephone Corporation Japan (NTT), Kanagawa, Japan, in He has been researching parallel image processing and superhigh-definition (SHD) image communication networks. In 1996, he was a visiting researcher of the Washington University in St. Louis. St. Louis, MO. He was a director of digital cinema developing project, and is currently a group leader of the media processing systems research group in NTT Labs. He is a member of IEEE, IEICE of Japan and ITE of Japan. Hiroyuki Kimiyama is a Senior Research Engineer of NTT Network Innovation Laboratories. He received his B.E and M.E degrees from Tohoku University, Japan in 1988 and 1990 respectively. He received his Ph.D. degree from the University of Electro-Communications in He joined the Nippon Telegraph and Telephone (NTT) Corporation in Currently, he is studying distributed and parallel processing technologies for ultra-high-quality video handling system and storage server system over IP networks and SDN. He is a member of ACM, and the Institute of Electronics, Information and Communication Engineers (IEICE) of Japan, and the Information Processing Society (IPSJ) of Japan. Hisao Uose is Executive Manager at Network Solutions Business Headquarters, NTT Advanced Technology Corporation. He received the B.E. and M.E. degrees in electrical engineering from Kumamoto University in 1978 and 1980, respectively. He joined Nippon Telegraph and Telephone Public Corporation (now NTT) in He was a guest associate professor and a guest professor at the National Center for Science Information Systems during and , respectively. He was in charge of operating GEMnet2 at NTT Service Evolution Laboratories since its inauguration until the end of March, Junichi Nakagawa is a Senior Research Engineer, and Supervisor of NTT Service Evolution Laboratories. He received B.E. and M.E. degrees in mechanical engineering from Waseda University in 1988 and 1990 respectively. He joined NTT in He is currently in charge of managing GEMnet2, NTT s R&D testbed network, and XFARM, NTT s R&D cloud platform. He is a member of IPSJ and IEEE.