This summer’s Olympics in Tokyo have met the challenges of Covid to accelerate broadcast technology.
Tokyo will see many firsts when it hosts the Games of the XXXII Olympiad. Not least that the 2020 Games will be held a year later than planned and less than a year before the 2022 Winter Olympics in Beijing.
The global pandemic also means there will be no international visitors to Tokyo, but Olympic Broadcasting Services (OBS) will build on virtual reality (VR) technology introduced in Rio in 2016 and the 2018 Winter Olympics in PyeongChang to broadcast TV.
giving viewers the feeling of being part of the Games from the comfort of their own home. OBS is the host broadcaster for the Olympic, Youth Olympic and Paralympic Games.
It was founded in 2001 by the IOC. It provides broadcasts for use by all rights broadcasters (eg the BBC, Eurosport) around the world, and it also helps broadcasters prepare for the Games;
OBS oversees the establishment of the International Broadcast Center (IBC), home of the Games’ broadcasting operations. It also prepares the connections at each competition venue, where the production and technical facilities of OBS and the rights holders broadcasters are located and from which international television and radio signals will be produced.
OBS expects to produce 56 live feeds and nearly 9,500 hours of content during the 17-day event, which consists of 339 events across 50 sports disciplines. It has a team of nearly 8,000 people from more than 70 countries, with specialties and skills in various sports broadcasts.
Tokyo will be the first Olympics to be natively produced in ultra-high definition (UHD) – or 4K – and high dynamic range (HDR). UHD content has a resolution of 3840 x 2160 pixels, four times the pixel count of Full HD, to give more detail.
HDR technology improves the contrast between black and white pixels for an accurate image with more tones. Previous Games coverage has been done in parallel coverage with broadcasters, but this year will be the first time the native broadcast for the OBS world feed will be produced directly as UHD and HDR for all competitions and ceremonies of this year’s Games.
Until recently, there was no universal standard for UHD and the adoption of HDR was not a matter of course, said Yiannis Exarchos, chief executive officer of OBS. He particularly values HDR.
“It brings a level of detail, especially in outdoor sports, both in color and light rotation, which really makes the images significantly more realistic than you get in HD; it’s not just about resolution.
” OBS has developed a standard that extracts the best possible HDR output from the UHD solution, making it more efficient and durable because OBS does not have to double the broadcast resources.
The closing ceremony will be broadcast in 8K to take full advantage of the traditionally vibrant spectacles. 8K doubles the number of pixels from 4K to 7,680 x 4,320 and is 16 times larger than HD.
Other sports (athletics, gymnastics, judo and some > < swimming events) will be available in 8K in Japan, although other broadcasters can pick up the feed and experiment with trial broadcasts. There are already plans for 8K broadcasts by Chinese broadcasters for the 2022 Winter Olympics in Beijing.
Another first in Tokyo is immersive audio. All venues will deliver an immersive audio feed as discrete channels in a 5.1.4 format. These are five microphones placed on the left, right and center and on either side or back for surround sound, with a dedicated bass channel and four speakers above the noise source.
There will be 85 separate 5.1.4 audio feeds available to national broadcasters. Immersive audio brings a three-dimensional experience to viewers or listeners. Exarchos says: “I’m a big audio fan because I believe that audio is in a sense the carrier of emotions… so I’m happy that for the first time, in all sports, we will do 5.1.4 immersive audio.
” Countries such as Japan, China, USA, Brazil and most of Europe (via Discovery and Eurosport channels) can experience 5.1.4 audio transmission.
Bringing the 3D quality of the Games to the viewers of events held in arenas, OBS has partnered with Intel to bring its True View technology to Tokyo. TrueView is based on directors using images from virtual cameras around a location to provide perspectives that cannot be seen by physical cameras.
It allows viewers to select the angle from which they want to view the camera, with options for three or six degrees of motion. It uses a series of high-resolution cameras positioned to capture the entire playing field, connected to on-site servers, based on Intel’s Xeon processors.
Data from the cameras is sent to the cloud for synchronization, analysis and processing. In the production suite, engineers can use virtual stationary and tracking cameras to create content that focuses on certain points in the game, perhaps the most exciting action or sequences, for commentator analysis.
Images from the virtual cameras are rendered and converted into compressed digital video in the cloud. Up to 200 terabytes of raw data is processed per event, including height, width, depth, and relative attributes to create high-fidelity 3D video.
TrueView supports the common industry standard video codecs (H.264, H.265, MPEG and AAC for audio) for use on various platforms and devices. The encoded video is converted into bitstreams for live streaming. The bitstreams are converted by the rights holder to decompress the video into a series of images, which are displayed and broadcast sequentially.
The output of volumetric content allows viewers to see all perspectives of the game, whether to follow a particular player or see the game from anywhere on the pitch – including the umpires. In Tokyo, TrueView will be used for basketball games. It’s, Exarchos says, one of the sports where it could be really excellent, because of the three-dimensional nature of the sport.
“It’s also a very fast-paced sport, moving in many different directions and there’s significant vertical movement.” OBS will produce five to seven volumetric replay clips for each game, which will be made available to directors to integrate into live coverage. “I’m pretty sure some of these clips will go viral on social media platforms,” he adds.
Tokyo is the first step for broadcasters to discover how to create exciting content. It could be someone “walking” onto the playing field, turning around and looking at the athletes around him or her, Exarchos suggests. He expects this to develop and is looking forward to what will be broadcast during the 2024 Olympics in Paris.
An added benefit is that True View saves costs and carbon footprint by eliminating the need for a camera crew to travel to the site.
“Covid, travel bans and postponements are challenges that must be seized to move the broadcaster forward.” Yiannis Exarchos, OBS
While Exarchos is sworn to secrecy and cannot reveal details of the opening ceremony, he can confirm that it will be broadcast with multiple cameras for an immersive virtual reality experience. VR is not new to the Olympics;
it was used in 2016 in Rio and during the 2018 Winter Games in South Korea. For the opening ceremony there are six 180° camera systems and one 360° VR system.
The ceremony will also include experimentation with 5G technology to ensure that content is quickly received at high resolutions and flipped for global audiences. “I believe that 5G can be a big game changer for broadcasting,” says Exarchos. “It can help us get rid of a lot of the technical limitations, equipment and regulatory needs, as well as licensing RF transmissions…
We need to find a way that we can use all these interesting technologies in a way that is as effective as we do.” with the tools we have now.” VR has massive bandwidth, and 5G’s reduced latency could solve syncing issues around live events. “5G provides ample bandwidth and also largely beats the digital issues of latency,” he continues.
There will also be opportunities to progress using mobile phone screens and the computing capabilities; “hence our partnership with Intel,” says Exarchos. The VR experience will be “significantly improved” compared to that of PyeongChang, covering 47 live events and between 50 and 100 VR highlights.
“We’ve noticed that people tend to experience highlights on VR,” says Exarchos. “For Beijing, our common goal, with Intel, is to produce an 8K VR product.” The 2018 Winter Games introduced Intel True VR, with OBS broadcasting 30 events featuring live and video-on-demand VR events.
There will be three to six camera angles recording each event for content compatible with most commercially available VR headsets. Viewers can watch an immersive ‘VR Cast’ seen from different angles, as well as graphics and picture-in-picture overlays. True VR can also be used for post-event highlights, and production teams can overlay stats or picture-in-picture content.
The quest for stats by viewers has also led to Intel’s 3D Athlete Tracking (3DAT) being used at the Games this year. Originally a coaching tool, it uses AI to identify 20 skeletal points on an athlete. When reenacting the race, these indicate points where pressure or stress is applied or to analyze moments of acceleration and deceleration.
In Tokyo, it will be used after the races for viewers to analyze the race in detail. Intel’s director of sports performance technology, Jonathan Lee, explains how much data processing is involved. “We’ll be tracking all eight or nine sprinters, so that’s nine videos at 60 frames per second for 10 to 12 seconds.
It’s not just detecting athletes, but tracking skeleton points… We’re using AI because the athletes are next to so we can’t see the sprinter’s arm in lane five, for example,” he says. . AI is used to distinguish which sprinter is which at any given time.
Data of interest to an athlete and coach may be too detailed for a viewer, but stats such as top speed, when a sprinter reaches that top speed, and when they slow down are likely to be shown. With 3DAT, athletes and coaches can judge a race and see that they maintain their top speed for example 20 or 30 meters and then start to slow down at 70 meters.
“To be honest, to be able to process all that information quickly, you see color indicators, almost like a heat map,” Lee says. The classic image of the sprinters running to the finish line is embellished with colors that indicate their speed, from yellow to red and darker as they go faster, explains Lee.
This in itself is a visual representation, but post-production teams can overlap when a particular sprinter reaches their top speed and what speed that is. “We will work with broadcasters to understand how they use this kind of content…
maybe coaches and athletes find some things interesting, we might find fans find them attractive too, and we could start weaving them too,” suggests Leen The technology is designed to complement coverage from rights broadcasters.
The 3DAT clips with overlapping “heatmaps” can be used by commentators to investigate what happened. Intel’s graphics partner receives the video from OBS, takes the 3DAT data and renders the clips AI processing is done in the cloud on scalable Xeon processors It is flipped in less than 60 seconds.
To be able to have volumetric replay, possibly also with analysis, and to be able to use it as a fourth or fifth replay after a race is a big problem,” says Exarchos. “I’m sure it will get faster and faster… and hopefully we’ll do more in Beijing.”
How to be there, without being there
The Tokyo Games are expected to be popular around the world as people come out of a pandemic hibernation. Exarchos expects viewing figures to exceed Rio’s 3.3 billion worldwide. (The 2012 London Olympics were watched by 3.6 billion people worldwide, only the 4.4 billion viewers of the 2008 Beijing Games).
The absence of foreign visitors to Tokyo will create a different experience for the viewers, as much of the Olympic atmosphere is derived from the audience’s reactions. Flying the flag and the crowd roaring for their favorite is part of many events, and OBS has plans to let spectators and fans show their virtual support.
It provides the rights holders with digital tools that allow end users to encourage athletes or countries or the Games in general. This cheer is shown in the halls. This is a last-minute development, Exarchos admits. “We have decided to accelerate it, given the situation in Tokyo,” he says.
This outside engagement of fans will be a virtual presence – they will be asked to cheer or post a selfie to celebrate a win. “It’s not just passive spectators, they will be virtually present in the field,” he explains. “We have made a commitment to deliver the same Games [regardless of the Covid pandemic].
We provide exactly the same coverage. Since the [initial] postponement of the Games, we’ve been thinking about ways to handle the situation. You never want to miss a challenge,” he adds.
Tokyo’s innovation team is planning a VR booth at a location outside of Tokyo, allowing fans to wear VR headsets and take part in the Games virtually using 360° projections and in collaboration with local production companies. There are similar plans in China; however, all are subject to permission from the local broadcasters.