Live At-Home Production 2.0

Live At-Home Production 2.0

klaus.weber@grassvalley.com, larissa.goerner@netinsight.net, peter.walker@calrec.com

Abstract – Tasked with producing more live coverage with squeezed resources, broadcasters are turning to at-home/REMI (remote-integration model) production. Correctly implemented, at-home production can reduce the movement of people and equipment; increase the utilization of equipment; reduce on-site set-up times; and maximize the efficiency of production teams. Historically there have been three major challenges with at-home production; Latency and how to mitigate it; Control and how to extend workflows to the venue; Infrastructure and how raw signals are transported back to base. This paper will clearly explain how these issues are now solved by interconnecting and orchestrating distributed production resources over Wide Area Networks. Solutions partners Net Insight, Calrec and Grass Valley will explain in this paper how a shared technology approach is already providing broadcasters with a complete, proven and easy way to generate significantly more live content.

IP studio transition is the number one technology topic in the broadcast industry, but it’s full potential can only be enabled by creating new workflows, like at-home production and extending the studios over a Wide Area Network.

The concept of at-home production – or remote, REMI or centralized productions - has been well-developed for several years, and there have been several implementations of long-distance transmission of raw feeds. However, these implementations have all been using existing technology which has been adapted to solve the current challenges. Now the market must move one step further and improve the implementations of at-home productions.

The new workflow has been created for several reasons, but is mainly driven by the need to accomplish more with less, although the initial drivers alter depending on the region.

Producing more live content is one of the major drivers as multi-platform delivery demands an increase in content production. At-home production enables the same staff to produce more programs per day, and enables production companies to use the best available freelancers in the market for various consecutive productions. At large events, like the Olympic Games or the FIFA World Cup, at-home production can increase the amount of content with fewer technical staff at the event – maximizing output while minimizing costs.

advertisment

Having people working centrally and only sending a small, fast vehicle to provide a significantly small load of equipment to the arena, enables substantial cost savings and prevents empty runs of production equipment, while providing the at-home based production crew access to central archives and an established studio production environment. On-air talent can do more than one production, and the best operators for replay, vision mix and audio can be deployed to create high-quality content.

Those advantages are widely acknowledged across the industry, but what has been lacking was a focus to bring the at-home workflow to the next level by providing innovative audio-visual, networking and transport solutions; we call this at-home production 2.0.

The following sections will describe the major challenges and state of the art solutions for enabling better video, audio and transport workflows for at-home production.

Camera Signal Transmission

I.Status overview

Live At-Home remote productions have been a topic in the broadcast industry for some time now [1] and as explained before, this is mainly driven by the requirement for producing more content with the same or even less budget.

advertisment

In typical OB productions, a large amount of very expensive equipment is “on the road” for most of the time and can only be used for a very limited time producing content. At the same time, a large production team needs to travel following the equipment to the same places. Being able to keep most of the equipment and operational staff in-house and only have a minimum amount of equipment and operational staff on-site will offer a much more efficient way to produce high-quality content.

Unfortunately, with most of the remote production solutions available today all the different input and output signals from the camera base stations are transmitted from the production site to the studio building. These solutions require full camera chains (see Figure 1) at the production site and do not offer any improvements in the utilization of camera equipment.

Live At-Home Production 2.0

FIGURE 1: TYPICAL CAMERA CHAIN

advertisment

In addition, a great deal of dedicated and expensive hardware is required for converting, multiplexing and modulating the multiple video, audio and control signals onto IP interfaces. Add to that the set-up, management and control of these solutions is very complex and time consuming — this can lead to unpredictable errors.

Making Live At-Home remote productions a more flexible and cost-efficient alternative to traditional OB productions needs to be achieved. But in addition, it needs to offer a workflow where a reduced number of staff is required to travel.     

II.Camera IP transmission solutions

When IP productions are mentioned, most think only about an IP studio infrastructure, but those IP infrastructures can be used for the next-generation of Live At-Home remote productions as well. With the availability of today’s larger bandwidth IP networks, it’s becoming easier to produce live programming efficiently with multiple cameras across those networks. 

Most system cameras use bidirectional fiber transmission to interconnect the camera head and the camera base stations (see Figure 2).  

Live At-Home Production 2.0

FIGURE 2: IP BASED CAMERA SIGNAL TRANSMISSION

advertisment

Some of the latest production cameras use a transmission protocol based on standard 10GbE technology (see Figure 3) and can be connected directly to commercial-off-the-shelf (COTS) switches on an IP network, transporting the full protocol of each camera through an IP network to the camera base stations connected to another COTS switch on the same IP network.

Live At-Home Production 2.0

FIGURE 3: XF TRANSMISSION PROTOCOL

advertisment

By facilitating the complete transmission protocol between the camera heads and camera base stations over IP networks, remote productions can be more easily realized — without any trade off in image quality, signal latency or transmission stability.

First installations using this technology, called Direct IP, are in use at different locations around the world (see Figure 4) and have proven that they offer an alternative workflow with more flexibility and better cost efficiency.   

Live At-Home Production 2.0

FIGURE 4: DIRECT IP REMOTE PRODUCTION OVER IP NETWORKS

advertisment

But unfortunately, bandwidth requirements for a fully uncompressed and bidirectional signal transmission are quite demanding. The IP bandwidth required for each camera system can be between approximately 2 GB and

9 GB, depending mainly on the video format and frame rate of the cameras. In many cases, the IP infrastructures available today do not offer the bandwidth as required for a fully uncompressed multicamera production. 

 

III.Use of video compression

Since the IP bandwidth available today does not always offer the capability of handling the uncompressed camera transmission protocol, some form of compression needs to be applied. Regular short distance point-to-point camera transmission systems over fiber cables have no practical bandwidth limitations. Therefore, they do not offer a selectable compression system to reduce the bandwidth, so that an external compressing system such as built into the Nimbra system from Net Insight needs to be used (see Figure 5).   

Live At-Home Production 2.0

FIGURE 5: XF TRANSMISSION WITH J2K COMPRESSION APPLIED

advertisment

For the camera transmission protocol, the largest bandwidth is required for the video content and there are multiple video streams between the camera and the base station. If industry standard wrappers such as SMPTE ST 2022-6 are used, the video content can be easily extracted and different kinds of compression can be applied.

If JPEG 2000 is used, typically the bandwidth requirements for visibly lossless signal transmission is only 10% of an uncompressed transmission. Nevertheless, an even higher compression rate can be applied if the IP bandwidth available requires it, but slightly reduced performance would need to be accepted in these cases.

But if the image quality is not affected at only 10% of the bandwidth, why should compression not be used in all cases? Compression introduces latency to the signal transmission and that can be critical in some cases and needs to be taken into consideration.

First, the audio signals embedded into the camera transmission protocol need to be delayed the same amount as the video signals. But in addition, all the other audio sources from the production site need to be synchronized with the camera audio and video signals. 

For the camera shader working at the centralized control room, any latency of the signal on the shading monitors is a challenge. This is especially the case for exposure control, so latency should be kept as small as possible. Experiences in the past, especially with wireless cameras, show us that the transmission latency of around 100 millisecond (msec), as is typically generated by JPEG 2000 compression, is still manageable for most applications. But the additional network latency needs to be taken into account, which might increase the overall latency to 150 msec or even more.

There are applications such as the production of fast-action sports or productions under very demanding lighting conditions, where the total transmission latency will not allow for acceptable camera shading with Live At-Home remote productions.

advertisment

IV.Summary Camera Signal Transmission

Live At-Home remote productions offer multiple benefits compared to traditional OB productions. They include much improved flexibility and better utilization of equipment, but also include reduced cost for traveling and more efficient use of the operational staff. In addition, highlight editing, archiving replays, high-speed camera recordings, etc., are much easier to manage if all the camera signals from multiple production sites are available in real-time at one central location.

The introduction of compression to the video streams of the camera transmission system allow using Live At-Home remote productions at many more locations. In combination with uncompressed Direct IP remote applications, they offer, in many cases, an attractive alternative to traditional OB productions.      

At-Home Production Audio

  1. Monitoring / IFBs

When remotely controlling a broadcast production over a long distance, the audio portion poses an additional challenge; announcers need to be able to hear themselves, and often their co-announcers and guests, and sometimes other ambient sounds in what they perceive to be real time. Too much latency in the monitoring signal path will cause them to hear echoes or delays that can make it very difficult to converse.

Content that is being fed to you that you can also hear naturally needs to have a low-latency signal path. Most critical is a person’s own voice. In order to hear yourself live in monitoring - and not be put off by it - the sound needs to incur less than 5ms from being spoken to being heard. For this reason, local sources in the monitor mix cannot incur the delay imposed by them being mixed at a broadcast facility and returned to the venue over a long-haul IP connection.

In some cases, such as an interview with a guest at a remote site, the problem is solved by simply not mixing the guests voice into their monitoring, but this is not acceptable for most on-air talent. Use of side-tone is also an option to feed a person their own-voice in real-time, but for an announcer to perform well, they should be able to hear themselves how they sound on air, with EQ and dynamics applied. It is for this reason that practical audio workflows for At-Home-Production have been lagging behind those of video.

Sounds other than your own voice can be a little more forgiving of latency; with headsets or in-ear monitors, much of it can be filtered out, and what sound you do hear already incurs some delay of its own in getting from the source to your ear, which will be offset if that sound in the monitoring is picked up from a mic much closer to the source. The human brain is also more forgiving in terms of latency for lip-sync of people you are talking with, but to keep dialogue natural and free flowing between presenters that cannot hear each other without monitoring, it makes sense to minimize the latency of those feeds as well.

The solution to “real-time” audio monitoring for At-Home-Production is to handle the monitor mix locally, at the venue (see Figure 6):

Live At-Home Production 2.0

FIGURE 6: LOCAL MIXER FEEDS THE BROADCAST CHAIN AND PROVIDES REAL TIME MONITOR FEEDS WITH A MIX MINUS RETURN FROM THE BROADCAST FACITLY

  1. Remote Control

The need for a local monitor mix has led to at-home style productions still requiring an audio operator at the event, but many modern audio mixing products offer remote control via IP, meaning the monitor mix can be set up and controlled remotely from the broadcast center. Some of these products are compact and cost-effective, yet they are true broadcast mixers, with assistive features that can reduce the operational burden such as auto-mixers, as well as providing robust hardware redundancy. Having the monitor mix as part of the program chain, rather than splitting mics off into a standalone monitor mixer, means one person can control all of the mixes, and the on-air talent get to hear themselves exactly as they sound on air.

A lesser skilled/local A2 is needed just to plug in the mics and headsets. Mic gains, fader levels, signal routing and much more can all be controlled remotely.

Being able to control an audio mixer, from anywhere in the world using a web-app is a great leap forwards. The web-app can be accessed by multiple people in different locations, so if it’s a compact “headless” mixer at the venue (i.e. one without a control surface), a local technician can use it to check the mics and monitors they have connected. They can also do the routing for the mic and monitors, and also to and from the IP interface, or choose to leave this to an engineer or the main A1 back at HQ.

Only being able to control from a web-app adds some burden to the A1 at the main facility if they need to make adjustments when on air, and they may therefore need an assistant. To counter this, we allow the remote mixer to be controlled directly from the surface of the main mixer at the facility, in exactly the same way they control their local sources and destinations, so the A1 can have all local and remote sources on their board, controlling them all in the same way as if the venue was in a studio next door. They can freely adjust the remote mic gains, fader levels, routing, send and bus output levels from the comfort of their own familiar surface. Both the web-app, and the surface control is all standard IP, so it is easy to connect them wherever you need it.

That said, users should be mindful of the effect of control lag if dynamically controlling the remote mix whilst on-air. The latency of the network connection will directly affect how responsive the remote mixer is. For best results and assured performance, use a private/leased IP connection that provides fixed routing and latency.

  1. Audio Format, Connectivity and Transport

With the monitor mix and control taken care of, we just need to concern ourselves with how we get the audio back

and forth between the venue and the broadcast center.

Broadcasters like flexibility and options, and equipment that is designed specifically for broadcast provides this. A broadcast audio mixer provides I/O options for analogue, AES3, MADI, SDI, AES67 and more, so you can get all your signals, mics, monitors, as well as backhaul and return feeds, without needing extra boxes and interconnects. You can choose how you transport the audio between the venue and the facility (see Figure 7).

Live At-Home Production 2.0

FIGURE 7: AUDIO AND REMOTE CONTROL FOR AT-HOME PRODUCTION

SDI is popular for many reasons, and it can be passed through IP codecs, so passing your SDI feed/s through the audio mixer and having it embed its audio output reduces the connections to the codec. It also provides a convenient way to keep audio in sync with video, and using familiar workflows.

IP codecs can also pass MADI and AES3, so there are options and flexibility in how you get the audio between sites if you want to keep the audio easily accessible. AES67 is also becoming more widespread. Having a mixer that can input/output AES67 directly means you do not need an external IP codec; it can connect directly with an IP network, although it does require PTP sync with the same reference as the one at the broadcast facility.

When separated by a WAN or long haul extended LAN, the jitter can cause PTP problems, so separate PTP clocks with GPS references at both ends is normally necessary. However, engineers are working to overcome this. If PTP can be passed over a long-haul connection, it will further simplify kit and config requirements at the remote site, paving the way for easier elemental streaming.

Rather than having audio embedded into video, elemental streaming is the goal of ST-2110. Having separate streams for video, audio and metadata (passed over the same network), all guaranteed to be time-aligned thanks to the IP protocols, makes it quicker for each process in a broadcast workflow to get at what they need, reducing the packetization and de-packetization and therefore improving efficiencies. Additional bandwidth needs to be factored in when transporting audio separately to the video, just as it does for remote-control data connections, although these requirements are negligible compared to that of the video feeds that accompany it.

 For assured performance, audio, video, metadata and control data should all be passed over a private/leased IP network with fixed routing and latency. Such connections are available almost anywhere in the world, with bandwidth that scales very cost-effectively to suit simple as well as complex productions.

  1. Summary, At-Home Production Audio

Modern broadcast audio mixers should provide automated and assistive functionality that reduces the burden of operators. Along with IP control interfaces that allow them to be controlled from anywhere in the world, and MADI, SDI and AoIP options for audio input and output, mixing consoles can offer a one-box solution for At-Home-Production audio workflow, virtualizing remote locations into a broadcast facility.

 

At-home production signal transport

The transport of the above described video, audio and data signals for at-home production implies several challenges.

The main requirements for an at-home transport solution is a fully manageable and switchable one-box-solution, providing all required interfaces to both the broadcasting as well as the telecom infrastructures, is it IP, SDH, WDM or the public Internet.

Figure 8 shows the various signals that need to be transported for an at-home production. Video signals can arrive in a variety of formats and transported within SDI, SMPTE 2022-6 or 2110. Camera signals especially need the return signals for synchronization and genlock. Audio signals are either embedded, or arrive as, AES-67, SMPTE 2110 or MADI signals, and these are usually transported over native interfaces or transparent Ethernet pipes. Ethernet also allows the transport of a variety of peripheral equipment and control data, such as intercom, tally or graphics and can be further used for file transfer and on-site internet access.

Before at-home production came into play, the PGM feed and several camera feeds of the entire production were usually recorded at the OB truck. As this became obsolete more backup of signals was required. Backing up signals across the public internet therefore became a welcome solution for many applications. Here a trade-off between latency and quality allows for a good compromise, ensuring no downtimes occur during production.

Live At-Home Production 2.0

FIGURE 8: AT-HOME PRODUCTION SIGNAL TRANSPORT

The following sections describe the primary challenges and requirements on transport over Wide Area Network and telecom infrastructures.

 

I.General transport challenges

Packet-loss and delay variation (Jitter) in networks cause a variety of problems, from synchronization issues to drop-outs, glitches and downtimes. Much stricter requirements and SLAs are required to properly transport media across telecommunication infrastructures compared to normal data traffic. However, engineering the required Quality of Service for media transport is an operational challenge to many providers. An IP technology that supports synchronous scheduling and switching of services will make it possible to provide the required Quality of Service for media services and enable the required reliability to manage even the biggest networks across the globe [2]. Enhancing the transport solution with common methods like Forward error correction and Hitless 1+1 protection additionally guarantee a proper signal transport over any underlying telecom provided infrastructure.

II.Available bandwidth and signal compression

Running at-home productions over common infrastructures introduces a trade-off between cost and bandwidth. The luxury of dedicated dark fiber networks is therefore either connected to high costs or simply not available at many event locations and regions. Therefore, video signals carrying most data need to be compressed. At-home production requires very low-latency and almost lossless compression of the raw feeds. JPEG2000 offers the best compromise with its very low latency of about 1-2 frames and efficient visual compression up to 1:12. Audio, due to its nature of requiring lower bandwidth usually does not require any compression, but demands a transparent transport over any underlying infrastructure and synchronization of video, audio and data at the egress.

III.Synchronization and multistream alignment

At-home production requires a high level of synchronization for an accurate playout of audio, video and data services. In an all IP setup, the transport and generation of PTP timestamps is crucial to enable proper alignment of all signals. Hence, one of the requirements for transport equipment is to provide the studio clock at the venue, by either generating it out of the return signal or the transparent transport of PTP timestamps over Ethernet.

 

IV.Remote monitoring and orchestration

Monitoring and orchestrating long-haul transmission with the large amount of services needed for at-home productions is extremely complex. Several requirements need to be considered to enable full control of the remote venue:

  • Network performance monitoring to allow link utilization monitoring, and to resolve other issues which may arise, without interruption
  • Automatic signal routing across large networks to allow easy source and destination routing from the remote venue to the at-home gallery
  • Service-aware provisioning of the video, audio and data services for the at-home production
  • Fault management to enable and automate the right mechanisms in case of error, such as rerouting, hitless switchover or backup signal routing.

V.Summary signal transport

While at-home production over Dark Fiber is like an extended IP Studio production, the more commonly adopted production techniques over telecom Wide Area Network infrastructures require more in-depth knowledge and monitoring of the underlying infrastructure. A solution to combine both sides i.e. the media/broadcasting and the telecom/networking side, is crucial to ensure a successful implementation of at-home productions. Accurate monitoring, link enhancement and synchronization are crucial factors to its future success.   

The next step of innovation

As multiplatform distribution needs much more content to be produced, new ways must be introduced to ease efficient workflows like At-home production Several thousands of at-home productions have been made across the globe for recurring or weekly sport events, elections or lower tier productions, but also unilateral coverage of the Olympic Games, Soccer World cups or the Australian open. Every production brought more knowledge about how to improve production and communication as well as how to enhance the applied products.

Live At-home production 2.0 is the next step towards fully distributed and plug-and-play workflow, enabling to connect, orchestrate and to use the best resources across Wide Area Networks and allowing content producers and broadcasters to create more with less.               

 

References

  1. Weber, Klaus, ” High Bandwidth IP Networks – How they change the Landscape of Live Remote Production”, Presented at the ABE conference, August 2016 in Sydney, Australia
  2. Olsson, Dr. Bengt, “IP QoS Objectives For Broadcast Services” NAB Broadcast Engineering Conference Proceedings, Stockholm 2014