Addressing Live Video Latency

This article originally appeared in the Feb. 2012 issue of Next Gen Mobility

With video-friendly mobile devices proliferating, and wireless and wire IP networks offering increasing speed, viewing good quality video anyplace is increasingly feasible. For some content a delay (latency) of even several seconds is not a concern. However, for other events – such as real-time news gathering in which the studio converses with a reporter in the field while live video goes to a national or even international audience – delay is something to minimize, even at significant cost. 

We have all seen the awkward pauses while the reporter waits for the studio’s delayed question, then the studio waits for the reporter’s delayed reply. In these cases latency of less than three seconds is generally a requirement, two seconds or better is nirvana.

The flow from camera to screen, and how each step is handled, are key to estimating latency. The typical steps and typical delays are:

1.)    camera to hardware or software video encoder (typically between 50 ms < E < 200 ms)

2.)    preparing the encoded video for transmission (P < 5 ms)

·         putting video and audio content into packets
·         encrypting if necessary
·         adding redundant forward error correction

3.)    transmitting on wireless or wired network(s), single network or aggregated (bonded networks(150 ms < T < 5 seconds) to the Internet cloud’s central hub

4.)    receiving and buffering content for error correction and decryption (50 ms< RB < 20 secs) at the central hub

5.)    processing content (PC < 5 ms)

·         error correction
·         decryption
·         stripping content from packets

6.)    transmitting from the hub’s media server in the Internet cloud to viewers

·         without transcoding (Tx < 100 ms)
·         with transcoding to multiple quality levels (Tx <300 ms

7.)    transferring and buffering content

·         in smartphone video player (30 < B < 40 secs)
·         in a desktop video player (B < 3 secs)

8.)    displaying content on the viewer’s screen (D < 5 ms)

Encoding is faster in hardware. But software encoders and the computers they run on are very much a challenge to hardware encoders. Even in software, where delays up to 200 ms occur, the impact on applications that require low latency is tolerable.

Transmission preparation, even with substantial content creation for FEC and processing for encryption, is usually handled in computer RAM (News - Alert) which can easily outpace the input-output from storage. The complete delay is negligible – less than 5 ms.

Transmission varies by media (e.g., satellite, terrestrial wireless, copper wire, and fiber). Satellites are about 22,200 miles above the Earth. Even at the speed of light (186,000 miles/sec) that means about 230 ms roundtrip. More direct terrestrial links are relatively short distances apart at the speed of light, but they can vary greatly with respect to performance – fiber can be very clean and very fast, copper wire is clean but not as fast as copper, and wireless networks range from adequate to terrible, especially in urban locations with tall buildings and many competing signals. This is true from field to hub, and from hub to viewer.

The first real substantial delay in most live video transmissions occurs when the content is received at the hub, and it must be buffered for error correction and for decryption. There are many algorithms for both error correction and for encryption/decryption (for another article). 

FEC is needed most for wireless transmissions, such as with satellite and terrestrial wireless networks. Typical FEC overhead is in the range of 2 percent to 7 percent for satellite, and 5 percent to 10 percent for terrestrial wireless networks. Encoding and decoding the FEC is less than 5 ms each for live video streams, but the delay comes in the necessity to first buffer then maintaining enough incoming content to perform the FEC decode processing. This usually involves a time window of two to seven seconds. The greater the buffering time window is, the more time for correcting errors. If a signal is relatively clean, the delay can be set at the lower number. For very noisy signals requiring much repair, such as in wireless networks in urban areas, a more typical delay for FEC buffering will be closer to seven seconds. If a delay is completely tolerable, the delay can be 10-20 seconds to ensure the cleanest results.

Processing received content is again fast once the buffer is initially established and maintained. This is also only a few milliseconds. The media server that transmits to the viewer will add a small delay (about 100 ms) without transcoding, and a bit larger with transcoding (300 ms).

The second buffer delay is introduced by the viewer’s video player’s play-out buffer. Dedicated hardware decoders with substantial processing power, as in game players, are fast. But in current smartphones, substantial buffering is necessary to provide a smooth picture on a small screen. Upon getting the feed as the last step, the iPhone (News - Alert) will introduce a substantial delay of about 35-40 seconds. 

There is constant improvement in FEC algorithms and in video player offerings, the two sources of substantial buffering delay. As wireless networks get cleaner, less buffering is needed by FEC. Faster computer processing means less delay for smooth play-out. 

William E. Steele is chairman and CEO of KenCast (News - Alert) (www.kencast.com).