A low-latency yet highly reliable wireless video transfer system Tobias Welti Institute of Embedded Systems Team High Performance Multimedia & Data Acquisition
Target Specifications 30 fps interactive large data low-latency volume highly reliable wireless video transfer use tablet computer IEEE 802.11 no dropped frames 2
Interactive Applications Photo: Hubsan 3
Live Interview on TV 4 SRF
Remote Controlled Drone (FPV) Photo: Hubsan Photo: Skyzone 30 km/h: 2.1 m in 250 ms Wireless Technology Band Latency IEEE 802.11 2.4 / 5 GHz 250 ms Wideband FM 5.8 GHz 60 150 ms Wireless HD 60 GHz 30 ms 5
System Description Camera Unit IEEE802.11 1000 1101 0111 0101 0000 1110 1111 0001 1011 100 ms Display Unit Photo: Hubsan Photo: Apple Inc. 6
Sources of Latency Processing Transmission Processing 33 15 37 15 100 ms 7
Low Latency: How-To Problem Processing Time Idea Get a head start Transmission Speed Transmission Amount Retransmission 8
Get A Head Start Image Sensor Image Processor Image Buffer (400 lines, 33 ms) Image Sensor Image Buffer (16 lines, 1.3 ms) Image Processor 9
Pipelining A B C time step 1 A B C step 2 A B C A B C step 3 A B C A B C step 4 A B C transmission A B C 10
Low Latency: How-To Problem Processing Time Transmission Speed Idea Get a head start Transmit fast Solution Pipelining, small buffer Transmission Amount Retransmission 11
IEEE 802.11g vs IEEE 802.11n 12
Transmission Latency Single uncompressed TV frame: 3.3 Mbit Mbps ms fps 28 118 8 61 54 18 108 31 30 13 Markus Bertschi
Congestion 14 defensivedriversdiscount.com Barry Belfast Batchelor, Telegraph PA
Available Channels 5 GHz 23 Channels (CH) No Overlap Moderate use 15
Available Channels 2.4 GHz 14 Channels Overlap Heavily used 16
Congestion Setup Ch 1, max speed Measuring AP Measuring STA Ch 1/2/3, 100 kbps..100mbps Jamming AP Jamming STA 17
Congestion 18
Low Latency: How-To Problem Processing Time Transmission Speed Transmission Amount Idea Get a head start Transmit fast Send less data Solution Pipelining, small buffer Use multiple streams Retransmission 19
Kompression JPEG/MPEG H.264 H.265 Lossless: factor 2..3 Lossy: factor 100..200 20
Low Latency: How-To Problem Processing Time Transmission Speed Transmission Amount Idea Get a head start Transmit fast Send less data Solution Pipelining, small buffer Use multiple streams Compress data Retransmission 21
The Problem: Packet loss 22
Retransmission 802.11 Standard 802.11 Multicast 23
Low Latency: How-To Problem Processing Time Transmission Speed Transmission Amount Retransmission Idea Get a head start Transmit fast Send less data Alternative Reliability Scheme Solution Pipelining, small buffer Use multiple streams Compress data 24
Forward Error Correction 25
Low Latency: How-To Problem Processing Time Transmission Speed Transmission Amount Transmission Reliability Idea Get a head start Transmit fast Send less data Avoid Retransmission Solution Pipelining, small buffer Use multiple streams Compress data FEC, Backup Stream 26
27
28
29
30
Low Latency: How-To Problem Processing Time Transmission Speed Transmission Amount Transmission Reliability Idea Get a head start Transmit fast Send less data Avoid Retransmission Solution Pipelining, small buffer Use multiple streams Compress data FEC, Backup Stream 31
Our Solution Processing Transmission Processing 10 5 50 15 80 ms 100 ms 32
Conclusion Don t buffer whole image Pipelined processing Use suitable wireless standard Apply adaptive compression Additional reliability methods 33
Applications? Photo: Hubsan 34