RT-WiFi: Real-Time High-Speed Communication Protocol for Wireless Cyber-Physical Control Applications

RT-WiFi: Real-Time High-Speed Communication Protocol for Wireless Cyber-Physical Control Applications Yi-Hung Wei, Quan Leng, Song Han, Aloysius K. Mok, Wenlong Zhang, Masayoshi Tomizuka Department of Computer Science, The University of Texas at Austin Department of Computer Science and Engineering, University of Connecticut Department of Mechanical Engineering, University of California, Berkeley 1

Wireless Control System Wireless Control Systems A control system where the control loops are through wireless communication network Advantages: Enhanced mobility Reduced deployment and maintenance cost Applications: Industrial Process Control and Automation Healthcare Robotic Control 2

Existing Technologies and RT-WiFi 3

Outline Introduction RT-WiFi Design Performance Evaluation Case Study Conclusion and Future Work 4

RT-WiFi Design Goals 1. Real-time data delivery and high sampling rate Aim to provide at least 1 khz sampling rate minimum requirement for many mechanical control systems 2. Flexible Configuration Configurable parameters: sampling rate, predictability of real-time data delivery, reliability, co-existence with regular WiFi networks 3. Transparent System Design Use commercial-off-the-shelf network card Transparent to upper layer protocols 5

Overview of a Control System using RT-WiFi Network 6

Review of Regular WiFi Channel access method in regular WiFi: CSMA/CA Carrier Sense Random Backoff STA1 busy data wait Data ready STA2 busy wait data Data ready Carrier Sense Random Backoff Carrier Sense Random Backoff 7

Main Design of RT-WiFi: TDMA on WiFi TDMA: Time Division Multiple Access Divide the channel into time slots and assign time slots to each station. With TDMA, we do not need carrier sense and random backoff. Each station knows when it can send its packets. STA 1 Send STA 2 Send STA 1 Send STA 2 Send STA 1 Send STA 2 Send An example of TDMA schedule 8

Architecture of RT-WiFi Protocol 9

Architecture of RT-WiFi Protocol Based on Timing Synchronization function (TSF) in IEEE802.11 Synchronize all the stations in the RT-WiFi network. Generate timer interrupt 10

System Architecture of RT-WiFi Protocol Coordinate channel access among the stations Link: Broadcast link, transmit link, receive link, shared link Superframe: Slot 0 AP Broadcast Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7 Shared STA1 AP STA2 AP STA3 AP AP STA1 AP STA2 AP STA3 11

System Architecture of RT-WiFi Protocol Configuration parameters: Sampling rate Co-existence with regular WiFi network Reliability (Retransmission) 12

Hard Real-Time Traffic How to Enforce TDMA in the Presence of CSMA/CA? For RT-WiFi to grab the channel, we use shorter interframe space (IFS) RT-WiFi Busy Channel RT-IFS RT-WiFi Data Bounded Length Regular WiFi Regular WiFi Traffic DIFS Regular WiFi Traffic Deterred After RT-WiFi grabs the channel, RT-WiFi traffic will have precedence over regular WiFi traffic for as long as needed. When bounded latency communication is not needed, regular WiFi can use the channel. 13

Soft Real-Time Traffic (Enabling Co-existence with Regular WiFi Network) 1. Assume bounded length for regular WiFi data frames to ensure bounded latency Limit maximum transmission unit to at most some upper bound Limit lowest data rate to at least some lower bound 2. For RT-WiFi: Enable carrier sense Use shorter interframe space (IFS) Time Slot RT-WiFi Busy Channel RT-IFS RT-WiFi Data ACK Guard Interval Bounded Length Regular WiFi Regular WiFi Traffic DIFS Regular WiFi Traffic 14

How to Increase Reliability in TDMA? Retransmission In-slot retransmission Out-of-slot retransmission Time Slot Data Ack Guard Interval Ack Wait (a) Regular time slot Time Slot Data Data Retransmission ACK Guard Interval ACK Wait (b) Time slot with in-slot retransmission ACK Wait 15

Outline Introduction RT-WiFi Design Performance Evaluation Case Study Conclusion and Future Work 16

Testbed setup Setup: RT-WiFi baseline vs. regular WiFi UDP socket program to emulate control applications Metrics: MAC layer to MAC layer latency Packet loss ratio Environment: Interference free environment vs. office environment Timeslot size: 500 μs 17

Results of RT-WiFi baseline (TDMA only) in Interference Free Environment Link Max Latency(μs) Mean Latency(μs) Latency stdev. (μs) Loss Ratio RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi STA1->AP 535 16448 173 191 4.88 231.60 0.19% 0% STA2->AP 529 13387 172 181 2.52 214.15 0.16% 0% STA3->AP 525 13589 174 202 3.01 236.75 0.21% 0% AP->STA1 827 16472 184 250 5.29 342.24 0.06% 0% AP->STA2 544 17465 187 298 3.98 360.66 0.04% 0% AP->STA3 1055 17049 188 248 4.87 325.50 0.01% 0% 16-32x 47-90x The delay in RT-WiFi is caused by the delay of the physical layer in initiating transmission due to factors such as listening to other beacons that we do not fully understand at this point. 18

Results of RT-WiFi baseline (TDMA only) in Office Environment Link Max Latency(μs) Mean Latency(μs) Latency stdev. (μs) Loss Ratio RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi RT-WiFi Wi-Fi STA1->AP 3865 100078 176 401 25.86 1491.69 8.64% 0% STA2->AP 4193 81499 171 348 27.62 1000.60 9.97% 0% STA3->AP 3861 75298 174 429 25.16 1221.72 7.55% 0% AP->STA1 1197 78089 184 788 16.86 2861.42 9.52% 0% AP->STA2 1342 78923 189 790 15.19 2806.56 8.09% 0% AP->STA3 2186 77860 189 799 19.03 2855.89 9.31% 0% 2.0-4.3x 36-184x The delay in RT-WiFi is caused by the delay of the physical layer in initiating transmission due to factors such as listening to other beacons that we do not fully understand at this point. 19

Flexible Channel Access Controller Experiment Setup Network A: Regular WiFi network 10 Mbps UDP traffic generated by iperf Network B: UDP program to emulate control applications Compare: Regular WiFi RT-WiFi baseline RT-WiFi with co-existence enabled RT-WiFi with co-existence enabled and one in-slot retransmission enabled Metrics: MAC to MAC layer latency Packet loss ratio 20

Flexible Channel Access Controller Experiment Results Max Latency (μs) Mean Latency (μs) Latency Stdev. (μs) Loss Ratio Regular WiFi 62629 580 1679.04 0% RT-WiFi baseline 953 183 27.75 50.21% RT-WiFi co-ex 2507 220 149.27 10.92% RT-WiFi co-ex + retry 2831 254 166.37 4.96% 21

Outline Introduction RT-WiFi Design Performance Evaluation Case Study Conclusion and Future Work 22

Case Study: A Mobile Gait Rehabilitation System Wireless Motion Sensors RT-WiFi Station Sensing Signals for Health Monitoring (100Hz) Remote Doctors and Physical Therapist Mobile Robotic Device Wireless Smart Shoes RT-WiFi Station Sensing Signals for Controlling Assistive Robot (1KHz) Control Signals for Mobile Assistive Robot (1KHz) Sensing Signals for Health Monitoring (100Hz) RT-WiFi AP Sensing Signals Control Signals A Host Computer Running Control Applications RT-WiFi Station 23

Integrate RT-WiFi with Smart Shoes IEEE 1588 Integration of smart shoes with a RT-WiFi station 24

Simulation Result of a Wireless Control System ECDF of tracking errors 25

Outline Introduction RT-WiFi Design Performance Evaluation Case Study Conclusion and Future Work 26

Conclusion and Future Work Designed and implemented RT-WiFi protocol Up to 6 khz sampling rate Predictable real-time data delivery Flexible wireless communication platform Future work Dynamic network management Power management Extension to mesh topology Fault tolerance 27

Thanks & Questions? 28

References [1] Y.-H. Wei, Q. Leng, S. Han, A. K. Mok, W. Zhang, and M. Tomizuka, RT-WiFi: Real-Time High-Speed Communication Protocol for Wireless Cyber-Physical Control Applications, Technical Report http://www.cs.utexas.edu/users/yhwei/pub/rtwifi-tr.pdf. [2] J. Song, S. Han, A. K. Mok, D. Chen, M. Lucas, M. Nixon, and W. Pratt, WirelessHART: Applying wireless technology in real-time industrial process control, in IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS), 2008, pp. 377-386. [3] W. Zhang, X, Zhu, S. Han, N. Byl, A. K. Mok, and M. Tomizuka, Design of a network-based gait rehabilitation system, in IEEE International Conference on Robotics and Biomimetics (ROBIO), 2012, pp. 1773-1778. [4] S. Han, A. K. Mok, J. Meng, Y.-H. Wei, P.-C. Huang, Q. Leng, X. Zhu, L. Sentis, K. S. Kim, and R. Miikkulainen, Architecture of a cyberphysical avatar, in International Conference on Cyber-Physical Systems (ICCPS), 2013. 29

References [5] Y.-H. Wei, Q. Leng, S. Han, A. K. Mok, W. Zhang, M. Tomizuka, T. Li, D. Leith, and D. Malone, RT-WiFi: Real-time high speed communication protocol for wireless control systems, in IEEE Real-Time Systems Symposium Work-in-Progress Session (RTSS WiP), 2012. [6] ISA 100, http://www.isa.org/isa100. [7] X. Zhu, S. Han, P.-C. Huang, A. K. Mok, and D. Chen, MBstar: A real-time communication protocol for wireless body area networks, in IEEE Euromicro Conference on Real-Time Systems (ECRTS), 2011, pp. 57-66. [8] IEEE 802.11 working group, http://www.ieee802.org/11/. [9] J. C. Eidson, Measurement, control and communication using IEEE1588, Springer Publishing Company, Incorporated, 2010. 30

Backup Slides 31

Communication Requirements for Future Wireless Control Systems Real-time data delivery High sampling rate Tracking performance of PD control systems with different sampling rates (partial amplification) 32

System Architecture of RT-WiFi Protocol Message reception module Message transmission module 33

RT-WiFi Time slot Time slot size: T Timeslot T GuardInter val T Data T SIFS T ACK Example: UDP socket with 200 bytes application layer payload IEEE 802.11g with 54Mbps data rate 802.11 ERP-OFDM coding scheme PLCP Header APP UDP IP MAC 34

Sampling Rate Higher sampling rate is better for control applications, but requires smaller time slot size. Sampling rate depends on: Packet size and data rate Synchronization accuracy Computational resource Time slot size: T Timeslot T GuardInter val T Data T SIFS T ACK 35

In-slot retransmission Reliability Out-of-slot retransmission Retransmit in the next time slot of this link 36

Co-existence with Regular WiFi Network Assumption on WiFi: Conforms CSMA/CA Limit maximum transmit unit (MTU) size Limit lowest available data rate For RT-WiFi: Perform carrier sense Use shorter inter-frame spacing 37

System Implementation Hardware and software platform Atheros AR9285 Wi-Fi card Ubuntu 12.04 with Linux kernel 3.2.0 Open source Linux compat-wireless driver Two software modules from compat-wireless driver, mac80211 and ath9k, are incorporated with the TDMA design to build the MAC layer of RT-WiFi. Highest sampling rate supported in experiments: 6 khz 38

Results in Interference Free Environment Latency Empirical CDF for uplink (STAs->AP) Latency Empirical CDF for downlinks(ap->stas) 39

Results in Office Environment Latency Empirical CDF for uplink (STAs->AP) Latency Empirical CDF for downlinks(ap->stas) 40

Simulation of a Wireless Control System After integrate RT-WiFi with smart shoes, we collect traces of each sensor data packets to get the network dynamics. Then, we run simulation of the controller based on the traces to evaluate the performance of the wireless control system. A proportional-derivative (PD) controller was implemented in the simulation to track a sinusoidal signal with unit magnitude. Simulation were conducted with different controller gains and reference frequencies. 41

Simulation of a Wireless Control System ECDF of tracking errors with K p =0.5, K d = 0.005 ECDF of tracking errors with K p =1, K d = 0.01 42

Transmission Time (μs) Transmission Time vs. Payload Size 350 Transmission Time vs Payload Size 300 250 200 150 100 50 0 0 500 1000 1500 2000 Payload Size (bytes) 43

6 khz experiment results Case Max Delay Mean Delay Delay STD Loss Ratio 6 khz 2125 167 31.24 11.88% 44