of Time-Triggered Ether-Networks Using Off-The-Shelf-Components AMICS Workshop 2011 Till Steinbach, Franz Korf, Thomas C. Schmidt {florian.bartols,till.steinbach,korf,schmidt} @informatik.haw-hamburg.de March, 31st 2011
Agenda 1 Motivation 2 3 4 Motivation 5 2 / 27
Motivation Motivation 3 / 27
Motivation Real-time Ethernet in automotive applications: In-vehicle networks are very complex New automotive applications require more bandwidth Real-time Ethernet is widely deployed in backbones of industrial plants Comparable real-time characteristics in vehicles Motivation 4 / 27
Motivation Low cost performance analysis with time-triggered packet generation: Missing performance analyzer instruments in tool chain Tools for standard switched Ethernet are not suitable Flexible embedded pc based tools gain importance Motivation 5 / 27
Agenda 1 2 Ethernet Measurement Approaches Real-time Ethernet 3 4 Ethernet Measurement Approaches Real-time Ethernet 5 6 / 27
Related Work Ethernet Measurement Approaches Distinction between software and hardware based measurements Ethernet Measurement Approaches Real-time Ethernet 7 / 27
Related Work Ethernet Measurement Approaches Distinction between software and hardware based measurements Collection on one network port only Analyzer NuT Ethernet Measurement Approaches Real-time Ethernet 7 / 27
Related Work Ethernet Measurement Approaches Distinction between software and hardware based measurements Collection on one network port only Distributed measurement with synchronized timebase Analyzer NuT Analyzer Ethernet Measurement Approaches Real-time Ethernet 7 / 27
Related Work Ethernet Measurement Approaches Distinction between software and hardware based measurements Collection on one network port only Distributed measurement with synchronized timebase One physical clock and several network ports Ethernet Measurement Approaches Real-time Ethernet Analyzer NuT 7 / 27
Real-time Ethernet 3 strategies to enable real-time characteristics on Ethernet-based networks Ethernet Measurement Approaches Real-time Ethernet 8 / 27
Real-time Ethernet 3 strategies to enable real-time characteristics on Ethernet-based networks Token-based systems Used in Ethercat in process automation Ethernet Measurement Approaches Real-time Ethernet 8 / 27
Real-time Ethernet 3 strategies to enable real-time characteristics on Ethernet-based networks Token-based systems Used in Ethercat in process automation Bandwidth limiting systems For instance AFDX in Airplanes (A380 and Boeing 787) Ethernet Measurement Approaches Real-time Ethernet 8 / 27
Real-time Ethernet 3 strategies to enable real-time characteristics on Ethernet-based networks Token-based systems Used in Ethercat in process automation Bandwidth limiting systems For instance AFDX in Airplanes (A380 and Boeing 787) Time-triggered systems Automotive industry (FlexRay) and process automation (Profinet) Ethernet Measurement Approaches Real-time Ethernet 8 / 27
Time-Triggered Ethernet 3 Traffic Classes in TTEthernet Ethernet Measurement Approaches Real-time Ethernet 9 / 27
Time-Triggered Ethernet 3 Traffic Classes in TTEthernet 1 Time-Triggered Highest priority and used for hard real-time data Strictest requirements in transmission Ethernet Measurement Approaches Real-time Ethernet 9 / 27
Time-Triggered Ethernet 3 Traffic Classes in TTEthernet 1 Time-Triggered Highest priority and used for hard real-time data Strictest requirements in transmission 2 Rate-Constrained Also suitable for real-time communication Complies to the AFDX protocol Ethernet Measurement Approaches Real-time Ethernet 9 / 27
Time-Triggered Ethernet 3 Traffic Classes in TTEthernet 1 Time-Triggered Highest priority and used for hard real-time data Strictest requirements in transmission 2 Rate-Constrained Also suitable for real-time communication Complies to the AFDX protocol 3 Best-Effort Standard Ethernet traffic No guarantee in transmission Ethernet Measurement Approaches Real-time Ethernet 9 / 27
Time-Triggered Ethernet 3 Traffic Classes in TTEthernet 1 Time-Triggered Highest priority and used for hard real-time data Strictest requirements in transmission 2 Rate-Constrained Also suitable for real-time communication Complies to the AFDX protocol 3 Best-Effort Standard Ethernet traffic No guarantee in transmission Ethernet Measurement Approaches Real-time Ethernet Time synchronization for a global time base Works transparent to the Ethernet-Protocol-Layer 9 / 27
Time-Triggered Ethernet chassis TT TT 0 LIN-BUS RC 0 cycle LIN gateway door t TTE-switch 0 TT RC BE BE TT BE cycle TTE-switch t Ethernet Measurement Approaches Real-time Ethernet BE 0 diagnosis BE BE t t TT RC BE Time-Triggered Message Rate-Constrained Message Best-Effort Message 10 / 27
Agenda 1 2 3 Concept Accessing the Current System Time Assigning Timestamps to Frames 4 Concept Accessing the Current System Time Assigning Timestamps to Frames 5 11 / 27
Problem statement Available OS tools: Tools running in Userspace (e.g. ping, traceroute) Precision in milliseconds provided by OS Scheduling and hardware-layer produce overhead OS tools measure round-trip-time Concept Accessing the Current System Time Assigning Timestamps to Frames 12 / 27
Solution Our approach: Measurement on a low-level base Kernelspace in Linux RT-Linux utilization for enhanced interrupt handling Exact timing behavior by using special functions Concept Accessing the Current System Time Assigning Timestamps to Frames 13 / 27
Concept Assembling Application Calculation Userspace TTEthernet Stack TT Device Driver Timestamping TT-Network under test BE Device Driver Timestamping Kernelspace Hardware Hardware Physical Layer Concept Accessing the Current System Time Assigning Timestamps to Frames 14 / 27
Accessing the Current System Time Precision Jiffies Counter Concept Accessing the Current System Time Assigning Timestamps to Frames Methods 15 / 27
Accessing the Current System Time Precision Jiffies Counter do_gettimeofday() Concept Accessing the Current System Time Assigning Timestamps to Frames Methods 15 / 27
Accessing the Current System Time Precision Jiffies Counter do_gettimeofday() get_cycles() Concept Accessing the Current System Time Assigning Timestamps to Frames Methods 15 / 27
Accessing the Current System Time Precision Jiffies Counter do_gettimeofday() get_cycles() Concept Accessing the Current System Time Assigning Timestamps to Frames Methods 15 / 27
Assigning Timestamps to Frames 2 approaches to get timing information of a single frame 1 Storing information in Kernelspace High implementation cost (limited Kernel functions) Additional information has be stored Concept Accessing the Current System Time Assigning Timestamps to Frames 16 / 27
Assigning Timestamps to Frames 2 approaches to get timing information of a single frame 1 Storing information in Kernelspace High implementation cost (limited Kernel functions) Additional information has be stored 2 Modifying Ethernet frames with timestamps A lightweight approach Assigning timestamps to frames in an easy way Measurement can be done promptly without reading and storing information Ethernet Frame Header Data Measurement Payload CRC Concept Accessing the Current System Time Assigning Timestamps to Frames Send Recv Seq 16 / 27
Agenda 1 2 3 4 Validating the Approach Measurement Results Validating the Approach Measurement Results 5 17 / 27
Validation Sample Measurement 3ms System Schedule for each TT-Frame 350µs static switch delay Server Application receive timestamp BE 1 TT send timestamp 4 3 ms measurement frames 3 system schedule TT-Network under test 2 Validating the Approach Measurement Results 18 / 27
Validation Sample Measurement L a te n c y (µs ) 6 0 0 5 8 0 5 6 0 5 4 0 5 2 0 5 0 0 4 8 0 4 6 0 4 4 0 4 2 0 4 0 0 3 8 0 3 6 0 0 2 0 0 4 0 0 6 0 0 8 0 0 1 0 0 0 1 2 0 0 1 4 0 0 1 6 0 0 F ra m e s iz e (B y te ) s td. c o n fig Validating the Approach Measurement Results 19 / 27
Validation Sample Measurement L a te n c y (µs ) 6 0 0 5 8 0 5 6 0 5 4 0 5 2 0 5 0 0 4 8 0 4 6 0 4 4 0 4 2 0 4 0 0 3 8 0 3 6 0 s td. c o n fig 6 4 B y te th re s h o ld Validating the Approach Measurement Results 0 2 0 0 4 0 0 6 0 0 8 0 0 1 0 0 0 1 2 0 0 1 4 0 0 1 6 0 0 F ra m e s iz e (B y te ) Observation: A specific hardware and driver delay has to be substracted for each measurment 19 / 27
Validation Applied Validation Approaches 1 High precision Ethernet analyzer tool BE Server Application TT 4 system schedule measurement frames 3 ms 3 2 t1 2 port1 1 duplicate measurement frames Ethernet Analyzer t = t2 - t1 TT-Network under test t2 port2 1 Validating the Approach Measurement Results 20 / 27
Validation Applied Validation Approaches 1 High precision Ethernet analyzer tool BE Server Application TT 4 system schedule measurement frames 3 ms 3 t1 port1 duplicate measurement frames Ethernet Analyzer t = t2 - t1 TT-Network under test t2 port2 2 Mathematical framework Analytical method to calculate latency, bandwidth and jitter 2 2 1 1 Validating the Approach Measurement Results 20 / 27
Validation Results in Comparison measurement hardware mathematical approach sniffing model min. framelength 355µs 364µs 355.125µs max. framelength 471µs 479µs 471.445µs Validating the Approach Measurement Results 21 / 27
Validation Results in Comparison measurement hardware mathematical approach sniffing model min. framelength 355µs 364µs 355.125µs max. framelength 471µs 479µs 471.445µs Difference of 9µs is due to the used duplication switch measurement hardware mathematical approach sniffing model min. framelength 355µs 355µs 355.125µs max. framelength 471µs 470µs 471.445µs Validating the Approach Measurement Results 21 / 27
Latency Comparison COTS switch and TTEthernet Switch Validating the Approach Measurement Results 22 / 27
Latency Distribution Graph Validating the Approach Measurement Results 23 / 27
Agenda 1 2 3 4 5 24 / 27
Conclusion: Performance analysis with COTS in single-digit microseconds precision Synchronized packet generation Utilization of RT-Linux, modified device drivers and TTEthernetstack 25 / 27
Conclusion: Outlook: Performance analysis with COTS in single-digit microseconds precision Synchronized packet generation Utilization of RT-Linux, modified device drivers and TTEthernetstack Validation using hardware with specifically low receive and copy delay Adjustment of this approach on an ARM-based micro controller to increase resolution 25 / 27
Thank you! Thank you for your attention! Website of research group: http://www.informatik.haw-hamburg.de/core.html 26 / 27
TTEthernet Synchronization Method End System Sync Master 1. PCF (time) 2. PCF (new Time) End System Sync Client Switch Compression Master 2. PCF (new Time) 2. PCF (new Time) End System Sync Client 1. PCF (time) 2. PCF (new Time) End System Sync Master 27 / 27