Wireless Control That Simply Works Zigbee Network Performance Typical Results and Implications for Application Design Skip Ashton, Ember Corporation Copyright 2004 ZigBee TM Alliance. All Rights Reserved.
Testing Goals Establish Baseline of Zigbee network performance Join times may or may not be important based on application Throughput is an often requested parameter for application design Latency per hop and for the system Maximum messaging rate the network can sustain These results are initial test results and only represent one software implementation Evaluate single chip, network coprocessor and two chip solutions This type of testing will need to include multi-vendors to ensure interoperability does not effect results 2
Test Methodology Join Test Configuration Throughput Test Configuration Testing Done Using: EM2420/Atmega 128 and EM250 Standard development hardware kits 31 devices in join testing and up to 7 hops in throughput and latency EmberZNet Software all using table routing not tree Time stamping of packets using debug backchannel and packet trace Varied message type, packet size, security Test application written to initiate tests and gather output Nodes are not specially configured for these tests but are operating under expected typical configuration 3
Join Time Join Test Results Use of security on joining results in 150 millisecond longer join time Number of parents has impact on success rate due to broadcast collisions Join times need to be considered for new device joining network or mobile/sleeping device that has lost parent and is reassociating Impact of Security % Succeed Avg Join Time St Dev 96.00 706.45 109.32 No security EM250 96.22 604.88 11.59 Security Level 5, unsecure joining EM250 96.44 750.99 103.91 Security Level 5, secure joining EM250 95.33 763.50 102.02 Impact of Number of Parents One Parent 98.44 706.66 112.90 Many Parents, One permit joining 95.22 707.18 113.80 Many Parents, All permit joining 94.33 705.52 114.64 4
Expected Throughput Calculated theoretical payload throughput MAC only (102 byte payload) 115.9 kbps APS no security (88 byte payload) 100.0 kbps APS w/security (70 byte payload) 79.5 kbps This is based on the 802.15.4 MAC delays and Zigbee packet headers. Node processing delays will result in lower throughput than these results Multihop network also reduces throughput 5
Calculated Impact of Node Processing Delay 140000 120000 Calculated Maximum Throughput This represents calculated single hop performance only. Variation is due to payload capacity. 100000 Applicaiton Throughput 80000 60000 40000 MAC APS APS w/security 20000 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Node Processing Delay (milliseconds) 6
Typical Throughput APS Messages Zigbee APS Messages EM250 - No Security, No Retry 50000 45000 40000 35000 30000 25000 20000 15000 10000 5000 Application Throughput Throughput Data is for 91 byte payload Highest throughput at single hop smallest interpacket delay Peaks at 46 kbps for application throughput Performance drops after 2 hops due to packet loss Even at 5 hops, performance is higher than 25 kbps S1 Hops S3 S5 62.5 58.6 53.7 48.8 31.3 34.2 39.1 43.9 millisecond delay 15.6 0 Note: Throughput is based on expected throughput given the interpacket spacing and adjusted based on percent of successful packets from the test 7
Typical Throughput Adding APS Reply Zigbee APS MessageEM250 - No Security, APS retry Throughput Data is for 91 byte payload 50000 Highest throughput at single hop with 45000 smallest interpacket delay 40000 Peak remains at 46 kbps for application throughput 35000 30000 25000 20000 15000 10000 5000 A pplication thoug h p ut Performance drops quickly as reply consumes additional bandwidth There is a throughput penalty for knowing if message was delivered S1 hops S3 S5 62.5 58.6 53.7 48.8 31.3 34.2 39.1 43.9 millisecond delay 15.6 0 8
Typical Throughput Adding Security S1 hops S3 S5 S7 Zigbee APS Messages EM250 - Security, No retry Throughput Data is for 73 byte payload (reduced 50000 maximum payload due to security) 45000 Highest throughput at single hop smallest interpacket delay 40000 Peaks at 37 kbps for application 35000 throughput 62.5 58.6 15.6 31.3 34.2 39.1 43.9 48.8 53.7 millisecond delay 15000 10000 5000 0 30000 25000 20000 A pplic a tio n th ro u g h p ut Smaller max payload decreases maximum throughput Performance drops after 2 hops due to packet loss Even at 7 hops, performance is higher than 15 kbps 9
End to End Latency 120 100 End to End Latency - EM250 APS Message Latency Data is for 5 and 73 byte payload Data is measured at application interface milliseconds 80 60 40 5 byte Sec 73 byte Sec 5 byte -No Sec 73 byte No Sec Increases linearly per hop count Packet size and security increases per hop latency 20 0 1 2 3 4 5 6 7 Hops 10
Per Hop Latency 40.00 35.00 30.00 Per Hop Latency EM250 APS Messages Latency Data is for 5 and 73 byte payload Previous data is divided by hop count milliseconds per hop 25.00 20.00 15.00 Illustrates a clear per hop latency that varies with packet size and security 10.00 5.00 0.00 1 2 3 4 5 6 7 Hops 5 byte Sec 73 byte Sec 5 byte -No Sec 73 byte No Sec 11
Round Trip Latency 180 160 140 120 Round Trip Latency - EM250 APS Messages Latency Data is for 5 and 73 byte payload Measured at application interface Increases linearly per hop count Milliseconds 100 80 60 40 5 byte - Sec 73 byte - Sec 5 byte - No Sec 73 byte - No Sec Packet size and security increases per hop latency All data is higher than one way messages, but not double 20 0 1 2 3 4 5 6 7 Hops 12
Round Trip Latency Per Hop Round Trip Latency 40.00 35.00 30.00 25.00 20.00 15.00 Per Hop Latency - EM250 APS Reply 5 byte Sec 73 byte Sec 5 byte -No Sec 73 byte Sec Latency Data is for 5 and 73 byte payload Previous Data is divided by hop count The per hop latency that varies with packet size and security does not double with round trip message 10.00 5.00 0.00 1 2 3 4 5 6 7 Hops 13
Transmit Time of Packet Removed 40.00 35.00 30.00 25.00 Per Hop Latency - EM250 APS Messages OTA Time Removed Removal of OTA time results in 5 and 73 byte packets without security having similar latency Security impacted by packet size due to time required for processing milliseconds 20.00 15.00 10.00 5.00 0.00 1 2 3 4 5 6 7 Hops 5 byte -No Sec 73 byte No Sec 5 byte Sec 73 byte Sec 14
Impact of Two Chip Solution Zigbee APS Messages EM2420 - Security, No retries 50000 45000 This is using EM2420 and Atmega 128 processor. Security processing done using AES engine in EM2420 in stand alone mode. 40000 35000 30000 25000 20000 15000 10000 Application throughput Throughput Data is for 73 byte payload Highest at single hop smallest interpacket delay Peaks at 26 kbps instead of 37 kbps for single chip hops S1 S2 S3 S4 S5 53.7 S6 48.8 43.9 5000 15.6 31.3 34.2 39.1 millisecond delay 0 Performance drops rapidly to about 10 kbps Use of two chips decreases steady state expected throughput from 15 kbps to about 10 kbps when using security 15
Two Chip Latency 250 200 End to End Latency - EM2420 APS Messages Latency Data is for 5 and 73 byte payload Increases linearly per hop count similar to single chip m illiseconds 150 100 Security increases per hop latency even higher than on single chip 50 0 1 2 3 4 5 6 7 Hops 5 byte No Sec 73 byte - No Sec 5 byte - Sec 73 byte - Sec 16
Two Chip Per Hop Latency Per hop Latency 40.0 35.0 30.0 25.0 20.0 15.0 Per Hop Latency - EM2420 APS Messages Latency Data is for 5 and 73 byte payload Increases linearly per hop count similar to single chip Per hop latency is higher than single chip Security impact is larger 10.0 5.0 0.0 1 2 3 4 5 6 7 Hops 5 byte No Sec 73 byte - No Sec 5 byte - Sec 73 byte - Sec 17
Two Chip Time with Transmit Time Removed 40.0 35.0 30.0 25.0 Per Hop Latency EM2420 APS Message Latency - OTA Time Removed Results for 5 and 73 byte similar and close to single chip with security off. Impact of security is substantially higher in two chip solution. m illiseconds 20.0 15.0 10.0 5.0 0.0 1 2 3 4 5 6 7 hops 5 byte No Sec 73 byte - No Sec 5 byte - Sec 73 byte - Sec 18
Application Design Application Data models must reflect expected throughput Join time for mobile nodes must be considered in application design Throughput rate needs to be considered when transferring fixed blocks of data Exceeding bandwidth results in lost messages normally application would need to backoff in this case Use of reply provides indication message was received, but lowers overall throughput Per hop latency is consistent but varies with packet size and security settings application can set packet size and security settings but expected hop count depends on the topology Use of security lowers overall throughput and increases latency Application designers should turn on security early in product testing to ensure impact on system is understood Use of Single Chip solutions reduces the impact of security on throughput and latency 19
Additional Work Needed Analysis and optimization of results and software performance This represents some of the early performance data and some areas of improvement are expected as results are analyzed Improvements expected in stack processing time and security processing time Many to one routing throughput Gateway device represents bottleneck and needs to be characterized to allow system design to avoid this bottleneck Network processor testing needs to be completed Multi-vendor testing to evaluate the impact Changes to Zigbee specification and impact on results New mobile node procedure expected to be quicker than these results Windowing method in Zigbee fragmentation should improve throughput Reductions in payload will have incremental impact on application throughput 20
Thank You! Skip Ashton Skip.ashton@ember.com 617 951 1201 www.ember.com 21