IBR-DTN in IEEE based wireless networks

Transcription

1 Diploma thesis Diplomarbeit IBR-DTN in IEEE based wireless networks cand. inform. Stefan Schmidt Institut für Betriebsysteme und Rechnerverbund Mühlenpfordtstr. 23, 1. OG D Braunschweig Professor : Prof. Dr. Lars Wolf Tutor : W.B. Pöttner and J. Morgenroth Date : Braunschweig,

2

3 Technische Universität Braunschweig Institut für Betriebssysteme und Rechnerverbund Diplomarbeit IBR-DTN in IEEE based wireless networks von cand. inform. Stefan Schmidt Aufgabenstellung und Betreuung: Prof. Dr. L. Wolf, Dipl.-Inform. W. Pöttner und Dipl.-Inform. J. Morgenroth Braunschweig, den

4 ii

5 Affidavit I hereby declare that this thesis has been written only by the undersigned and without any assistance from third parties. Furthermore, I confirm that no sources have been used in the preparation of this thesis other than those indicated in the thesis itself. Erklärung Ich versichere, die vorliegende Arbeit selbstständig und nur unter Benutzung der angegebenen Hilfsmittel angefertigt zu haben. Braunschweig, iii

6

7 Abstract This diploma thesis discusses the implementation and evaluation of an IEEE convergence layer for IBR-DTN with segmentation and neighbor discovery. This includes work on all layers from hardware and drivers to the IEEE Linux stack and the related configuration and performance tools and finally the convergence layer of the delay tolerant networking daemon IBR-DTN. Existing drivers for two IEEE transceiver had to be corrected, extended and partially new written. These drivers have been integrated with the IEEE Linux stack to offer a consistent interface for applications. The combination of driver and stack has been evaluated with regard to round trip time, frame loss and throughput with a newly written performance utility. Furthermore, the related configuration utilities have been extended with the option to set up a node to communicate with other nodes running the Contiki IEEE implementation. A new architecture for the IBR-DTN convergence layer for IEEE has been designed to cope with the limitations of an earlier prototype. The new architecture offers support for segmentation of bundles larger than the Maximum Transmission Unit (MTU) of IEEE Bundles are transparently segmented, transmitted and assembled in the convergence layers. A neighbor discovery mechanism for dynamic discovery of nodes has been implemented based on the IP neighbor discovery IETF draft. Evaluation of round trip time, bundle loss and throughput have been repeated for varying bundle payload. Additional factors such as compressed bundle header encoding have been added to the evaluation of the IBR-DTN convergence layer. The evaluation closes with a comparison of the results against the prototype version. Bottlenecks have been revealed and potential solutions proposed. In the final conclusion the work and potential future work items are discussed. v

8

9

10

11 Contents 1 Motivation 1 2 Introduction IEEE Delay Tolerant Networking Wireless Sensor Networks Related Work Other DTN Implementations Other non IP based Convergence Layers Hardware Setup Imote2 Processor Radio Board CC2420 IEEE Transceiver USB Host and SD Card Extensions WiFi USB Transceiver T-Mote Sky Qi Hardware ATUSB Transceiver Base System Linux Kernel IEEE Stack Development Setup with KVM Lowpan-tools Configuration Utilities AUTO ACK Improvements Compatibility with the IEEE Contiki Implementation Drivers CC Reliable Transmit Automatic ACK ATUSB Driver AT86RF231 Improvements IBR-DTN Modules ix

12 Contents 7.2 Convergence Layer IEEE Convergence Layer Architecture Interface with the Kernel Stack Configuration Discovery Addressing Compressed Bundle Header Extension Segmentation Acknowledgements Compatibility with µdtn Evaluation Automated Test Suite Lowpan-perf an IEEE Test-Suite Delay Tolerant Networking Test-Suite IEEE Round-Trip Time IEEE Throughput IEEE Packet Loss DTN Round-Trip Time DTN Throughput DTN Bundle Loss Comparison with Earlier Measurements Conclusion Future Work Literature 67 A Tools 69 A.1 IEEE Sniffer A.2 DTN Performance Tools A.3 IEEE Evaluation Tooling B Code 75 B.1 Lowpan-perf Code B.2 IBR-DTN Convergence Layer Code Acronyms 99 x

13 List of Figures 2.1 Layer diagrams with IEEE and supplementary protocols Top and bottom view of the Processor Radio Board [1] Air Live USB WiFi transceiver with antenna connectors Top view of the T-Mote Sky board Top view of the Qi Hardware ATUSB transceiver Architecture of the IEEE Linux Stack Wireshark flow-graph view for IEEE association IBR-DTN event handling and modules [2] Class diagrams for the architecture Receiving side of the new IEEE CL in IBR-DTN Transmitting side of the new IEEE CL in IBR-DTN Bit positions of the DTN LoWPAN segment header Bit positions of the DTN LoWPAN extended segment header RTT average IEEE with different payload sizes RTT median IEEE with different payload sizes and AACK, ARET disabled RTT median IEEE with different payload sizes and AACK, ARET enabled IEEE throughput with different payload sizes DTN round trip time with dtnping with payload sizes from 100 to DTN round trip time with dtnping with payload sizes from 1 to DTN round trip time with dtnping with payload sizes from 1 to 100, but with 46 to 49 filtered out Median DTN throughput with different options xi

14 List of Tables List of Tables 9.1 Throughput measurement results Ratio between the size option of dtnping and the number of segments for the bundle RTT results for old and new measurements xii

15 1. Motivation This thesis bridges the gap between two different research areas. Wireless Sensor Networks (WSNs) on one hand Delay Tolerant Networkings (DTNs) on the other. Small and cheap sensor nodes collect information about the environment they are located in. Examples for such sensors are ranging from measurement of air pollution in cities over alarm systems detecting forest fires to herding behavior research of animals. The wireless part in WSNs together with an independent power source like batteries or solar power allows the sensor nodes to freely move during their lifetime. This possible movement brings in different types of problems. Energy efficiency is an important topic for sensor nodes. Conserving power increases the lifetime of the nodes which is important. With IEEE a specification for low power wireless connectivity is defined and transceivers based on the specification are used on sensor nodes. The IEEE wireless link is often the only connection to the outside world. Connectivity is not always available and in combination with DTN the data could be stored and later forwarded when connectivity is restored or other nodes are available to forward the data. With IBR-DTN a lightweight and efficient DTN implementation is available for embedded Linux systems. A typical sensor node, like the T-Mote Sky, has way too little system resources, such as available memory and CPU power, to run a full Linux system. Not even µclinux, a small Linux for embedded devices, is feasible in most cases. Some operating systems targeting such small devices are Contiki or TinyOS. Neither is able to run IBR-DTN. With the Imote2 a good hardware platform is available that combines the small form factor of WSN nodes with enough power of an ARM CPU and memory to run a full Linux system and also offering the needed IEEE transceiver. This combination makes it possible to run IBR-DTN on the Imote2. Interconnecting IEEE based wireless-sensor networks with traditional IP based ones could be done by enhancing the Imote2 with a WiFi transceiver, based on IEEE [3], and enabling IBR-DTN to listen on both interfaces. With such an interconnect the bundles can be routed from DTN nodes in IP based networks to nodes in IEEE and back. Through the interconnect sensor data would reach the hosts responsible for the analysis of the data and administration requests or software updates would get back to the sensor nodes. For all these operations no constant network connectivity needs to be available. The store-and-forward concept of DTN allow the data to reach its destination even when no direct connection between source and destination is available. IBR-DTN has Convergence Layers (CLs) to adapt to different underlying network technologies on which the Bundle Protocol (BP) operates. For IEEE a specific CL is implemented 1

16 1. Motivation directly on the Media Access Control (MAC) layer the standard provides. Doing so avoids other protocols to sit in between IEEE and the BP. With a MTU of only 128 byte the IEEE specification allows for very small frames and adding the overhead of different protocols in between would decrease the throughput notably. Using DTN mechanisms in a WSN would allow the network to be more robust against temporary disconnects and other network troubles. The data would still be available even if the network connection is only sporadically available. Furthermore, sensor nodes could save power by keeping the transceiver disabled for long time intervals. Bundles could get exchanged based on a fixed schedule or when enough data has been cached to be transfered. For such cases the transceiver only needs to be enabled for a short time-frame to announce itself to other nodes in range, transfer the stored bundles and maybe receive others. This would keep the power used for wireless connectivity to a minimum and still enable data communication on the cost of delays. If the communication is not time-critical this can be a good trade-off between sensor node lifetime and communication latency. 2

17 2. Introduction In this thesis, research about DTN via IEEE Low-Rate Wireless Personal Area Networks (LR-WPAN) a a practical implementation of a DTN CL for IBR-DTN 1 directly on the MAC layer of IEEE will be shown. In chapter 1 the motivation and areas of possible research are shown. This is followed by the current chapter introducing the various technologies needed for this thesis followed by some research on related work about the different topics in chapter 3. Chapters 4 and 5 give an overview about the used hardware and describe the base system used for development. Improvements of the hardware drivers needed for the IEEE communication are documented in chapter 6. Before explaining details of the implementation of the CL in chapter 8 a general overview on IBR-DTN is given in chapter 7. Finally, an evaluation is presented in chapter 9 followed by a conclusion and potential future work in chapter 10. To give the reader an easier start into the work of this thesis a short introduction to the various technologies needed can be found in the subsections of this chapter. The general idea and main features of IEEE are explained in section 2.1 while the functionality of DTNs is introduced in section 2.2. The concept of WSNs is briefly documented in section IEEE The IEEE specification for LR-WPANs is discussed, formulated and specified by an IEEE working group. The main objectives are low power consumption as well as low cost to allow for widespread usage with little or no infrastructure. This comes with a tradeoff on speed and connectivity range. As the focus is on communication between devices and not between users such a tradeoff is admissible. Figure 2.1 shows how a complete network stack based on IEEE could look like. Protocols like ZigBee, IPv6 over Low power Wireless Personal Area Networks (6LoWPAN) as well as DTN are able to complete the IEEE specification to a full network stack. It also shows that only the lowest two layers are defined by IEEE specification. Nothing above the physical and the MAC layer is specified in IEEE This allows for flexibility but can also introduce the need for additional protocols on top. Routing is such an example. Like in Ethernet only nodes within the broadcast domain are reachable in IEEE without an interconnect to other networks. Such a functionality has to be implemented within the 1 3

18 2. Introduction ZigBee 6LoWPAN DTN IEEE MAC layer IEEE physical layer Radio hardware Figure 2.1.: Layer diagrams with IEEE and supplementary protocols supplementary protocols if needed. Addressing on the other hand is specified and allows for nodes nearby to send plain IEEE frames to these addresses and exchange data without routing. Looking at the possible network topologies LR-WPANs can work either in a star topology with a central node coordinating the traffic as well as in a peer-to-peer topology where peers can exchange data directly. A combination of both methods is also possible. Such a combination is labeled as cluster tree in the specification and describes the combination of several Personal Area Networks (PANs) into a mesh network. In the cluster tree topology the respective PAN coordinator relays the frames to the connected PANs. The standard defines two types of devices that can be part of the network. Reduced Function Devices (RFDs) are only capable of basic radio operations like sending and receiving frames directly addressed to themselves. Such devices are usually only used in really low-power environments and only as leaf nodes in the network topology. More widespread are Full Function Devices (FFDs). They cover the full specification and can also operate as PAN coordinator. In IEEE such a PAN coordinator is responsible for operating the PAN available to other devices. This is done by sending out beacons for the available PAN and hand out addresses once another node associates with the PAN. This can be compared with a WiFi access point handing out IP addresses via DHCP. The coordinator can also relay frames for other devices to support a star topology or to aid RFDs. Other features covered by the specification are Carrier Sense Multiple Access with Collision 4

19 2.2. Delay Tolerant Networking Avoidance (CSMA/CA), Acknowledgements (ACKs), retransmissions, reserved time slots for real-time use and optional security through cryptography. Furthermore different physical layers for different frequency bands as well as different modulations are specified Delay Tolerant Networking The concept of DTNs originated from research departments around the aerospace industry. A network link between different planets exposes problems of delayed transfer that common network protocols can not cope with. Since the initial research the topic got more interesting for others, especially the possible uses in disruption tolerant networks have been investigated. The problem with many networking protocols is that they can only exchange data when a connection from the source to the destination is actually established. It does not matter over how many hops this connection is routed, but it has to be there for the full time of the data exchange. Such a reliable end-to-end network connectivity is not always given. Be it due to movements, disruptions of a wireless link or other circumstances. With the DTN concepts such an established connection is not needed at all. One of the basic ideas in DTN is the idea of store and forward. Before storing the bundle the payload is added and the destination address attached. Once a connection to another node becomes available the bundle gets forwarded until it eventually reaches its destination. Various flavors of this idea can be implemented. Custody transfer is one of it. It forwards the responsibility together with the bundle to the next node. Together with status reports on delivered bundles or in case of errors this allows a reliable transfer of bundles. Other simple mechanism such as flooding the bundle to every reachable node are possible to increase the probability of the bundle reaching its destination Wireless Sensor Networks The term WSNs describes a range of devices operating in low-power mode for a long time with no maintenance required. Collecting sensor data is the main objective. The collected sensor data can range from environmental monitoring in urban areas over process and machine monitoring to heartbeat and general health monitoring. What they all have in common is the constraint on available power, various sensors to collect data and a wireless link to transfer the collected data. Often the time at which collected data needs to arrive at the destination is not critical and a wireless link to the destination is not always available. Such a scenario makes the usage of DTN interesting in WSNs. Due to its low power requirements IEEE transceivers are often used in such sensor nodes. And that is exactly what brings those two research topics together in this thesis. Allowing IBR-DTN to operate in IEEE based WSNs opens up new 5

20 2. Introduction possibilities for the concept of DTN as well as different data transfer strategies for WSNs. A sensor node powerful enough to operate with IBR-DTN will not be as power efficient as the smaller nodes collecting the data, but it will be able to act as boarder router between the WSN and other wireless technologies like WiFi. Such an interconnect would ease the administration and pickup of collected data from the sensor nodes. 6

21 3. Related Work In this chapter other research work related to the topic of this thesis is introduced. Research on the given topic as well as finding related research topics was the first phase of work on this diploma thesis. In section 3.1 an overview is given about other available DTN implementations. Section 3.2 then looks at other CLs not based on IP. Already in 2003 data MULES [4] have been discussed as a three-tier architecture for sparse sensor networks. The authors define a store and forward architecture of sensor data being picked up and delivered by so called mobile agents from the wireless sensors to an access point. The generic idea of wireless sensor nodes with limited power and radio coverage, fixed installments of access points and mobile agents handling the actual data transfer is something that does match the offerings DTN has. Often the collected data of the sensors is not time-critical and would be better delivered over a more power efficient mechanism to allow for a longer life-time of the sensors. In such cases the collected data could get packed together into bundles, stored, and forwarded once the next mobile agents shows up. With custody transfer it would be possible to assign the responsibility of the bundle to the mobile agents which in turn would store and later deliver the bundle at the access point. The notification of the delivered bundles would reach the sensor node with one of the next mobile agents appearing in connectivity range Other DTN Implementations Three major software projects are implementing the BP [5] and related routing protocols, CLs, and more for Linux. The BP defines the format of the data unit, called bundle, used in DTNs. DTN2 [6] is the reference implementation from the Delay Tolerant Networking Research Group (DTNRG) at the Internet Research Task Force (IRTF). It is the successor of the first reference implementation DTN1. DTN2 aimed for a re-write in C++ with standard template libraries to sort out different problems with DTN1. Namely, the lack of internal interfaces to modularize and extend the code base as well as over optimized code and data structures made the code hard to understand and maintain. Looking at the CLs it has a main focus on TCP and UDP, like almost all DTN implementations, but offers other CLs as well as can be seen in section 3.2. With Interplanetary Overlay Network (ION) [7] there is another implementation of the DTN ideas and the BP in particular. In contrast to DTN2 or IBR-DTN it has a narrower focus with regard to the target architecture and what limitations the environment imposes to the software. ION was specifically designed to run in interplanetary spacecrafts as well as in flight control 7

22 3. Related Work centers. The imposed limitations range from statically allocated memory blocks at boot instead of dynamic allocations to the need for highly efficient bandwidth utilization for the slow wireless links between the spacecraft and the control center. The demand for efficient bandwidth usage was also the force behind the invention of the Compressed Bundle Header Encoding (CBHE) [8] which was first implemented and used in ION. In this regard ION has some of the same requirements as DTN in WSNs have. With IBR-DTN [2] the third major DTN implementation is at the same time the basis for the work of this thesis. Details about IBR-DTN can be found in chapter 7. Smaller implementations or implementations for special use cases are PyDTN 1, Java BP-RI 2, DASM [9] and JavaDTN Other non IP based Convergence Layers The main focus in this thesis is less on the general BP implementation but more on the actual CL and the lower level protocol used to transfer the bundles between nodes. Naturally, the TCP and UDP CLs are the ones with the most users. They are easy to test and TCP/IP knowledge and networks are widely available. None the less some other research can be found on the topic of non IP based DTN CLs. The above mentioned DTN2 implementation has support for AX25, Ethernet as well as Bluetooth. The Bluetooth CL is based on the Radio Frequency Communication (RFCOMM) profile and provides a socket interface over the serial port emulation. The Ethernet CL implementation sits on top of a PF PACKET family socket with the SOCK RAW type providing raw network protocol access. DASM [9] is another DTN implementation focused on the use on Symbian based smartphones. The main research area here are Ad-Hoc networks as backup or addition to infrastructure-based communications. DTN is used to store and forward bundles between the different Ad-Hoc nodes. DASM has CLs for WiFi and Bluetooth. The WiFi CL is built on top of TCP, but the Bluetooth CL is based on RFCOMM like the Bluetooth CL of DTN2. DTNLite [10] is an implementation for TinyOS and is loosely based on the idea of the DTN overlay architecture. It uses custody transfers to offer reliable message delivery over a network of sensor nodes. The key research aspect is the identification and selection of the next custody hop on the way to the destination. While the DTNLite implementation is loosely based on the DTN architecture it does not conform to the BP and thus isn t able to exchange bundles with other implementations. This makes it unusable for this thesis as interoperability is a high priority. The paper does not cover the details of the radio link and its interface to the CL. However, it

23 3.2. Other non IP based Convergence Layers was mentioned that the Berkeley MICA motes 4 have been used. What was found about the MICA motes suggests that they use a radio with an 916 MHz ISM transceiver. While the IEEE specification covers such an ISM band it is unlikely that the MICA motes are actually using IEEE ContikiDTN [11] is a DTN implementation for the Contiki OS. Its only CL is bound to TCP due the underlying usage of the µip stack and its protosockets library limitations. No usage reports of ContikiDTN with an IEEE CL have been found. The description suggests that no other CLs can be implemented to work together with ContikiDTN as the TCP CL was integrated directly into the daemon handling the bundles. It was not possible to investigate how much work it would be to split the CL handling out of the daemon and thus allow for other CLs as the ContikiDTN source code was not made available to the public. This also prohibits experimentation or evaluation of the implementation. The last implementation discussed here is also based on the Contiki OS and aims for basic functionality of a modular DTN system. The µdtn [12] project offers different modules to handle routing, storage, discovery, custody, and other elements. The bundle agent as well as the services offered to applications and the CL are built in. The CL is implemented directly for IEEE instead of using additional layer 3 and 4 protocols like IP and TCP. So far the basic work has been done to discover nodes, send and receive bundles as well as store and forward them. CBHE is implemented as well and some continuing work has been started to optimize different areas of the implementation as well as increasing compatibility with IBR- DTN. In sections 5.5 and 8.9 the compatibility between the µdtn implementation and the CL developed for this thesis are discussed

24

25 4. Hardware Setup During this thesis various hardware was used. This chapter explains the used devices and their use-cases allowing the reader to follow the explanations in the following chapters easier. To operate IBR-DTN within an IEEE network at least a transceiver for the radio interface is needed. With ATUSB described in section 4.4 such a transceiver is available as a small Universal Serial Bus (USB) device which can be connected to a normal computer with Linux as operating system. A fully integrated hardware solution to operate IBR-DTN on a sensor node is what the Imote2 can offer. The Imote2 as well as the radio transceiver on it are explained in section 4.1 and respectively. Additional hardware components like a USB WiFi transceiver are described in section 4.2 and another sensor node in section Imote2 Processor Radio Board The Imote2 [13] platform from Memsic (formerly from Crossbow) is a modular WSN hardware. Originally designed by Intel and Crossbow the platform allows to combine different sensors and other hardware components with a base board. In contrast to other WSN hardware the Imote2 is powerful enough to run a full Linux system. While the Imote2 platform has different boards available to extend the Processor Radio Board (IPR2400). None of them were used in this thesis. Only the Interface Board (IIB2400) was used to gain access to the serial console and the JTAG interface of the hardware. The platform is equipped with a PXA271 XScale ARM System on Chip (SoC) running at a CPU frequency from 13 to 416 MHz. Additional hardware features are 32 MB of SDRAM as well as 32 MB of flash storage. Next to these main components the SoC features many different peripherals either directly integrated on the board or available through the basic and advanced connectors. While the CC2420 IEEE transceiver is directly connected on the board over a Serial Peripheral Interface (SPI) and some General Purpose Input Outputs (GPIOs) the connectors are offering advanced connections like USB host, Secure Digital Input Output (SDIO), Inter-Integrated Circuit (I2C) and more for the extension boards. The IPR2400 itself has the USB client interface exposed for connections with a host computer. During the development this interface has been used to transfer data and supply the device with power. 11

26 4. Hardware Setup Figure 4.1.: Top and bottom view of the Processor Radio Board [1] Figure 4.1 is a top and bottom view of the IPR2400 showing the antenna (1), the CC2420 (2), the board connectors (3) and the USB connector (4) on the left and the bottom side with the PXA processor (5) and power management chip (6) on the right [1] CC2420 IEEE Transceiver The transceiver used on the Imote2 is a CC2420 [14] from TI (formerly ChipCon). In IEEE terms the CC2420 is a FFD and thus able to operate as as normal node as well as coordinator of the PAN. Out of the three specified bands available in IEEE the CC2420 only supports the 2.4 GHz band. With channels from 11 (2.405 GHz) to 26 (2.480 GHz) all 16 channels on the band are supported. Data rates up to 250 kbit /s are theoretically possible. The CC2420 is connected to an SPI port of the PXA271 as well as to some GPIOs for signalling and interrupts. The chip itself features a First In First Out (FIFO) buffer for each direction. The size of 128 byte per FIFO does match the MTU of an IEEE frame. Therefore the chip allows to buffer a full frame before pushing it to the host when receiving or buffering it before transmission. Interrupts are issued once a Start Frame Delimiter (SFD) has been received in the FIFO. This interrupt triggers on both the receive as well as the transmit FIFO. The receive FIFO has one more interrupt which gets issued once either a full packet is available in the FIFO or if the filling state of the FIFO reaches a certain threshold. This threshold is adjustable. The receiving FIFO also offers a GPIO which goes high after more than one byte is available inside the FIFO. This GPIO is called FIFOP. Like the threshold adjustment for the FIFOP interrupt there are other parameters that can be configured in registers. Configuration options for power saving or range extension are other parameters that could be useful in later deployments. Like other packet-based radios the CC2420 implements most of the physical layer, as well as parts of the data link layer in hardware. In particular the CC2420 offers CRC calculation and 12

27 4.2. WiFi USB Transceiver verification, Clear Channel Assessment (CCA), auto ACK, link quality indication as well as data encryption directly in hardware. Due to this extensive hardware support the software complexity is reduced and allows the CC2420 to be used even on smaller and cheaper micro-controllers. As disadvantage one could see the lack of an automatic re-submit of frames if an ACK is missing. Other transceiver chips like the Atmel AT86RF231 offer this feature. Another problem might be that the CC2420 only supports the 2.4 GHz band. The 2.4 GHz Industrial Scientific and Medical (ISM) band is free to use for all kinds of devices. Popular technologies like WiFi and Bluetooth are using this band as well as baby monitors, home streaming solutions and others. This results in a crowded frequency spectrum and possibly bad performance. The support for the other specified bands from IEEE would allow for flexibility here. One has to keep in mind here that the other bands are only specified for slower data rates USB Host and SD Card Extensions The Imote2 platform does not have an extension board with a WiFi transceiver available. To route between these different wireless technologies a WiFi transceiver would need to be added to the Imote2. The basic and advanced connectors of the board are offering a range of communication interfaces for such a transceiver. Interfaces like SPI or SDIO are a good choice if a complete new board should be designed containing the WiFi transceiver and its antenna. The Imote2 advanced connector also offers a USB host interface. The decision was made for the USB host interface as it allowed to use off the shelf USB transceiver dongles instead of building a complete new board. For the prototyping purpose of this task the solution which could get completed in a shorter time-frame was more sensible. The WiFi transceiver used is described in section 4.2. The choice to use the USB host interface for the WiFi transceiver also left the Secure Digital (SD) interface available for SD memory cards. With only 32 MB of internal flash the Imote2 is well equipped for a node in a WSN, but not prepared for storing larger bundles that can appear in DTNs. Using the SD interface for memory cards allows for a cheap storage extension of up to 16 GB. The usage of SD memory cards was not implemented during this thesis, but could be a useful addition for future work with this hardware WiFi USB Transceiver The choice of the WiFi transceiver was driven by different demands. The most important one was the need for a USB device with a driver available in the mainline Linux kernel. USB is the hardware connection available and a driver already in the mainline Linux kernel reduced the efforts to bring the device into a working state. Two other demands were likewise connected. To allow for a good performance of the WiFi transceiver an external antenna connector as well as support for the 5 GHz spectrum is beneficial. The IEEE a specification describes the use of the 5 GHz spectrum for WiFi. This 13

28 4. Hardware Setup avoids the overcrowded 2.4 GHz spectrum. Given the fact that the CC2420 operates in that spectrum as well a separation of the radio signals on two different bands helps to avoid interference between the two transceivers. The external antenna connector can help to improve the performance further if a longer distance needs to be bridged. Those four demands have reduced the market of available devices. Especially an external antenna connector is not common on consumer electronics like USB WiFi transceivers. In the end an AirLive X.USB-3 was chosen as it was available and fulfilled all requirements. It offers two R-SMA antenna connectors as well as dual band radio. The internal WiFi chip is an Atheros 9170 which is actively supported by Atheros itself in the mainline Linux kernel. On figure 4.2 one can see the whole USB device with one antenna connected and the second one taken off to show the antenna connector. Figure 4.2.: Air Live USB WiFi transceiver with antenna connectors 4.3. T-Mote Sky The T-Mote Sky is based on the MSP430 micro-controller and is widely used in the Contiki community based around WSNs. In this thesis the main use-case for the T-Mote Sky was sniffing the air traffic between different devices to monitor and debug the traffic. Capturing directly on the air interface allowed to isolate the debugging from potential problems in the software stack itself. To use the T-Mote Sky as a sniffer it has to be flashed with a special firmware which sets the device into promiscuous mode and routes all received frames over the 14

29 4.4. Qi Hardware ATUSB Transceiver serial link to the host computer. On the host these frames are injected into Wireshark 1 for further exploration. The method is described in [15] and more information about setting up the capturing can be found in annex A.1. The T-Mote Sky is also used as the main development device for the µdtn implementation. Compatibility between µdtn and the implementation described in this thesis is aimed for and details can be found in section 5.5. Figure 4.3 shows a top view of the T-Mote Sky board. The chip labeled with 1 is the CC2420 transceiver with an on-board antenna on the right and an external antenna connector below. The MSP430 micro-controller is placed on the bottom of the board and is not shown here. Figure 4.3.: Top view of the T-Mote Sky board 4.4. Qi Hardware ATUSB Transceiver All devices already mentioned are using the CC2420 IEEE transceiver from TI. It is built into the Imote2 as well as the T-Mote Sky devices. To test our software on other transceivers and allow for interoperability with other devices a second transceiver has been added to the test setup. The ATUSB IEEE USB transceiver consists of an AT86RF231 [16] IEEE radio and an ATmega32 micro-controller to intermediate between the USB on one and the SPI communication on the other side. For the host computer the device offers the interface of the IEEE radio over USB and thus allows to use the IEEE Linux stack as well as IBR-DTN with the IEEE CL on a normal Linux host instead of an embedded device. The AT86RF231 from Atmel is in many aspects comparable with the CC2420. It supports automatic hardware ACKs, CRC calculation and other hardware acceleration features. It is also bound to the 2.4 GHz ISM band. It does however only have one FIFO shared between the receiving and transmitting states. On the other hand it also supports automatic resubmission of frames if an acknowledgement is missing in the standardized period of time or when a CCA

30 4. Hardware Setup fails. This allows CSMA/CA compliance directly on the hardware level as it also allows to set the back-off time interval and number of re-transmits. Like the CC2420 the AT86RF231 is a FFD in IEEE terms and thus able to operate as a PAN coordinator and relay frames for other nodes. The ATUSB is designed as open hardware. The schematics and other hardware description files are available under a liberal Creative Commons Attribution-ShareAlike 3.0 Unported license 2 license and allow easy modifications of the original hardware. The firmware running on the micro-controller is available under an open source license and allows modifications on a very low level of the system, the USB interface to the host PC. These circumstances make the hardware interesting for research as it allows to understand all details without having to deal with the support of the hardware vendor to gather all needed information. On figure 4.4 a top view of the ATUSB board is shown. The chip labeled with number 1 is the AT86RF231 radio transceiver and number 2 is the Atmel ATmega32 micro-controller. The antenna is printed directly on the Printed Circuit Board (PCB) and can be seen on the left side of the board. Figure 4.4.: Top view of the Qi Hardware ATUSB transceiver

31 5. Base System This chapter covers all software components that have been modified to support IBR-DTN with the IEEE CL. The implemented CL in IBR-DTN uses a classic socket interface to communicate with the IEEE Linux stack. The Linux stack itself is composed of hardwarespecific drivers, a MAC implementation to provide functionality not available in the hardware and a socket interface that user-space programs like IBR-DTN can use. A description of the involved components is given in the following sections Linux Kernel IEEE Stack The starting point for all driver modifications is a recent Linux kernel based on version It offers good hardware support combined with a stable and well maintained code base. The hardware drivers as well as the IEEE networking stack are implemented within the kernel. They can be used either as loadable kernel modules or directly built into the kernel. The groundwork for an IEEE Linux implementation was done by the Linux-ZigBee 1 project. The project name is misleading and based on the historical idea to write a complete ZigBee implementation. This turned out to be impossible as the ZigBee forum settled for a license agreement which is incompatible with the GNU General Public License (GPL) open source license under which the Linux kernel is released. The project s focus shifted to implement only the open standards in the same area, IEEE and 6LoWPAN. Most of the original code base of the Linux-ZigBee projects was contributed by Siemens and currently initial support for 6LoWPAN under Linux is being developed by Siemens as well. So far, the Linux-ZigBee stack is not yet fully integrated into the Linux kernel. Especially the software MAC implementation will need more work until it is completed and ready for a mainline merge. This delays the integration of other parts like the hardware drivers as well because in the current form the hardware drivers are dependant on the MAC implementation. Decoupling these two parts will help to bring the more mature hardware drivers into mainline and provide a working implementation while keeping the MAC implementation separate until it is completed. Three components form the core of the current IEEE Linux implementation. They can be seen in figure 5.1 together with the user-space lowpan-tools configuration utilities and IBR-DTN as an example application using the socket interface as well as at86rf230 as example hardware driver

32 5. Base System lowpan-utils set() / get() IBR-DTN send() / recv() ieee af_ieee xmit() recv() mac start() stop() set_channel() set_hw_addr_filt() at86rf230 Figure 5.1.: Architecture of the IEEE Linux Stack The af ieee module implements the IEEE socket address family. It is used by all applications for data exchange with the stack and later the radio. Outbound data from an application is put into a work queue and scheduled for transmission. Inbound data from the radio is delivered through af ieee to the application. All aspects related to the physical layer are controlled by the ieee module. Setting the correct channel and setting the hardware address filter are two examples of configuration parameters the ieee modules uses to configure the different hardware drivers. While the initial parameters are set by the module itself, it offers a netlink based interface for other applications to get and set these parameters. This functionality is used by the lowpan-tools configuration utilities and further explained in section 5.3. The interface is not limited to the configuration utilities. Other applications can use this interface as well. IBR-DTN relies on it to read out the current short address as well as the configured Personal Area Network Identifier (PAN ID). The last component of the IEEE Linux stack is called mac It was intended to provide a full software implementation of the IEEE MAC layer. Such a software implementation would provide software replacements for missing functionality the hardware MAC layer implementations may have and would also support radios that offer not much more than the basic physical layer. In its current state the mac module offers none of this functionality. Software ACKs as well as software CSMA/CA as two major components of such a software MAC layer that are not yet implemented neither are other components. If one considers that most if not all IEEE transceivers have the full MAC layer implemented in hardware 18

33 5.2. Development Setup with KVM it is questionable if the software implementation is needed at all. For possible future transceivers which do not implement hardware ACKs it is still unclear if a software implementation could meet the timings required by the specification Development Setup with KVM Details for the driver development of the ATUSB device can be found in section 6.2. Hardware drivers in Linux are not separated from the rest of the kernel. This required a robust setup to test early versions of the driver without the risk of crashing the development system and possible losing some data in the process of developing the driver. With the Kernel Virtual Machine (KVM) 2 the Linux kernel offers a full virtualization solution which offers pass-through devices to the virtual machine. The system inside the virtual machine has full control over the USB device which allows the development and testing of a driver for the device. In the initial stage of the development of the ATUSB driver, this setup allowed to debug crashes of the driver and kernel without bringing down the host system. The fast reboot times and the shortened testing cycle aided the debugging process. In the later stage a second virtual machine on the same physical machine has taken over the role of the second node. Again the second ATUSB device was passed to the virtual machine. To avoid interference between the two virtual machines the two ATUSB devices where connected to different USB buses. All further development on the IEEE CL as well as the evaluation was done with this setup Lowpan-tools Configuration Utilities While the IEEE Linux stack is implemented within the Linux kernel, the configuration and test utilities are bundled in the lowpan-tools package. They cover everything from channel, PAN and address setting to more advanced tasks like operating as PAN coordinator. In figure 5.1 it is possible to see that all configuration operations are executed via the ieee module of the kernel implementation. The nl abbreviation in the module name refers to netlink 3, a message based communication protocol often used to interface between kernel and user-space for configuration. Other network related system configuration tools like iproute2 using it as well. Configuration of low-level properties like radio channel and PAN ID can be done with the iz application of the lowpan-tools package. It also handles the association requests when a node tries to join an available PAN. To operate a FFD as apan coordinator, the izcoordinator application is included. It handles incoming association requests, hands out short addresses valid in this PAN and keeps track of already assigned addresses with leases. The lease file allows keeping track of the different nodes inside the PAN and their assigned addresses over a reboot

34 5. Base System Figure 5.2.: Wireshark flow-graph view for IEEE association In figure 5.2 a packet trace of a successful association request and response is shown. The flow graphs shows the initial association request with the long address set as source address as no short address was assigned yet. The association response contains the assigned short address for the node and in the following data packets only short addresses are being used AUTO ACK Improvements Transmitting data over a wireless link is always prone to collisions and other interferences on the air interface. While a MAC layer, like the MAC layer of IEEE , may use mechanisms like CSMA/CA to avoid collisions, it can not be guaranteed that frame will arrive at its destination. Other radio transmitters may use the medium at the same time without testing if it is already in use. For moving nodes, physical limitations like wireless connectivity range are also reasons for packet loss. To provide a reliable service to the upper layer, the data-link layer of IEEE describes an ACK mechanism for frames. Only acknowledged frames can be considered delivered to the destination. IEEE defines stringent response time constraints for ACK messages. For the 2.4 GHz DSSS PHY this time is up to 54 symbols which equals 864 µs for an acknowledgement frame to arrive after the corresponding data frame has been transmitted. Given this response time limit required by IEEE the different transceiver chips often offer the option to ACK the frame directly in the hardware. This feature is called automatic ACK. Before the hardware is able to ACK the incoming frame several checks of the frame data need to be done. A first check is done to verify if the received data is actually intact. This uses a hardware check of the Frame Check Sequence (FCS), in IEEE a 16 bit Cyclic redundancy check (CRC), is done. After a successful FCS test it will be verified if the destination matches its 20

35 5.5. Compatibility with the IEEE Contiki Implementation own address. Thus, the hardware needs information about the addresses associated with the local node. Due to the dynamic nature of the associated addresses this information needs to be provided by the upper layers to the hardware. In most cases this is done by writing the current addresses and PAN ID into hardware registers or RAM. The information is updated on every change of the properties. This way the hardware can rely on the registers holding current data and match the frame destination address with the ones stored in the registers. If the address matches, the hardware generates an ACK frame for the sequence number used in the incoming frame. This ACK is send back to the original sender of the data frame. If a sender node does not want to receive an acknowledgement it can set the acknowledgment request subfield to 0. This is the case for beacon frames as well as broadcast frames. In [17] it was shown that it is possible to conform to the IEEE specification even when doing the ACKs in software. Yet, an micro-controller operating system like Contiki is quite different from the general purpose Linux operating system used in this thesis. Achieving the required response times using a normal, non-realtime Linux kernel seems unlikely. The goal of piggy-backing additional data in the ACKs, as described in the paper, is specific to the method of energy saving and does not apply here Compatibility with the IEEE Contiki Implementation The IBR-DTN CL is aiming for compatibility with the µdtn implementation on Contiki [12]. To reach this goal, several aspects need to be considered and implemented. One difference on the IEEE communication layer in Contiki is that it does not run the PAN with a coordinator. Due to this no short addresses are handed out automatically. Instead the short addresses are set manually for every node in Contiki. The default lowpan-tools on the other hand do not offer the possibility to set the address manually. They completely rely on the PAN coordinator to provide the short address and assign this address after receiving the association response from the coordinator. Running the coordinator on a node operated with lowpan-tools is impossible as Contiki does not perform the required association request. To improve this situation, support for setting all required parameters manually on the command line was added to lowpan-tools. The functionality to set the PAN ID as well as the channel was already used in other commands while the option to set the short address was completely missing. To ease the setup of such a configuration a new set command was introduced into the iz command line configuration utility. The set command takes the interface to operate on as first parameter followed by the properties to set. PAN ID, short address and channel is this case. This allows configuration of all needed parameters in one command line. Supplied with this information it was possible to send and receive IEEE frames between a node operated by Contiki and a node operated by the IEEE Linux stack. For further 21

36 5. Base System considerations relating to BP and CL compatibility between IBR-DTN and µdtn see

37 6. Drivers In section and section 4.4 the hardware transceivers used in this thesis have been introduced. The introduction gives an overview about the hardware features available on the transceivers. In this chapter a follow up is given with an emphasis on the driver written to use the hardware under Linux. The chapter starts with the CC2420 driver in section 6.1 and the improvements to this driver implemented during the work on this thesis in section Followed by the description of the newly written driver for the ATUSB hardware in section 6.2 and the related improvements to the AT86RF231 driver in section CC2420 A driver for the CC2420 IEEE transceiver is already included in the IEEE Linux stack. While the originally included driver was able to communicate with the chip over the provided SPI interface, it had many small problems that made it impossible to send or receive data from an application over the transceiver. In [1] the problems with the highest priority had been repaired. The send as well as the receive path between application and driver had been improved and a basic functionality for data transfer has been established. In the end some problems remained open and aimed to be addressed here. One effect was the unreliable sending of frames. A delay of 100 ms was added in the application to make the data transfer more reliable. Considering that under ideal circumstances a frame with the maximal size would only need 4 ms to be transferred over the air interface this delay is quite high and added significantly to the bad performance due to the high latency. But even with this workaround the packet loss was higher then usual indicating that still some problems with sending remain. Investigating the actual root cause of this problem was part of the work on this thesis. The findings, proposed and implemented solutions and failures are described in the following section Reliable Transmit Approaching the CC2420 driver to fix all outstanding problems avoiding a reliable transmit of frames was one of the first major tasks during this thesis. Having an unreliable combination of hardware and driver easily confuses the higher layers and make the development and debugging more problematic later on. 23

38 6. Drivers The CC2420 offers two different ways to send off a frame over the radio. In both cases the data gets filled into the TXFIFO before signalling the chip that the data is ready to send. Two different signals, called strobes in this context, are possible for sending. The STXON strobe is issued if the frame should be send over the air interface without testing the channel in advanced. The alternative is the STXONCCA strobe which tests if the channel is busy before sending and ignores the request if the channel is already in use. The former is not compliant with CSMA/CA used in IEEE To implement CSMA/CA the STXONCCA strobe has to be used to archive the demanded CCA behaviour. With the STXON command strobe it can happen that the frame is corrupted during the transmission or corrupts other frames being transmitted by other nodes at the same time. It therefor should be avoided if possible. To understand if the STXONCCA succeeded or got ignored because the channel being busy during the transmit attempt the status byte offers a TX ACTIVE bit indicating a currently active transmission. Without this information one could only requesting the chip to transmit the data and hope for the arrival without further knowledge about the outcome of the transmit attempt. As this is the only reliable method to transmit data over the radio it was chosen while improving the driver. It is also used in other drivers for the same chip like the driver 1 written for the Contiki operating system. The critical part turned out to be the part to read out the TX ACTIVE bit from the status byte in the short time frame it was actually set. Depending on the frame length this can be the case for different time intervals. With the theoretical speed of 250 kbit /s on the 2.4 GHz ISM band the formula looks like this for the transmit time: payload size in bit/250 kbit /s = time in seconds As the IEEE header already occupies some bytes of the frame independent from the actual payload size the minimal length can be taken as at least 10 byte. The maximum size is defined by the maximum possible frame length which is 128 byte. This offers a transmission time range from 320 µs up to ms from the minimal to the maximal frame size. Ignoring possible hardware latencies this is also the time in which the TX ACTIVE bit is set in the status byte during a successful transmission. The problem now arises with the combination of the PXA271 ARM SoC, the SPI hardware controller on this system and the Linux kernel. It turned out that is was not possible to reach the 320 µs time-limit needed to reliable transfer even small frames. Reading out the status byte from the CC2420 over SPI was not fast enough to meet the timing requirements. Using the getnstimeofday() function inside the Linux kernel it was possible to instrument the relevant code section and measure the time spent to read out the status register after a STXONCCA command strobe. A high variance for the time spend in the spi sync() call was measured during various test runs. Ranging from 300 µs minimum to a maximum of 26 ms. The cluster of measured time arranged 1 f=core/dev/cc2420.c 24

39 6.1. CC2420 itself around 8 to 9 ms for the call. During a test this was not fast enough to allow the driver reliable identify if a frame was transmitted using the TX ACTIVE bit as indicator. Very often a frame was not detected as send but still reached the destination without a problem. The detection worked better for longer frames as the TX ACTIVE bit is set for a longer time interval. The required time-limit of 320 µs for small frames was only reached in less than 1% of the tests. Various tweaks to the involved SPI components have been made to aim for better performance in this use case. The CC2420 is connected to the GPIO pins that can be controlled by either a hardware accelerated SPI controller or directly by software. Aiming for best performance the hardware accelerated mode was used in the beginning. Simply increasing the speed of the SPI clock did not result in any measurable differences. The SPI controller allows for Direct Memory Access (DMA) enabled transfers which is good for chips which are transferring large block of data over SPI like a flash storage device. DMA is enabled by default. Suspecting some overhead for the small number of bytes that need to be transfered for the CC2420 it was disabled and the controller switched into Programmed Input/Output (PIO) mode. While bigger transfers seem to take more time, as one would expect, the timing for all frame size was still above the time limit. The controller has various configuration options for either DMA or PIO mode like setting the transmit and receive thresholds for transfers. None of the settings helped to achieve the time limit during the measurements. As a try of last resort the hardware accelerated SPI controller was disabled to rule out problem with the controller hardware. A software SPI implementation was used to speak the protocol on the connected GPIO lines. While a communication between the driver and the chip was successful in this setup the timing for the status byte reading got worse. To the best of my knowledge about this problem it seems more related to the scheduling and interrupt handling of the Linux kernel instead the actual hardware. TinyOS 2, a special purpose operating system is able to operate the CC2420 transceiver on the Imote2 hardware. This indicates that the hardware itself is not the problem. The author of the SPI controller driver, and various other parts of this ARM system, have been contacted and after discussing the various attempts with him he suggested that it may not be possible to reach the needed timing with the available driver but it could be possible to reach them with a special purpose driver for the SPI controller optimized on timing for the controller. This was way beyond the scope of this work and at this point it was decided to start working with some other IEEE hardware and see if better results could be achieved with it in a shorter time frame Automatic ACK Before the decision was made to switch to a different transceiver an additional change had been integrated into the CC2420 driver to enable hardware accelerated automatic ACK of incoming frames. Such a change allows the sender to get feedback about the send frame. With automatic

40 6. Drivers ACK enabled the CC2420 expects the local short as well as long address and the PAN ID written into its RAM to allow the hardware to decide if the frame was targeted at this node or not. And reply with an ACK for the received frame if needed. An initial change to enable this feature for the CC2420 was already integrated into the driver by another developer. He added the needed code to react on address changes and update the information in RAM. For the long address the byte have been written in the wrong order resulting in every frame with the long address set as destination address got ignored by the hardware. The short addresses are assigned by the coordinator when associating with the PAN. For the association request frame and response the long address have to be used. A packet trace for an association request with the involved addresses can be seen on figure 5.2. With the problem described above it failed already during the earliest steps, the association with a PAN. Correcting the ordering and actually enabling the automatic ACK feature during hardware initialization made the feature work as expected. The implementation details of the change are similar to the implementation done for the AT86RF231 driver. The details and a relevant code listing can be found in section ATUSB Driver With all the problems to build a reliable sending path for the IBR-DTN CL with the CC2420 it became clear that an alternative hardware was needed to allow working on the CL even with an uncertain outgoing of the work on the CC2420 driver and its related SPI problems. The chosen hardware was a simple USB device called ATUSB from Qi Hardware 3. It consists of an AT86RF231 and an Atmel ATmega32U2 micro-controller. More details about the hardware can be found in section 4.4. While the AT86RF231 is a transceiver chip that handles the Radio frequency (RF) side and some MAC layer related parts of the IEEE specification the Atmel ATmega32U2 microcontroller takes care of the communication between the host over USB and the AT86RF231 over SPI. A small firmware, which can be updated over USB, handles the protocol adaption between SPI and USB. The design as a simple USB device with the size of a Bluetooth USB dongle allows it to run on every system that offers a USB host port and supports a driver for the hardware. Initially this is only the case for Linux based systems. The cost of switching to this new hardware was to help with the driver development as the hardware had just been out of the factory and no driver was written yet besides some basic factory testing tools for hardware verification. As already described in section 5.1 the socket user-space interface stays the same for any hardware driver used and thus the CL can cope with any new driver. A driver for the AT86RF231 chip itself was already written and part of the IEEE Linux stack. The driver expected to communicate with the AT86RF231 directly over SPI. Given that this is the only interface the hardware offers this was a natural choice when the

41 6.2. ATUSB Driver driver was written. For ATUSB the SPI communication needed to be transparently handled over an USB protocol. The approach that was chosen was to simply let the firmware on the Atmel micro-controller map the register access over SPI to similar commands over the USB protocol. On the driver side the complementary protocol was implemented to offer a virtual SPI master device the existing AT86RF231 driver could attach to. This way the complete SPI communication would be transparently transported over USB while the chip as well as the AT86RF231 drivers does only have knowledge about SPI. Abstracting the USB details seemed like a good idea to re-use an existing and proven to work driver for the RF chip. Based on a minimal driver skeleton that only allowed to read out the ID registers of the chip the driver was extended to implement the virtual SPI master. The USB protocol was already specified and implemented in the ATUSB firmware. The Linux SPI framework is flexible enough to write a SPI master driver without an actual SPI controller underneath and also allows to hint the system auto-loading the AT86RF231 driver when the ATUSB driver is available. The main work concentrated on the asynchronous SPI transfer handling. The SPI write and read commands in the Linux SPI framework are asynchronous. To transport all these commands over USB to the firmware and eventually the radio chip itself. All software pieces involved needed to handle the transfers asynchronous as well to avoid deadlocks or races. This was more complicated then doing the transfers synchronous but still possible to do within the USB implementation of the Linux kernel. In the atusb transfer function shown in listing 6.1 all SPI read and write transfers are processed. The incoming data gets classified into three different SPI commands. Register read, frame-buffer read and write. Based on the classification the corresponding function is called to handle the SPI transfer of the data. Listing 6.1: ATUSB SPI transfer code s t a t i c i n t a t u s b t r a n s f e r ( s t r u c t s p i d e v i c e s p i, s t r u c t s p i m e s s a g e msg ) s t r u c t a t u s b l o c a l a t u s b = s p i m a s t e r g e t d e v d a t a ( s p i >m a s t e r ) ; s t r u c t s p i t r a n s f e r x f e r ; s t r u c t s p i t r a n s f e r x [ 2 ] ; i n t n ; c o n s t u i n t 8 t t x ; u i n t 8 t rx ; i n t l e n ; i n t r e t v a l = 0 ; i f ( u n l i k e l y ( l i s t e m p t y (&msg >t r a n s f e r s ) ) ) d e v e r r (& a t u s b >udev >dev, t r a n s f e r i s empty\n ) ; r e t u r n EINVAL ; a t u s b >msg = msg ; / C l a s s i f y t h e r e q u e s t / n = 0 ; l i s t f o r e a c h e n t r y ( x f e r, &msg >t r a n s f e r s, t r a n s f e r l i s t ) i f ( n == ARRAY SIZE ( x ) ) d e v e r r (& a t u s b >udev >dev, t o o many t r a n s f e r s \n ) ; r e t u r n EINVAL ; 27

42 6. Drivers x [ n ] = x f e r ; n ++; t x = x[0] > t x b u f ; rx = x[0] > r x b u f ; l e n = x[0] > l e n ; msg >a c t u a l l e n g t h = l e n ; i f (! t x l e n!= 2) goto b a d r e q ; i f ( n == 1) i f ( rx ) dev dbg (& a t u s b >udev >dev, r e a d 1\n ) ; r e t v a l = a t u s b r e a d 1 ( a t u s b, t x [ 0 ], rx +1, len 1) ; e l s e dev dbg (& a t u s b >udev >dev, w r i t e 2\n ) ; i f ( t x [ 0 ] == (CMD REG CMD WRITE RG TRX CTRL 0 ) ) msg >s t a t u s = 0 ; msg >c o m p l e t e ( msg >c o n t e x t ) ; e l s e r e t v a l = a t u s b w r i t e ( a t u s b, t x [ 0 ], t x [ 1 ], NULL, 0) ; e l s e i f ( x[0] > r x b u f ) i f ( x[1] > t x b u f! x[1] > r x b u f ) goto b a d r e q ; dev dbg (& a t u s b >udev >dev, r e a d 1+\n ) ; r e t v a l = a t u s b r e a d f b ( a t u s b, t x [ 0 ], rx +1, x[1] > r x b u f, x[1] > l e n ) ; e l s e i f (! x[1] > t x b u f x[1] > r x b u f ) goto b a d r e q ; dev dbg (& a t u s b >udev >dev, w r i t e 2+n\n ) ; r e t v a l = a t u s b w r i t e ( a t u s b, t x [ 0 ], t x [ 1 ], x[1] > t x b u f, x[1] > l e n ) ; r e t u r n r e t v a l ; b a d r e q : d e v e r r (& a t u s b >udev >dev, u n r e c o g n i z e d r e q u e s t : \ n ) ; l i s t f o r e a c h e n t r y ( x f e r, &msg >t r a n s f e r s, t r a n s f e r l i s t ) d e v e r r (& a t u s b >udev >dev, % s t x %s r x l e n %u\n, x f e r >t x b u f? :!, x f e r >r x b u f? :!, x f e r >l e n ) ; r e t u r n EINVAL ; While all communication and configuration is done over SPI interrupts are also available to signal specific events from the hardware to the driver. Again an approach was chosen to receive the interrupts in the Atmel micro-controller and raise an USB interrupt to inform the ATUSB driver about the event. Inside the driver the information gets passed to a virtual interrupt chip connected to the AT86RF231 driver. Similar to the SPI communication the interrupt handling transparently transports the interrupt information from the hardware over the firmware and the 28

43 6.2. ATUSB Driver ATUSB driver to the actual radio driver. On some specific parts with regard to the USB protocol and on the interrupt virtualization Werner Almesberger helped out. He designed the ATUSB hardware, the USB protocol and implemented the firmware for the Atmel micro-controller. The resulting driver was stable and showed no urgent problems during the initial testing. The measured packet-loss at a distance of 5 m between two nodes was about %. In contrast, a packet-loss of 10% had been measured with the CC24020 driver and related hardware. Still the initial performance evaluation revealed some problems with the architecture of the driver and firmware, but not the hardware itself. While being stable and reliable the driver was not as fast as one would have expected. The core problem is related to the fact that every SPI command gets packed into a single USB transfer and the same on the way back. This adds significant latency on top of the actual data transfer and slowed down the driver and thus the data transfer. The driver was used in this form for this thesis to work on the IBR-DTN CL and performing the evaluation. Nonetheless, the reader should know that there is already work on the way to improve the performance. Werner Almesberger started to move the send and receive state machine into the firmware of the Atmel micro-controller and adjust the USB protocol to only transfer larger blocks of data over USB to cut down the latency. Unfortunately this would have taken to long to get implemented properly to still be covered in this thesis. Once the new firmware and driver are available a switch is easily possible by updating the firmware on the ATUSB devices and replacing the current driver with the new one AT86RF231 Improvements While the AT86RF231 driver was already available in the Linux IEEE stack and the main work went into a SPI master driver to make it work with the ATUSB hardware there still have been some improvements to the core AT86RF230 driver itself. The motivation was to enable the extended operating mode of the AT86RF231. In contrast to the basic operating mode the extended mode uses the hardware MAC accelerator to comply to time-critical tasks of the IEEE specification. While the basic mode only enables simple radio transceiver functionality with the IEEE frame format the extended mode implements the CSMA/CA algorithm with automatic retransmission as well as automatic ACK generation and frame retransmission on missing ACK in hardware. Automatic ACK generation can be used without automatic retransmission being enabled but not the other way around. For automatic ACK to be working correctly the MAC of the chip needs to know about the currently used long address, the short address, the PAN ID, and also if the device is the coordinator of the network. Without these information the hardware can not decide if the frame is targeted for this device or not and generate the ACK accordingly. This is similar to the CC2420 chip as mentioned in section In listing 6.2 the needed code additions for the hardware address handling are shown. At the bottom of the listing one can see an ieee ops struct being filled with function pointers to functions of the driver. This mechanism is used within the IEEE Linux 29

44 6. Drivers stack to call from the stack into the driver if any event makes this necessary. The.xmit function handles transmissions while.ed is called for energy detection. The others should be clear from their naming, setting the currently used channel and starting and stopping the device. With.set hw addr filt a new function pointer was set to react on any address or PAN changes. The ifdef s around the code allowing the driver to be compiled with or without automatic ACK support. With the provided function the core code can now call into the driver once a new address or PAN is set. As can be seen in the function itself the AT86RF231 uses several registers to store this values for later use from the MAC accelerator. To write the information into the registers the device is not allowed to be in a transmitting or receiving state. To make sure this is the case the device gets stopped with at86rf230 stop before the register access and started again afterwards. Address or PAN changes are not happening very often, typically during the initial setup of the device, so this method works fine without influencing the performance. Listing 6.2: Hardware address filter setup # i f d e f ENABLE AACK s t a t i c i n t a t 8 6 r f s e t h w a d d r f i l t ( s t r u c t i e e e d e v dev, s t r u c t i e e e h w a d d r f i l t f i l t, u n s i g n e d long changed ) s t r u c t a t 8 6 r f l o c a l l p = dev >p r i v ; m i g h t s l e e p ( ) ; a t 8 6 r f s t o p ( dev ) ; msleep ( 1 0 ) ; i f ( changed & IEEE SADDR CHANGED ) a t 8 6 r f w r i t e ( lp, RG SHORT ADDR 0, f i l t >s h o r t a d d r & 0 x f f ) ; a t 8 6 r f w r i t e ( lp, RG SHORT ADDR 1, ( f i l t >s h o r t a d d r >> 8) & 0 x f f ) ; i f ( changed & IEEE PANID CHANGED ) a t 8 6 r f w r i t e ( lp, RG PAN ID 0, f i l t >p a n i d & 0 x f f ) ; a t 8 6 r f w r i t e ( lp, RG PAN ID 1, ( f i l t >p a n i d >> 8) & 0 x f f ) ; i f ( changed & IEEE IEEEADDR CHANGED ) a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 0, f i l t >i e e e a d d r [ 7 ] ) ; a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 1, f i l t >i e e e a d d r [ 6 ] ) ; a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 2, f i l t >i e e e a d d r [ 5 ] ) ; a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 3, f i l t >i e e e a d d r [ 4 ] ) ; a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 4, f i l t >i e e e a d d r [ 3 ] ) ; a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 5, f i l t >i e e e a d d r [ 2 ] ) ; a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 6, f i l t >i e e e a d d r [ 1 ] ) ; a t 8 6 r f w r i t e s u b r e g ( lp, SR IEEE ADDR 7, f i l t >i e e e a d d r [ 0 ] ) ; i f ( changed & IEEE PANC CHANGED ) i f ( f i l t >p a n c o o r d ) a t 8 6 r f w r i t e s u b r e g ( lp, SR AACK I AM COORD, 1) ; e l s e a t 8 6 r f w r i t e s u b r e g ( lp, SR AACK I AM COORD, 0) ; a t 8 6 r f s t a r t ( dev ) ; 30

45 6.2. ATUSB Driver r e t u r n 0 ; # e n d i f s t a t i c s t r u c t i e e e o p s a t 8 6 r f o p s =. owner = THIS MODULE,. xmit = a t 8 6 r f x m i t,. ed = a t 8 6 r f e d,. s e t c h a n n e l = a t 8 6 r f c h a n n e l,. s t a r t = a t 8 6 r f s t a r t,. s t o p = a t 8 6 r f s t o p, # i f d e f ENABLE AACK. s e t h w a d d r f i l t = a t 8 6 r f s e t h w a d d r f i l t, # e n d i f ; Enabling automatic re-transmit required some changes to the state machine as well. These changes are omitted from the listing as they are rather simple changes of the defined state names in the functions handling the state machine. It is also possible to set values for the number of re-transmissions for frames without ACK as well as for frames where CCA failed. The CS- MA/CA algorithm included a random back-off value for further tries to access the medium. The AT86RF231 includes a random number generator and some seed values can be set to feed this generator preparing random values for the back-off. During the evaluation none of this values have been set but the default reset values have been used. For frame loss this means three additional tries to send the frame and for CSMA/CA 4 additional tries to access the medium. 31

46

47 7. IBR-DTN IBR-DTN [2] is a fully RFC5050 [5] compliant implementation of the BP. The focus of the implementation is to be lightweight and able to run on different types of embedded Linux systems. Tested devices range from small embedded router boards like the RouterStation Pro with a MIPS CPU over various embedded ARM boards, the Imote2 ARM based sensor node up to desktop and server systems. For the embedded boards, mostly based on OpenWRT 1, uclibc is used as C library. This library is optimized for smaller devices and aims for a smaller footprint then the standard glibc. Nevertheless, glibc does also work fine with IBR-DTN and is used in desktop or server installations. To reach the goal of a lightweight implementation IBR-DTN tries to reduce external dependencies like third party libraries to a minimum. This avoids problems with dependencies not being available on the target platform or adding to much overhead to it. Another way to to keep the implementation small and scalable for deployment on embedded targets is modularization. On the one hand it allows for better abstraction of the involved tasks like routing, storage receive and transmit. On the other hand it also allows to keep the overhead small when only relevant modules are involved instead in specific actions like routing. IBR-DTN uses an event mechanism shown on figure 7.1 to allow the modules to be loosely coupled by sending events to each other over an event switch. More about the different available modules of IBR-DTN can be found in section 7.1. In addition to modularization IBR-DTN also relies on the usage of threads for concurrent tasks. A CL with a blocking recv() system call could for example be implemented as thread and the event switch and other modules would still be active while the CL waits for incoming data. The usage of threads allows for concurrency and good performance while enabling modules with heavier or long term tasks to be separated out into a private work queue. Furthermore, IBR-DTN allows to select different compile time options to disable or enable functionality during build time. This is often used for modules with external dependencies like the HTTP CL based on libcurl or the bundle security draft implementation based on openssl. All this different mechanisms allow IBR-DTN to have a small memory footprint and light processing overhead. The resident set size in RAM after several days is around 3200 KiB on a desktop system and 2400 KiB on an embedded board with OpenWRT as stated in [2]. An empirical benchmark of IBR-DTN, ION and DTN2 can be found in [18]

48 7. IBR-DTN #!! "!! $% &! $ #' %$ Figure 7.1.: IBR-DTN event handling and modules [2] 7.1. Modules As already mentioned, the architecture of IBR-DTN is build around an event mechanism to let the various modules communicate with each other over events. The main component, shown in the center of figure 7.1, is the event switch. The event switch and the various modules are briefly explained in this section to understand the general architecture of IBR-DTN before explaining the details of the IEEE CL in section 8. In the following each element of figure 7.1 gets a brief description in the context of IBR-DTN Event Switch: Dispatches events from one module to another. It allows the modules to communicate over events. A set of standard events for the various modules have been defined while others are possible for newly developed modules or extensions. Discovery Agent: The neighbor discovery module generates events for new nodes becoming available. Two different neighbor discovery protocols are currently implemented. The IP Neighbor Discovery (IPND) protocol and the DTN2 specific discovery implemented in DTN2 to allow for compatibility. Connection Manager: The Connection Manager is supposed to handle the various lower level protocols IBR-DTN use to communicate with other DTN nodes. Connectors to such lower level protocols are called CL and are handled by the Connection Manager. Currently CLs for UDP, TCP, HTTP and an earlier version for IEEE are available. Details about the interaction between the Connection Manager, the CLs and the Event Switch can be found section 7.2. Bundle Storage: Abstraction of the storage options available in IBR-DTN. Bundles can be stored either in memory for access but on the cost of higher memory usage, in plain files on the disk for less memory usage but slower access or in a sqlite database to allow for additional meta-data connected with the bundle. Bundles are stored and retrieved based on their ID or destination. Base Router: Manages the different routing modules. Static routing defined in the config file of the daemon, neighbor based routing by events from the Discovery Agent, Epidemic routing 34

49 7.2. Convergence Layer with Bloomfilters, and a retransmission module for bundles that are temporarily declined by a CL and subsequently re-queued. Wall clock: Abstraction of a global time inside IBR-DTN based on the host clock. Sending out a global time tick event every second which gets dispatched to all other modules allowing for periodic tasks. API: Application interface based on TCP or Unix Domain Sockets. Reused the bundle streaming protocol to ease application writing by linking to the library and having access to all protocol elements inside the application. Extension and customization: Due to the modular concept IBR-DTN is well suited for customization or extension on the various modules. The main idea is to implement the base class of the family one want to extend. Tools: The IBR-DTN distribution includes different tools to test the functionality of the IBR- DTN installation and enable developers to use them as a base for their own applications. For basic file transfer over the BP dtnsend and dtnrecv are included. With dtnping and dtntracepath it is possible to measure the round trip time and visualize the route between different nodes. A small utility that can be seen as an lightweight API alternative for scripts is dtntrigger Convergence Layer IBR-DTN currently features four different CLs. The UDP and the TCP CLs are based on the respective internet drafts. While the UDP CL is simple and allows only bundles up to the MTU of the connection the TCP CL [19] is much more powerful and allows for segmentation of the bundles and acknowledgements of those segments. The third CL is HTTP based and uses libcurl as external dependency for the HTTP protocol. Not being based on any internet draft this CL is meant for exploration and research only. The fourth and last CL is for IEEE networks and was implemented by myself during a previous thesis [1]. This CL is object to a complete re-write during this thesis to cope with limitations that have been ignored for the first prototype. The scope of duties of a CL is mainly to send and receive serialized bundles over the lower level protocol it adapts IBR-DTN to. Doing so it is closely connected to the Connection Manager and possible the Discovery Agent if neighbor discovery is implemented and enabled for this CL. Discovery announcements need to be send and received over the lower level protocols the CL abstracts before the gathered information about available nodes and services is made available to the Base Router for later bundle routing. As services can describe properties beyond routing information they can be passed to other modules as well. The main scope for the CL is the bundle transfer. Once the Base Router has a bundle available which needs to be send over the CL to its destination the Connection Manager gets informed and uses the queue() method of the CL to queue the bundle for serialization and transmit. The 35

50 7. IBR-DTN queue() method needs to be implemented from every CL because of the inheritance from the ConvergenceLayer class. The method takes a node and a job object as parameter to gain access to the destination and the actual bundle to be transfered. The job object itself does only hold an ID for the bundle. With this ID it is possible to retrieve the actual bundle object from storage and serialize it for the transfer. A BUNDLE FORWARDED event gets raised to inform the event switch, and all modules subscribed to this event, about the forwarded bundle. The serialized bundle is modified with CL specific information, like segment headers or other internal properties, and finally send via the CL specific transfer methods. For the other direction the CL has to wait for incoming data and process the data until it is able to deserialize the incoming data into a bundle. This is again very specific to the CL and its lower level protocols. For the UDP CL a bundle can only have the maximal size of the MTU. In such cases no further processing is needed. Other CLs like TCP or IEEE bundles can be subdivided into several frames. In this case the CL need to take care to receive all segments of the bundle before it can be deserialized successfully. For the IEEE CL is explained in details in chapter 8. For each bundle successfully deserialized a BundleReceivedEvent gets raised to the event switch. Modules subscribed to this event can now handle the received bundle based on their configuration. Examples would be storage, forwarding to other nodes, immediately reply to an echo request or delivery to an application. To make the CL known to the daemon an addition to the createconvergencelayers method is needed. This allows the daemon to start the CL as enabled in the configuration file. Listing 7.1: IBR-DTN config for IEEE Convergence Layer l o c a l u r i = d t n : / / 1. d t n n e t i n t e r f a c e s = l a n 0 n e t l a n 0 t y p e = lowpan n e t l a n 0 i n t e r f a c e = wpan0 n e t l a n 0 p o r t = 1920 # Routing s e c t i o n r o u t i n g = d e f a u l t # S t a t i c c o n n e c t i o n s s t a t i c 1 a d d r e s s = s t a t i c 1 p o r t = 1920 s t a t i c 1 u r i = d t n : / / 2. d t n s t a t i c 1 p r o t o = lowpan Listing 7.1 shows an example for such a configuration file for IBR-DTN with an IEEE CL configured. The interface is wpan0 and the type lowpan. The type lowpan is used here to refer to the IEEE CL for the ease of writing. The PAN ID of the network is specified as port. All numbers inside the configuration are in decimal notation equals a PAN ID of 0x0777 and the address of the node behind the static route is 0x8001 in hexadecimal. The local node name is specific in the dtn:// scheme and the name 1.dtn while the remote node is 2.dtn. Changing the local uri name to the ipn: scheme and a numeric value would be needed when switching to CBHE. All other address and service elements needed for CBHE are set in the relevant applications. For more details about operating IBR-DTN in CBHE mode see section

51 7.2. Convergence Layer The last block of the configuration file defines statically available routes to nodes. This can be used for a network with a central management or smaller networks. In a dynamic environment automatic discovery of neighbors is more common. Section 8.4 explains how this is done with the IEEE CL. In such cases the block about static routing can be omitted. 37

52

53 8. IEEE Convergence Layer The initial IEEE CL was added to IBR-DTN with release Since this time IBR- DTN evolved and this thesis is using a development snapshot around version for development and evaluation. To overcome different limitations of the first implementation parts have been rewritten around a new architecture. This chapter gives an overview about the architecture and explains the concepts going along in more detail in the respective subsections Architecture The original prototype implementation was not prepared for two major tasks. The first one was segmentation of bundles over several IEEE frames. This comes together with the second limitation. No knowledge about the currently active connections and its source nodes has been available. For transfers with only one frame per bundle this did not result in any problem. But with bundles spanning over several frames an abstraction of the different connections was needed. To resolve the first limitation two different approaches are possible. Either make sure that the core breaks up the data into bundles that fit into an IEEE frame or transparently split big bundles into several segments, transmit them and put them together into a bundle in the CL on the receiver side. The first approach would add the overhead of the bundle header to each frame and would reduce the payload size significantly. For the second approach the CL must be prepared to handle multiplexed frames coming in from different connections at a time. The incoming frames need to get de-multiplexed to the different connections inside the daemon. This is also the case for discovery frames which can come in at any point in time from different nodes. IEEE does not offer multiplexing mechanisms build in like UDP and TCP do with ports. The segmentation header in combination with the source address of the frame is used to distinguish the logical destination of the incoming frames. The details about the segmentation header can be found in section 8.7. Given that frame multiplexing need to be in place for discovery frames anyway it made sense to use it for segmentation of bundles as well. This saves a good amount of space for additional payload as the bundle header is not transfered with each frame. The remaining section will explain the the implementation of the IEEE CL for IBR- DTN As already explained in chapter 7 IBR-DTN makes heavy use of threads to de-couple 39

54 8. IEEE Convergence Layer Figure 8.1.: Class diagrams for the architecture 40

55 8.1. Architecture active elements of the daemon from each other. This is followed by the design of the CL. In brief two threads are responsible to send and receive bundles as well as handling node discovery. The worker thread resides inside the LOWPANConvergenceLayer class and handles all frames coming from the IEEE Linux stack. Another task that is handled directly inside this thread is the detection of discovery frames and the notification of the core about this new connections. An overview of all relevant classes is shown in figure 8.1. LOWPANConvergenceLayer is the actual CL registered at the IBR-DTN core. It is the entry points for all incoming and outgoing bundles. LOWPANConvergenceLayer inherits from lowpanstream callback. This allows the lowpanstream send cb() to access the socket managed by LOWPANConvergenceLayer for data transfers. The lowpanstream class itself is an abstraction of the segments belonging to a bundle. The bundle is seen as a stream of data which is filled by the different segments belonging to the bundle. The lowpanstream is used in the LOWPANConnection class to handle the incoming and outgoing serialized bundles. LOWPANConnection has no knowledge about the segmentation and does only abstract the different available nodes as connections to which bundles can be transfered and received from. Each LOWPANConnection creates a thread as well to handle the incoming and outgoing bundles. Both threads are communicating over a lowpanstream to exchange data. To explain the details of the implementation figures 8.2 and 8.3 are showing a view on the involved components and method calls for the receiving as well as the transmitting side of the new CL architecture. At the bottom of each diagram is the IEEE Linux stack that provides the socket based interface to receive and send data frames. These socket interface is solely used for data exchange. Configuration is done as described in section 8.3. On top of the IEEE Linux stack sits the worker class of the CL. Inside a thread it waits for incoming data with a blocking recv() system call on the socket interface. Each received frame can either be a data or a discovery frame. The later one is an announcement from other nodes and gets handled differently from the normal data frames. In section 8.4 the details of these discovery frames are described. For the current high level view it is only relevant that for newly discovered nodes the IBR-DTN core gets notified with a NodeAvailable event. For incoming data frames the worker identifies the relevant connection based on the source address of the frame and queues the frame in the lowpanstream. As the name already suggests this class provide a continuous stream of data out of the received segments. This is the place where the actual segmentation reassembly and disassembly happens. The thread within the LOWPANConnection class does read() all incoming data from the lowpanstream directly into the de-serializer allowing it to decode the received bundle over time and notify the core with an BundleReceivedEvent. On figure 8.3 the transmission path of a bundle is shown. The IBR-DTN core calls the queue() method on the CL with the job and destination. The worker thread inside the LOWPANConvergenceLayer now delegates the job with the included bundle to the LOWPANConnection instance of the respective destination node by calling the queue() method with the job as parameter. The serializer now write() s the data into the lowpanstream for consumption of the worker thread. 41

56 8. IEEE Convergence Layer BundleSerializer() IBR-DTN Core NodeAvailable() Address Connection Stream read() queue() Worker IPND recv() IEEE Linux Stack Figure 8.2.: Receiving side of the new IEEE CL in IBR-DTN Inside the lowpanstream, and invisible to the LOWPANConnection or LOWPANConvergence- Layer, the data gets segmented into chunks fitting inside an IEEE frame. These chunks get transferred by calling send cb() and passing buffer and length. The ready to send frames are getting finally transfered back into the IEEE Linux stack by the send() system call on the socket interface. An one or two byte header as the first bytes of the payload is used to identify segmented bundles and distinguish between bundles and discovery frames. A complete description of this header and its usage is given in section 8.7. In addition to the header in front of the payload another two bytes after the payload are used as a workaround for a limitation of the kernel IEEE stack. The source address of every frame is needed to map it together with other frames coming from this node. Normally one would use the recvfrom() system call to receive the source address together with the actual data. But in this case this system call does not work and no source address is made available. To work around this limitation the simpler recv() system call is used to only receive the data. To hint the source address to the receiver the IEEE short address is put into the last two bytes of the data inside the CL and read out again on the receiving side. As this information is only for use of the CL it gets cut off before the data gets assembled into a bundle. The position at the end of the frame was chosen to allow for easier removal of this workaround once the recvfrom() system call got fixed. 42

57 8.2. Interface with the Kernel Stack IBR-DTN Core Address Connection Stream write() queue() send_cb() queue() Worker IPND send() IEEE Linux Stack Figure 8.3.: Transmitting side of the new IEEE CL in IBR-DTN 8.2. Interface with the Kernel Stack As already mentioned, for data transfer the IEEE Linux stack offers a socket interface which can be used with well-known system calls like send() and recv(). As the protocol itself has less features than for example TCP/IP it does only offer a very limited set of socket options to be enabled Configuration Configuring the IEEE Linux stack is done over other mechanisms as the data transfer. The configuration of the stack itself is done via netlink only and all IEEE related parts of an interface are controlled via netlink as well. These attributes include channel, short address, and PAN ID. All these attributes can be set manually by using the lowpan-tools utilities or using the netlink kernel interface directly. The CL relies on the former for initial configuration of the interface and for the later for retrieving the attributes of an interface in the CL. The CL uses this configuration interface to retrieve the currently configured PAN ID and local short address from the stack. These information are used to fill in the source address and PAN ID for the socket communication. 43

58 8. IEEE Convergence Layer 8.4. Discovery Discovery of neighbor nodes is especially important in wireless Ad-Hoc scenarios. Availability of nodes can be very dynamic in Ad-Hoc wireless networks. Nodes can appear in wireless connection range due to movement and can disappear as quickly as they appear. To cope with such a dynamic environment, nodes have to be detected once they appear and marked as not available when connections no longer seem possible. With IPND [20] an internet draft was specified to handle dynamic DTN node discovery for IP networks. The draft describes a mechanism to periodically send out small UDP packets to announce the availability of a node and times out after a certain number of beacons have not been received to mark the node as unavailable. While the draft was designed for IP networks the IPND beacon format is agnostic to the transport mode. A beacon contains fixed and optional blocks to allow for node as well as for service discovery. A minimal beacon contains a version field, flags to indicate optional blocks, a sequence number and an Endpoint Identifier (EID) length field plus the EID itself. This implementation uses version 0x02 as specified by the last internet draft at the time of writing. A beacon with the minimal information set fits well into a single IEEE frame and still has room for additional service discovery blocks. Service blocks are flexible in what they are used for. A common use is the announcement of CLs available on the node as well as configured routes. The second type of optional information is a neighborhood Bloom filter. It expresses the information of 1-hop neighbors allowing the receiving node to determine if the link between the two nodes is bi-directional. The IPND draft specifies broadcast, multicast, and unicast packets for sending out beacons. In this implementation for the IEEE CL only PAN internal broadcast frames have been implemented. This allows to detect nodes available within the same PAN. Sending out PAN internal beacon frames is done by setting the correct PAN ID for the outgoing frame and use 0xFFFF as destination address. In contrast broadcasts targeted at all available nodes in all PANs have to set the PAN ID to 0xFFFF as well. The beacon period has been increased compared to the default settings to make a more power efficient use of the radio resource. Instead of one beacon every second the interval has been set to 5 seconds and the timeout after a node gets marked as unavailable is set to 30 seconds. These default values are configurable to adjust the settings for specific scenarios. The handling of the beacon frames itself is sitting directly inside the LOWPANConvergence- Layer class. During de-multiplexing of incoming frames the class checks for the discovery bit set in the extended header to identify discovery beacon frames. Such beacons are then passed to the respective announcement handler within the IBR-DTN core which parses the beacon information and notifies the other modules over the Events Switch about the newly available node. If no new beacons arrive in the given timeout range the node will be marked as unavailable again. The periodic beacon sending is done inside the LOWPANConvergenceLayer class as well. To get informed about the elapsed time the CL subscribes itself to the TimeEvent and triggers the 44

59 8.5. Addressing beacon sending after the specified beacon interval. To give the receiving node all information to handle bundle transfers back to the source node the optional service block has to be used. This service block contains information about the short address and PAN ID of the source node within a defined IEEE CL service block. Combining the information of EID, short address and PAN ID allows the receiving node to transfer bundles back to the source node. The IPND based discovery mechanism is not compatible with the current µdtn discovery implementations. µdtn currently only implements a very simple discovery without any further information about the node. Work is planned to also allow IPND discovery with µdtn and thus discovery compatibility with this CL implementation Addressing IEEE specifies two different addressing schemes. The 64 bit long hardware address and the 16 bit long short address. While the hardware address should be unique world wide the short address is only unique within a PAN. Distributing the short addresses is done by the PAN coordinator. All tests in this thesis have been done with short addresses handed out by the coordinator Compressed Bundle Header Extension Some first measurements showed that the header of the BP already uses a significant part of the IEEE packet. With a MTU of only 128 byte and resulting 115 byte payload the Self-Delimiting Numeric Values (SDNV) based BP header can sometimes use over 50% of the available payload size. From a DTN perspective it makes sense to have all needed information available in a bundle to ease forwarding once a link becomes available. But IEEE is not the only layer that has problems with this extensive header size. With CBHE a specification [8] was written to shrink the header. In [7] the author describes the problem with large headers on slow wireless links to spacecrafts. Many of the problem also apply to an IEEE setup. The wireless links are slow as well and the limited MTU is problematic when sending small bundles and each bundle contains a large header. To reach this goal the former string based EIDs containing the dtn:// scheme name and a scheme specific part have been exchanged with a new ipn: scheme name and plain numbers for the scheme specific part. A node and a service number represent everything the dtn:// scheme represented with strings. Due to the usage of numbers the dictionary containing the strings inside the primary bundle header can be omitted and the dictionary length is set to zero. This is only possible if every EID inside the header is encoded in the compressed form. Next to the 45

60 8. IEEE Convergence Layer source and destination EID the header can contain the EIDs for a report-to endpoint and the current custodian. The EIDs also have to be in the correct order inside the header. If all these requirements are fulfilled the header can be compressed. IBR-DTN checks if all EIDs are given in the correct format and generates the header automatically with the correct order of the EIDs if the ipn: scheme is used. Node numbers as well as service numbers have to be well-known inside the network. No protocol for distribution has been defined by the time of writing. This limitation makes CBHE only useful in DTNs under homogeneous management. A combination of the ipn: scheme from CBHE with IPND could allow to make the needed CBHE information available within the network without relaying on well-known information only available in an homogeneous managed network. This is left as an exercise for future work. As already mentioned, IBR-DTN does automatically enable CBHE once all requirements are fulfilled. For this to happen one have to switch to the ipn: scheme in the configuration as well as in the DTN using applications. A valid ipn: scheme for the local URI as well as a static route could look like this in the configuration file: local_uri = ipn:1... static1_uri = ipn:2 This allows IBR-DTN to set the source address correctly and find the relevant node for the static route. The destination node number and service numbers for both, source and destination have to be set by the applications. For a file transfer the receiving side could set the service number to 0 and wait for incoming bundles like this: dtnrecv --name 0 The file itself gets send like this: dtnsend --src 0 ipn:1 1MB-bundle.md5 IBR-DTN takes care of translating the service and node numbers into the correct header format. Not all tools shipped with IBR-DTN are able to do this correctly. For example the dtnping application has a way to set the EID but as the IBR-DTN daemon on the other side does directly reply to this echo requests and does not allow to specify the service number for the ipn: scheme the ping fails. The savings of header compression are significant for small bundles. For a bundle that fits just into the 114 byte payload size of an IEEE frame header already uses around 79 byte with short strings for the EIDs and the dtn:// scheme. With CBHE the header only uses up to 34 byte and thus allows 45 byte more payload to be send. For bigger bundles fragmented 46

61 8.7. Segmentation over several IEEE frames this overhead is less significant, but for small bundles it is. Especially administrative messages or sensor data can be small enough to fit into such small bundles Segmentation IEEE has no field in its protocol header that defines the next protocol inside the payload of the IEEE data. In contrast the Internet Protocol (IP) does have such a field in the header named protocol and defining the next protocol which is used. The frame type field bits in the frame control field of IEEE only allow for frame but not for payload differentiation. This 3 bit field within the frame control field has four known types while the other numbers are marked as reserved. The documented types are beacon, data, acknowledgement and command frames. While the other four available numbers in the 3 bit field are listed as reserved there might be problems with different MAC layer implementations, be it in software or hardware, when using these reserved types for payload tagging. Some MAC implementation may ignore the reserved bits or would not pass the information to upper layers. Thus the only option for a header was in the payload itself. The 6LoWPAN specification does the same and uses the first two most significant bits to indicate what type of 6LoWPAN frame it is. If these two bits are set to zero 6LoWPAN understands that it is not a 6LoWPAN frame and ignores it. To aim for this interoperability with 6LoWPAN the first two bits are always set to zero. The first byte is also called dispatch type field in this context. The remaining 6 bits are used for information about the segment available in the payload. The purpose of the segment header is to add the required information to transfer a larger bundles split over several frames. These information consists of the segment position within the currently transferred bundle, a flag to distinguish between data and discovery frames and a segment sequence number. As one can see in figure 8.4 bit 5 and 4 indicate the position of the segment within the current bundle. The four possible combination cover all cases that can occur. 00 for a segment in the middle of the bundle, 01 for the last segment as well as 10 for the first and 11 for segments which contain the complete bundle in one frame. The next bit allows for future expansions indicating an extended header byte following directly after the initial header. If set to one the extended header is available. Until now only one bit in the extended header is used. It indicates if the frame is used as discovery frame. All other bits are reserved. An example of such a header can be seen in figure Segment position LoWPAN compat Ext. header Sequence number Figure 8.4.: Bit positions of the DTN LoWPAN segment header 47

62 8. IEEE Convergence Layer Discovery Reserved Figure 8.5.: Bit positions of the DTN LoWPAN extended segment header The last three bits containing the sequence number of the segment. While the three bits can only cover 8 different sequence numbers this is enough to detect out-of-band frames. With IEEE using the stop-and-wait algorithm to transmit frames this is good enough for the detection of lost frames. The ordering of send frames is fixed with stop-and-wait as the next frame is only send if the previous frame was successfully send. Under normal conditions this is ensured with ACKs. In such a case no new frame would have been send if no ACK arrives in time. In a setup without ACKs the sequence number can help the CL to understand if the previous frame was correctly received or not Acknowledgements Until now the CL fully relies on the stop-and-wait mechanism of the IEEE layer to provide the upper layers with a reliable stream of packets without lost packets or packets in wrong order. In the current state the IEEE Linux stack fails to provide enough feedback on received frames to completely rely on it. No indication is available from the stack if a send() system call did succeed or fail. The previously mentioned sequence numbers in the frame headers are used as a second indicator to detect lost frames in the receiving CL. IEEE uses the stop-and-wait mechanism as it is simple enough to be implement even on small sensor nodes and still offers a reliable transfer and avoids congestions. The evaluation in chapter 9 shows that under good circumstances the frame loss is minimal. In bad cases with much more radio interferences the frame loss will be higher and the CL need to be prepared for such scenarios. The stop-and-wait mechanism depends on the fact that the incoming frames are acknowledged. When the ACK is received at the sender side it knows that the current frame has arrived at its destination and the next frame can be send. If the ACK is missing in a defined time-frame a retransmission of the frame is the reaction. A limited number of retransmission handle the case where the destination node is no longer available. Thinking ahead in the future one can think of scenarios where a selective NACK mechanism on bundle level might make sense. Given the target of small embedded systems the available storage, be it RAM or disk storage, will be limited. Sending out a selective NACK when no more space is available on the node could be used to avoid congestion Compatibility with µdtn The bundle transfer is not yet compatible between µdtn and the IBR-DTN CL. In latest tests µdtn used a non standard version in the primary bundle header to avoid conflicts in the de- 48

63 8.9. Compatibility with µdtn velopment stage. This makes it incompatible with IBR-DTN which expects version 6 set in the primary bundle header. On the other side the IBR-DTN CL requires the segment header for all frames now which makes it incompatible with with µdtn which does not support segmentation at all right now and does not expect the header in front of the actual bundle. 49

64

65 9. Evaluation 9.1. Automated Test Suite To allow the evaluation measurements to be deterministic and reproducible time was spent on writing test suites or use existing ones that can be run in in a semi-automated environment. Results from such measurements can be compared more easily against measurements with changes of small influence and make it easier in general to evaluate the implementation Lowpan-perf an IEEE Test-Suite This thesis works with DTN on IEEE wireless networks. For the evaluation it was of interest how efficient the implementation is. It is helpful to differentiate between the performance on the raw IEEE layer and the performance over the BP implementation in IBR-DTN. While a test-suite for IBR-DTN performance measurements was available a measurement tool for IEEE was completely missing. Lowpan-perf was developed to fill this gap. It was developed with the experience and use cases of a previous study. Lowpan-perf is developed as another tool in the lowpan-tools package already containing configuration utilities and more. The design was kept simple and allows only for Round Trip Time (RTT) and throughput measurements between two nodes connected over IEEE , both running Linux. For the measurement a client server concept is used. As one can see in listing 9.1 the server side gets started and no more interaction is needed on its side. All configuration as well as as log and measurement data output is happening on the client side. This was done to ease the collection of the output data. The listing also shows that different parameters can be used to parameterize the measurement on the clients side. The payload length and therefor the packet length is adjustable from 5 to 115 bytes. The first 4 bytes are used for packet flags (1 byte), number of packets (1 byte) and a sequence number (2 bytes) to avoid counting duplicates and detecting packet loss. The flag field includes information about the operation mode. The mode parameter can be set to either round-trip or throughput. Started in round-trip mode the client side is generating the packets and sends them to the server and measures the time it takes until it receives a reply. In throughput mode only one configuration packet is send to the server and all other packets are generated at the server side while the client takes care about evaluating the incoming packets and calculating 51

66 9. Evaluation the throughput. The last outstanding parameter is a simple packet count. It only provides the number of packets that should be used for the measurement. The measured data points are printed out as Comma Separated Values (CSVs) with one measurement per line. The captured logs will get post-processed with a python script using matplotlib, shown in annex A.1, to create useful diagrams. Collecting the raw data points of the measurements allows for various analysis runs without the need of new measurements. The collected data can get analyzed with various statistical methods at any later point. More about the post processing can be found in annex A.3. lowpan p e r f s e r v e r 0 x8001 S e r v e r mode. Waiting f o r p a c k e t s... Listing 9.1: Lowpan-perf in server mode Listing 9.2: Lowpan-perf in client rount trip time mode lowpan p e r f c l i e n t 0 x0001 p a c k e t s 5 r o u n d t r i p S t a r t r o u n d t r i p t ime measurement... CSV f o r m a t : Sequence number, p a c k e t l e n g t h, r t t secs, r t t u s e c s 0,5,0, ,6,0, ,7,0, ,8,0, ,9,0, Received 5 from 5 p a c k e t s A r i t h m e t i c mean r o u n t r i p time : s e c o n d s and u s e c s Minimal time s e c o n d s and u s e c s Maximal time s e c o n d s and u s e c s Delay Tolerant Networking Test-Suite Evaluating the implementation aims for different goals. The main goal is to quantify the performance of the IEEE CL. For this numbers to be useful they need to be set in context with other measurement results. In [18] an empirical comparison of performance for the three major DTN implementations was done. The tools used for this comparison have also been used here to evaluate the throughput performance of this implementation. No direct comparison can be done as the paper is using a TCP CL for all benchmarks. The IEEE CL is based on completely different physical and data-link layers. Nonetheless, the benchmarks are showing that IBR-DTN is able to nearly saturate a 1 Gbit link. That gives a good indication for the potential bottleneck being either in the CL or in the layers below. The test suite consists of different parts responsible for different parts of the measurement. On the nodes itself only a small script is running which generates the bundles and transfers them to the other side. While the receiving side only waits for incoming bundles. The control of all involved parts is done from the outside of the nodes over a ssh connection. Like the lowpan-perf measurement tool the test suite collects the raw data from the IBR-DTN logfile by analyzing the events for received bundles written to the log with a precise time-stamp. The raw data can be processed at any later point and analyzed with statistical methods. 52

67 9.2. IEEE Round-Trip Time The most interesting measurements one can compare against are the measurements done during the implementation of the first iteration of the IEEE CL as described in [1]. Chapter 9.8 goes into the details of this comparison IEEE Round-Trip Time Before meaningful measurements on the DTN layer can be done one needs to know what the limits of the underlaying layers are. The speed characteristics of the IEEE specification on the 2.4 GHz band are specified with 250 kbit /s. Against this theoretical maximum the implementation can be compared. The measurement series begins with a round-trip time measurement between two ATUSB devices connected to the same host computer. Each of the USB devices gets passed to a virtual machine running a basic Linux system with the evaluation setup. Figure 9.1 shows the average round-trip time in milliseconds for IEEE payload sizes from 5 to 115 bytes. The first 4 bytes are used for a test internal header including a sequence number as well as some control flags for the measurements. 115 byte as maximum is the largest payload possible in an IEEE frame. The results have been obtained by running the lowpan-perf test with frames. During the run the payload size incremented for each frame from 5 up to 115 byte and starts over with 5 byte after reaching the maximum. This allows for 400 frames for each payload size and the cycling of the payload lengths helps to avoid possible side effects when sending the same frame over and over again. All three measurements have been done under the same circumstances. Run over night at a normal home office with a WiFi access point nearby. The setup for the virtual machines is described in section 5.2. One virtual machine operating as coordinator and a second virtual machine for the client have been run on the same host computer for all tests. One would have expected a linear increase of the RTT over the payload size. Instead the RTT increases slowly until 50 bytes followed by a steep edge and a smaller one shortly afterwards. From around 70 bytes onwards the gradient flattens again. The second observation one can make from this graphs is the time difference between RTT measurements with automatic ACK and automatic retransmission (ARET) enabled and the ones without. Measurements without this feature enabled show a 50 ms faster RTT. That equates an time increase of 37% for nodes with automatic retransmission enabled. The automatic ACK feature does not have any significant influence on the RTT during the measurements. The main difference between the two different transmit modes is the CSMA/CA algorithm used in the automatic retransmission mode. The basic transmit mode leaves the CSMA/CA compliances to the driver while in auto retransmission mode the hardware MAC implements the algorithm and leaves only certain values to be configured in registers. The significant delay is not indebted by the CSMA/CA overhead alone but on the majority by additional SPI register writes, which turned out to be the bottleneck of the current driver architecture described section

68 9. Evaluation round trip time in milliseconds AACK enabled, ARET enabled AACK disabled, ARET disabled AACK enabled, ARET disabled payload size in bytes Figure 9.1.: RTT average IEEE with different payload sizes With automatic retransmission enabled the state machine of the hardware gets more complex and needs more registers to be read and written to handle all state changes correctly. This increases the register access, read or write, from 8 to 14 times during each frame transmission. With an AT86RF231 directly connected over SPI or the new driver architecture for ATUSB this will be less of a problem, but with the current driver it imposes the significant delay we see in figure 9.1. In figures 9.2 and 9.3 the plotted percentiles of the RTT measurements illustrate that 50 percent of the measurements are grouped closely together at the bottom near the minimum. The percentile 100 graph indicates that not all samples are grouped together that closely in the upper region of the measurements. For the minimum the measured RTT seems to be bound to the driver implementation and its architecture. In theory a fully filled IEEE frame needs slightly over 4 ms to be transfered. For the RTT this means 8 to 9 ms for the transfer plus processing overhead on both sides. With a minimum RTT around 147 ms for a fully filled frame this is way off the theoretical maximum and illustrates that every frame transfer suffers from the slow performance of the current driver. The 100 percentile graphs does also indicate that there exists a high variance on the maximum latencies for the RTT. Such outliner in the measurements indicate that there are other relevant components on the system influencing the performance. As Linux is a general purpose operating system and no changes for real-time scheduling have been applied the scheduler might as well be the source of the introduced latencies. A full analysis on this is not possible in the time budget of the evaluation. 54

69 9.2. IEEE Round-Trip Time round trip time in milliseconds percentile 0 percentile 25 percentile 50 percentile 75 percentile payload size in bytes Figure 9.2.: RTT median IEEE with different payload sizes and AACK, ARET disabled round trip time in milliseconds percentile 0 percentile 25 percentile 50 percentile 75 percentile payload size in bytes Figure 9.3.: RTT median IEEE with different payload sizes and AACK, ARET enabled 55

70 9. Evaluation On figure 9.3 the influence of the enabled automatic retransmission feature can be seen again. While the percentile calculation look somewhat similar to the ones seen in figure 9.2 the minimal measurements already start with a 45 ms higher delay then the comparable measurements with automatic retransmission enabled. The reason for this have already been discussed earlier in this section IEEE Throughput To complete the IEEE measurements the throughput between two nodes has been measured. The setup is the same as in the RTT measurements described in the previous section. Again lowpan-perf was used to perform the measurement this time in throughput mode. As both types of measurements, RTT and throughput, are bound by the limitations and the delay introduced by the driver no surprise was expected by these measurements. Table 9.1 shows a summary of the throughput achieved on the different payload sizes. As expected the throughput increases for higher payloads. The table also shows that the maximum throughput is only bytes/s which is 5.88 % of the theoretical maximum of 250 kbit /s specified in IEEE Payload size in byte AACK disabled ARET disabled AACK enabled ARET enabled AACK enabled ARET disabled byte/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s bytes/s Table 9.1.: Throughput measurement results An interesting observation can be made on figure 9.4. The throughput increases linear with the payload size. No step edges at 50 and 70 byte as in the RTT measurements could be observed. The 33% less throughput with automatic retransmission enabled is comparable to the RTT measurements with automatic retransmission enabled. 56

71 9.4. IEEE Packet Loss bytes per seconds AACK enabled, ARET enabled AACK disabled, ARET disabled AACK enabled, ARET disabled payload size in bytes Figure 9.4.: IEEE throughput with different payload sizes 9.4. IEEE Packet Loss During the evaluation many test runs have been done to gather the needed data for the evaluation as well as doing some stress test of the ATUSB driver and the other components of the system. Each of this runs did cover frames on various sizes. Mostly 400 frames for each of the 110 different payload sizes. In all these runs a packet-loss of more then 5 frames have never been seen. In most cases it have been either no packet-loss at all or up to two frames. As 2 frames packet-loss out of send frames equals only percent loss no further evaluation on packet-loss have been done. With automatic ACK and automatic re-transmit the hardware was also able to handle even this small percentage of lost frames on its own. None of our evaluation runs with these features enabled did show a lost frame in the environment used for all evaluation tests. In a different environment with more radio transmission around this could be different. A short test in a lab with other active IEEE devices as well as WiFi and Bluetooth activity frame loss did even happen with automatic retransmission enabled. No real evaluation was done in this environment as the results would have not been comparable with the results of the other measurement runs. 57

72 9. Evaluation 9.5. DTN Round-Trip Time The evaluation series on the DTN layer starts with RTT measurements as well. The test suite described in section has no instrumentation to measure the RTT between two nodes. But with dtnping a small utility from the IBR-DTN distribution was available to handle the measurements. While for the IEEE measurements 400 frames have been used for each data point the DTN measurements started with 400 bundles. For bundles with a higher payload this did result in multiple frames being transfered due to the segmentation of the bundle. A mapping between the dtnping payload size and the actual IEEE bundle being transfered can be found in table 9.2. As maximum payload size 1000 bytes have been used. Transferring bigger bundles in the needed quantity for the measurement was not feasible with the current performance of the driver. The usage of dtnping unfortunately ruled out RTT measurements with CBHE enabled. As mentioned in section 8.6 it was not possible at the time of the evaluation to use CBHE together with dtnping. In figure 9.5 the RTT for bundles from 100 to 1000 bytes is measured in 10 bytes steps. The graph has similarities with a stairway while the RTT increases over the payload length. The edges are matching exactly the wrap to an additional segment. For the increased payload size within a segment the RTT increases only minimal, but once a new segment needs to be used for the increasing payload size the RTT increases with a big step. The RTT measurements have also revealed some problems with the CL implementation that had been unnoticed until then. Figure 9.6 shows the oms of the problem. For bundles with specific payload size the measurement has been more problematic as for others. Payload sizes like 47, 48, 380, 490 and 600 seem to trigger a hang inside the CL and thus make the measurements difficult and sometimes not possible. In this cases the number of samples have been reduced to 100 bundles for these values to allow a timely evaluation. A first observations on the problem indicates that the values are near by places where a wrap into another segment happens. While time has been spent to analyze and repair the problem it was not possible to do this in time for this thesis. No. segments Size in dtnping Table 9.2.: Ratio between the size option of dtnping and the number of segments for the bundle 58

73 9.5. DTN Round-Trip Time 2000 Round trip time in milliseconds dtnping payload size in bytes Figure 9.5.: DTN round trip time with dtnping with payload sizes from 100 to Round trip time in milliseconds dtnping payload size in bytes Figure 9.6.: DTN round trip time with dtnping with payload sizes from 1 to

74 9. Evaluation 400 round trip time in milliseconds dtnping payload size in bytes Figure 9.7.: DTN round trip time with dtnping with payload sizes from 1 to 100, but with 46 to 49 filtered out 9.6. DTN Throughput The DTN throughput measurements started from 10 bytes payload and increased 10 bytes for each run until 800 byte had been reached. This measurement also suffered from the problems with the CL described in the previous section. Again the number of samples have been reduced to allow for the evaluation to show results in time for this thesis. In the throughput measurement it was possible to add CBHE to the test matrix while also test the normal dtn:// scheme. The influence of automatic retransmission has also been evaluated. Figure 9.8 shows the graphs for the tests with CBHE enabled and disabled in combination with disabled automatic retransmission. As baseline a graph with automatic retransmission enabled but without CBHE is also plotted. Unfortunately due to the reduced sampling rate the measured values have an increased variance. Still it is possible to see that the peaks for the throughput are shifted towards the right for the graph with CBHE enabled. This matches fact that a frame is able to contain more payload due to the reduced header size and the wrap up into another segments thus happen for a higher payload size of the bundle. 60

75 9.7. DTN Bundle Loss bytes per second ARET, no CBHE no ARET, no CBHE no ARET, CBHE payload size in bytes Figure 9.8.: Median DTN throughput with different options 9.7. DTN Bundle Loss Given the results of the packet-loss measurements on the IEEE layer in combination with the problems involved with the other evaluation runs for the DTN CL no further evaluation on bundle loss has been done. It still should be noted that during the evaluation no major problems did arise from bundle loss. Most of the problems have been related to the hangs in the CL instead Comparison with Earlier Measurements During the initial development of the IEEE CL in [1] an evaluation was done to get an impression of the current performance status. These values can now be taken as a baseline the current results can be compared against. This helps to give the current measurements a context. Improvements and decline can be spotted easily and allowed to be analysed. One a first sight it is easy to see in table 9.3 that the RTT has improved but is still too high for what the hardware should be able to achieve. The DTN overhead for RTT was reduced. This seems mainly related to the fact that the earlier measurements have been done on an embedded system with limited computing resources while the current measurements have been done on a recent Laptop system with a dual core CPU and 61

76 9. Evaluation 4 GB of RAM. The IBR-DTN implementation can make good use of these computing resources and due to its threaded architecture also of the two cores. As a baseline the RTT measured for the UDP CL on the loopback as well as on an Ethernet are also listed in the table. A RTT value of 40 ms seems unlikely to reach with the IEEE Linux stack, but 80 ms should be reachable with an optimized driver and stack. Prototype implementation New architecure UDP CL IEEE ms 150ms DTN over IEEE ms 170ms DTN over loopback 38ms DTN over Ethernet/UDP 41ms Table 9.3.: RTT results for old and new measurements 62

77 10. Conclusion This thesis investigated DTN over IEEE wireless networks. More concretely, IBR- DTN was extended to support this network technology. For this a CL layer is required that adapts IBR-DTN to the lower level protocols and allows transferring bundles back and forth. An early prototype implementation of such an IEEE CL did exist but had some major architectural drawbacks. For example the prototype did not implement segmentation of bundles but allowed only for bundles that fit into the payload of a single IEEE frame. While bundle segmentation and discovery were integrated into the CL it became obvious that a new architecture for the CL was needed. With the new design and implementation the CL is also extensible in the future without changes to the core elements. With bundle segmentation and discovery two major limitations of the prototype have been eliminated. Automatic discovery of new nodes without configuring static routes with prior knowledge of the network makes IBR- DTN suitable for the dynamic nature of WSNs. While the bundle segmentation already works for a lot of cases some limitations still need to be resolved to support reliable transfer of bundles without size limit over the IEEE CL. The following section will highlight some of the limitations and possible solutions. On the driver side unexpected problems required a change from the Imote2 platform to the ATUSB USB devices for testing the implementation. This had the benefit of using a normal Linux host for development but limits the deployment of IBR-DTN on a small sensor node like platform. The work on the Imote2 can be picked up at any later point and all work that has been done on the CL would be available without modifications as the hardware itself is abstracted by the driver and the Linux IEEE stack from the application. All goals set for the thesis have been reached even if some of them still have limitations. Further work can start from here as all relevant parts have already been integrated into the official IBR-DTN code base and should be part of the next release. Modifications to the Linux kernel, different drivers and configuration utilities are not yet fully integrated with the various code bases, but the code is available on the enclosed data CD and work is underway to integrate the changes back to the original projects. This will simplify maintenance and should help others starting from this work. The evaluation showed the performance of the implementation and revealed bottlenecks in the driver architecture and problems in the interaction between the IEEE Linux stack and the IBR-DTN CL. The driver architecture problems have been acknowledged by the manufacturer and work is underway to improve the driver to reach the full throughput the AT86RF231 63

78 10. Conclusion can offer. Major modifications are needed to allow for a reliable stop and wait transfer over the IEEE Linux stack as described in the following section Future Work Several work items for the future have already been mentioned in the relevant chapters of this work. This section summarizes and groups them into matching topics. On the hardware and driver level, several constraints complicated the implementation and the later evaluation more than initially expected.. The Imote2 with the CC2420 transceiver showed to be difficult to drive correctly from a Linux kernel driver with the tight timings constraints on the read out of the status information. With more time on hand a new attempt could try to resolve this issue and produce a fully functional CC2420 Linux driver. This wold allow running IBR-DTN with the IEEE CL directly on the Imote2 and participate in an IEEE network without additional hardware. If such a new attempt turned out successful, the initial work on extending the Imote2 with a WiFi transceiver could get picked up again. For later deployment an interesting option would also be to clock the Imote2 CPU to a frequency between 13 and 104 MHz. This will allow using the low power mode which can be run on a low voltage level of 0.85 V on the Imote2. This should significantly increase the life-time of the sensor node on battery. If either all attempts on the CC2420 driver fail or a more comfortable development environment is preferred, work could be done to enhance the functionality and performance of the ATUSB driver. As already mentioned, a redesign of the driver and its companion firmware is already in progress. Once it is available, it needs to be verified if all advanced features such as the extended mode for automatic retransmit handling and status information from the hardware are implemented and if not, add them to the new driver. Further work on the IBR-DTN IEEE CL is more comfortable to test with ATUSB as no software needs to be cross-compiled and deployed to an embedded system. An evaluation with the developed and described tools should reveal any performance improvements or regressions such a new driver would bring. The IEEE Linux stack on top of the drivers needs some work as well. Two limitations have been especially problematic for the work on the CL. The first one is the defective recvfrom() system call. While data can be received without problems the source address of the sender is not available. Without knowledge about the source of the received frame connections to different nodes at the same time are impossible. As workaround the senders short address was written into the last two bytes of each frame leaving the CL. This obviously wastes the already scare space in the frame and works only between two IBR-DTN instances. Fixing this issue at the source within the IEEE Linux stack would help for further work on the CL and compatible implementations. On the sending side different issues complicated the work on the CL. First of all better status information for the send() system call would help applications like IBR-DTN understand which problem occurred in the IEEE stack while sending the frame. The available hardware 64

79 10.1. Future Work offers different types of diagnostic information. For example the AT86RF231 chip offers the information if a frame has not been sent due to contention on the medium with the attempted number of retries or if no ACK has been received and thus the frame was considered lost. The CC2420 is also be able to offer information if the medium was free and if the frame was sent. Bookkeeping for acknowledged frames has to be done within the stack for the hardware. This available information should be combined and passed up to the application or directly handled with retransmission of frames within the stack. This would allow for a much more reliable usage of the stack. The application would be informed if a given frame has actually arrived at the destination. This way the idea of a simple stop and wait algorithm could be used for the CL to keep track on send frames and re-schedule bundles if segments are lost. Another possible work item for the IEEE Linux stack would be the split of MAC layer handling in software from the rest of the stack. The software MAC implementation is not in a state where it could be useful to supplement hardware with less features. On the other hand it blocks a potential merge into the mainline Linux kernel. Availability in the mainline Linux kernel would ease the development even further. Once picked up by the distributions, no changes to the kernel would be needed to get the stack and the different drivers working for the work on the IEEE CL of IBR-DTN. The CL has various problematic areas of itself. Support for IEEE long addresses is not implemented yet and some problems for reliable usage still remain. Another limitation is the fixed PAN used by the CL. It relies on the configuration available during startup of IBR- DTN. A more dynamic handling of the PAN as well as association handling would allow the CL to listen for beacons of available PANs, join them, requests an address and wait for IPND frames of available DTN nodes. Such an dynamic behavior would allow deployed nodes to find available DTN nodes without manual intervention. The evaluation also revealed that some problems with the new CL architecture remain. Some specific payload lengths seem to trigger not yet understood problems which make IBR-DTN stall and be unresponsive until restarted. These problems need to be reproduced in a deterministic way, debugged and repaired. On the topic of energy efficiency nodes could use time-based rendezvous points for data exchange. This could be helpful for moving nodes with a known moving pattern and allow them to save power by only enabling the radio at the given time, when other nodes are available for data exchange. Such a mechanism needs synchronized clocks and an algorithm to dynamically learn about such rendezvous opportunities during the first phase of operation. This is a bit off topic for the groundwork described here but still interesting to investigate. The core of IBR-DTN is flexible enough to handle most of these ideas. Only two items can be considered as future work on the core. One is to allow dtnping to be used together with CBHE. The dtnping application itself is capable of setting the required ipn: scheme as well as the source service number, but the daemon on the other side replying to the echo requests is not capable of setting the service number correctly. Implementing the µdtn compatible discovery mechanism is the second potential work item for the IBR-DTN core. Compatibility between µdtn and IBR-DTN can also be enhanced on the µdtn side. Fully implementing IPND over IEEE would help as well as the integration of the segment 65

80 10. Conclusion and discovery header implemented in IBR-DTN to allow for bundles bigger then a single IEEE frame. In general, a draft for an IEEE based DTN CL is missing to allow for future compatibility between possible different implementations. Further work in this area could result in a draft discussed in the wider DTN research community. 66

81 Bibliography [1] Stefan Schmidt. Delay Tolerant Networking on Embedded Linux Hardware. Study Thesis, Braunschweig, Germany, August stefan/papers/study-thesis-final.pdf. [2] Sebastian Schildt, Johannes Morgenroth, Wolf-Bastian Pöttner, and Lars Wolf. IBR-DTN: A lightweight, modular and highly portable Bundle Protocol implementation. Electronic Communications of the EASST, 37:1 11, Jan [3] IEEE Standard for Information Technology-Telecommunications and Information Exchange Between Systems-Local and Metropolitan Area Networks-Specific Requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications. IEEE Std (Revision of IEEE Std ), pages C1 1184, June [4] Rahul C. Shah, Sumit Roy, Sushant Jain, Waylon Brunette. Data MULEs: Modeling a Three-tier Architecture for Sparse Sensor Networks. Sensor Network Protocols and Applications, Proceedings of the First IEEE IEEE International Workshop on, May [5] K. Scott, S. Burleigh. Bundle Protocol Specification. RFC 5050, The MITRE Corporation, NASA Jet Propulsion Laboratory, December [6] Michael Demmer. DTN Reference Implementation. August http: //dtnrg.org/docs/presentations/ietf60/dtn-impl-ietf demmer.pdf. [7] Scott Burleigh. Interplanetary Overlay Network: An Implementation of the DTN Bundle Protocol. In Consumer Communications and Networking Conference, CCNC th IEEE, pages , jan [8] S. Burleigh. Compressed Bundle Header Encoding (CBHE). RFC 6260, Jet Propulsion Laboratory, May [9] Omar Mukhtar, Jörg Ott. Backup and Bypass: Introducing DTN-based Ad-hoc Networking to Mobile Phones. Florence, Italy, [10] Sergiu Nedevschi Rabin Patra. DTNLite: A Reliable Data Transfer Architecture for Sensor Networks. University of California,

82 Bibliography [11] Max Loubser. Delay Tolerant Networking for Sensor Networks. Master Thesis, Stockholm, Sweden, December johanmon/theses/ loubser.pdf. [12] Georg von Zengen. Unterbrechungstolerante drahtlose Sensornetze. Bachelor Thesis, Braunschweig, Germany, August [13] Crossbow Inc. Imote2 Hardware Reference Manual. September networks/category/6-user-manuals.html?download=56%3aimote2- hardware-reference-manual. [14] Texas Instruments. CC2420 Datasheet. March gpn/cc2420. [15] Wolf-Bastian Pöttner and Lars Wolf. IEEE packet analysis with Wireshark and off-the-shelf hardware. In Proceedings of the Seventh International Conference on Networked Sensing Systems (INSS2010), Kassel, Germany, June [16] Atmel. AT86RF231 Datasheet. September resources/prod_documents/doc8111.pdf. [17] Wolf-Bastian Pöttner, Sebastian Schildt, Daniel Meyer, and Lars C Wolf. Piggy-Backing Link Quality Measurements to IEEE Acknowledgements. In The 4th International Workshop on Wireless Sensor, Actuator and Robot Networks (WiSARN-Fall 2011), Valencia, Spain, [18] Wolf-Bastian Pöttner, Johannes Morgenroth, Sebastian Schildt, and Lars C Wolf. An Empirical Performance Comparison of DTN Bundle Protocol Implementations. In ACM MobiCom 2011 Workshop on Challenged Networks (CHANTS 11), Las Vegas, Nevada, USA, [19] M. Demmer, J. Ott. Delay Tolerant Networking TCP Convergence Layer Protocol. IETF Draft, UC Berkeley, Helsinki University of Technology, November [20] D. Ellard, D. Brown. DTN IP Neighbor Discovery (IPND). IETF Draft, Raytheon BBN Technologies, March

83 A. Tools A.1. IEEE Sniffer For debugging purpose it often makes sense to listen directly to the frames that are transfered over the air instead of listening to the packets available from the interface on the sending or receiving machine. For instance debugging a driver may need this technique as it is not possible to trust the code currently in development. During the work on this diploma thesis such a sniffer was used to listen to the frames transmitted between two different nodes. A T-Mote Sky with a special sniffer firmware was used as hardware to capture the frames from the air and inject them into the Wireshark network protocol analyzer [15]. A small user-space process takes the frames over a serial line and injects them into an Ethernet frame with the type 0x809A on the loopback interface. The ethernettype allows Wireshark to detect that frame does include IEEE data and use the correct dissector for the data. The sniffer is only available under Linux and needs root privileges for the Ethernet frame injection. The following command line does start the sniffer to listen on the ttyusb0 serial interface and injects the frames on the loopback device: sudo./sniffer --serial /dev/ttyusb0 --ethdevice lo Once started some status information will be printed out followed by the captured frames. At this point Wireshark can be started and capturing on the loopback interface will show the frames with the correct dissector chosen to look into the details of the header and payload. The packets will be shown in real-time but could also be saved for later analysis. While listening on the loopback interface there might come in some non-related packets during the sniffing. ICMP and DNS packets are the most common disquieter here. They can be filtered out with the expressions below entered in the filter entry box. Other protocols can be added as needed.!icmp &&!dns Another observed problem is that some packets are detected as 6LoWPAN or ZigBee while they are plain IEEE frames with user specific payload. The way the Wireshark dissectors 69

84 A. Tools are detecting if they are responsible for the packet is a bit fragile in this regard. The easiest solution is to disable the protocols not interesting for the current sniffing session. This can be done under Analyze - Enabled Protocols. Disabling the wrongly detected protocols let them fall back to the plain IEEE dissector. A.2. DTN Performance Tools For the purpose of performance evaluation a set of scripts have been written to evaluate different DTN implementations under various conditions [18]. We used these tools to measure our DTN throughput between two IEEE nodes. Using the same tools allowed us to easier compare the results with other implementations as well as with IBR-DTN in other scenarios and with other CLs. To run the test-suite bash as well as PHP are needed on the host and the nodes. The following steps are needed to install and configure the IBR-DTN test-suite: Checkout the IBR-DTN tools from the repository on your host Install PHP and timelimit on the host. Install PHP on the nodes. Install ntpdate on the nodes and set timezone correctly. Copy dtngenerator to the node Write configs for lowpan coordinator and client settings. Here it is important to add a entry for the logfile like the other example configurations. Write a matching prepare linux script Write a matching run throughput tests script Replace iptables mechanism A.3. IEEE Evaluation Tooling While lowpan-perf was the tool that did the actual measurements during the IEEE evaluation other tools have been used to process the raw data. The collected data was written out in CSV from lowpan-perf. Capturing the raw data separate from the processing of the data allowed us to use different methods of data analyzing while avoiding the overhead of the physical measurement for each. To plot the measurements a small python script was used with matplotlib as can be seen in listing A.1. Methods to parse the output from lowpan-perf, the IBR-DTN test scripts and dtnping are available. Combined with the matplotlib this allows to plot graphs with minimum, average, maximum as well as percentile calculations. 70

85 A.3. IEEE Evaluation Tooling Listing A.1: evaluation-plot-generate.py #! / u s r / b i n / python i m p o r t csv, s y s name = s y s. argv [ 1 ] i m p o r t m a t p l o t l i b, numpy m a t p l o t l i b. use ( GtkAgg ) m a t p l o t l i b. r c ( t e x t, u s e t e x =True ) m a t p l o t l i b. r c ( f o n t, f a m i l y = s e r i f ) m a t p l o t l i b. r c ( axes, t i t l e s i z e = small ) m a t p l o t l i b. r c ( legend, f o n t s i z e = small ) from p y l a b i m p o r t d e f d t n t h r o u g h p u t ( ) : d a t a = [ ] d a t a 2 = [ ] d a t a 3 = [ ] with open ( s y s. a rgv [ 1 ], rb ) as f : f o r l i n e i n f : i n d e x = f l o a t ( l i n e. s p l i t ( ) [ 0 ] ) # min = f l o a t ( l i n e. s p l i t ( ) [ 2 ] ) median = f l o a t ( l i n e. s p l i t ( ) [ 4 ] ) # max = f l o a t ( l i n e. s p l i t ( ) [ 6 ] ) d a t a. append ( ( index, median ) ) # d a t a 2. append ( ( index, median ) ) # d a t a 3. append ( ( index, max ) ) with open ( s y s. a rgv [ 2 ], rb ) as f2 : f o r l i n e 2 i n f2 : i n d e x 2 = f l o a t ( l i n e 2. s p l i t ( ) [ 0 ] ) median2 = f l o a t ( l i n e 2. s p l i t ( ) [ 4 ] ) d a t a 2. append ( ( index2, median2 ) ) with open ( s y s. a rgv [ 3 ], rb ) as f3 : f o r l i n e 3 i n f3 : i n d e x 3 = f l o a t ( l i n e 3. s p l i t ( ) [ 0 ] ) median3 = f l o a t ( l i n e 3. s p l i t ( ) [ 4 ] ) p r i n t i n d e x 3 d a t a 3. append ( ( index3, median3 ) ) f i g = f i g u r e ( ) ax = f i g. a d d s u b p l o t ( ) ax. s e t x l a b e l ( p a y l o a d s i z e i n b y t e s ) ax. s e t y l a b e l ( b y t e s p e r second ) ax. p l o t ( [ s [ 0 ] f o r s i n d a t a ], [ s [ 1 ] f o r s i n d a t a ] ) ax. p l o t ( [ s2 [ 0 ] f o r s2 i n d a t a 2 ], [ s2 [ 1 ] f o r s2 i n d a t a 2 ] ) ax. p l o t ( [ s3 [ 0 ] f o r s3 i n d a t a 3 ], [ s3 [ 1 ] f o r s3 i n d a t a 3 ] ) ax. l e g e n d ( ( ARET, no CBHE, no ARET, no CBHE, no ARET, CBHE ), l o c =4, shadow=true ) s a v e f i g ( ( median %s. pdf % ( name, ) ). r e p l a c e (, ). r e p l a c e ( /, ), t r a n s p a r e n t = True ) c l o s e ( ) d e f i e e e r t t m e d i a n ( ) : d a t a = [ ] with open ( s y s. a rgv [ 1 ], rb ) as f : r e a d e r = csv. r e a d e r ( f, q u o t i n g = csv.quote NONNUMERIC) f o r row i n r e a d e r : d a t a. append ( ( row [ 0 ], row [ 1 ], row [ 2 ] + row [ 3 ] / ) ) 71

86 A. Tools s i z e s = f o r x i n d a t a : s i z e s. s e t d e f a u l t ( i n t ( x [ 1 ] ), [ ] ). append ( x [ 2 ] ) s i z e s = s i z e s. i t e m s ( ) s i z e s. s o r t ( ) f i g = f i g u r e ( ) # ylim ( , ) ax = f i g. a d d s u b p l o t ( ) ax. s e t x l a b e l ( p a y l o a d s i z e i n b y t e s ) ax. s e t y l a b e l ( round t r i p time i n m i l l i s e c o n d s ) ax. p l o t ( [ s [ 0 ] f o r s i n s i z e s ], [ numpy. p e r c e n t i l e ( s [ 1 ], [ 0. 0, , , , ] ) f o r s i n s i z e s ] ) ax. l e g e n d ( ( p e r c e n t i l e 0, p e r c e n t i l e 25, p e r c e n t i l e 50, p e r c e n t i l e 75, p e r c e n t i l e 100 ), l o c =4, shadow=true ) s a v e f i g ( ( median %s. pdf % ( name, ) ). r e p l a c e (, ). r e p l a c e ( /, ), t r a n s p a r e n t = True ) c l o s e ( ) d e f i e e e r t t a v g ( ) : d a t a = [ ] d a t a 2 = [ ] d a t a 3 = [ ] with open ( s y s. a rgv [ 1 ], rb ) as f : r e a d e r = csv. r e a d e r ( f, q u o t i n g = csv.quote NONNUMERIC) f o r row i n r e a d e r : d a t a. append ( ( row [ 0 ], row [ 1 ], row [ 2 ] + row [ 3 ] / ) ) with open ( s y s. a rgv [ 2 ], rb ) as f2 : r e a d e r 2 = csv. r e a d e r ( f2, q u o t i n g = csv.quote NONNUMERIC) f o r row2 i n r e a d e r 2 : d a t a 2. append ( ( row2 [ 0 ], row2 [ 1 ], row2 [ 2 ] + row2 [ 3 ] / ) ) with open ( s y s. a rgv [ 3 ], rb ) as f3 : r e a d e r 3 = csv. r e a d e r ( f3, q u o t i n g = csv.quote NONNUMERIC) f o r row3 i n r e a d e r 3 : d a t a 3. append ( ( row3 [ 0 ], row3 [ 1 ], row3 [ 2 ] + row3 [ 3 ] / ) ) s i z e s = f o r x i n d a t a : s i z e s. s e t d e f a u l t ( i n t ( x [ 1 ] ), [ ] ). append ( x [ 2 ] ) s i z e s = s i z e s. i t e m s ( ) s i z e s. s o r t ( ) s i z e s 2 = f o r x2 i n d a t a 2 : s i z e s 2. s e t d e f a u l t ( i n t ( x2 [ 1 ] ), [ ] ). append ( x2 [ 2 ] ) s i z e s 2 = s i z e s 2. i t e m s ( ) s i z e s 2. s o r t ( ) s i z e s 3 = f o r x3 i n d a t a 3 : s i z e s 3. s e t d e f a u l t ( i n t ( x3 [ 1 ] ), [ ] ). append ( x3 [ 2 ] ) s i z e s 3 = s i z e s 3. i t e m s ( ) s i z e s 3. s o r t ( ) 72

87 A.3. IEEE Evaluation Tooling f i g = f i g u r e ( ) ax = f i g. a d d s u b p l o t ( ) # ax. s e t y l i m ( , ) ax. s e t x l a b e l ( p a y l o a d s i z e i n b y t e s ) ax. s e t y l a b e l ( round t r i p time i n m i l l i s e c o n d s ) ax. p l o t ( [ s [ 0 ] f o r s i n s i z e s ], [ numpy. a v e r a g e ( s [ 1 ] ) f o r s i n s i z e s ] ) ax. p l o t ( [ s2 [ 0 ] f o r s2 i n s i z e s 2 ], [ numpy. a v e r a g e ( s2 [ 1 ] ) f o r s2 i n s i z e s 2 ] ) ax. p l o t ( [ s3 [ 0 ] f o r s3 i n s i z e s 3 ], [ numpy. a v e r a g e ( s3 [ 1 ] ) f o r s3 i n s i z e s 3 ] ) ax. l e g e n d ( ( AACK enabled, ARET enabled, AACK d i s a b l e d, ARET d i s a b l e d, AACK enabled, ARET d i s a b l e d ), l o c =4, shadow=true ) s a v e f i g ( ( average %s. pdf % ( name, ) ). r e p l a c e (, ). r e p l a c e ( /, ), t r a n s p a r e n t =True ) c l o s e ( ) d e f i e e e t h r o u g h p u t ( ) : d a t a = [ ] d a t a 2 = [ ] d a t a 3 = [ ] with open ( s y s. a rgv [ 1 ], rb ) as f : f o r index, l i n e i n enumerate ( f, s t a r t =5) : v a l u e = f l o a t ( l i n e. s p l i t ( ) [ 2]) d a t a. append ( ( index, v a l u e ) ) with open ( s y s. a rgv [ 2 ], rb ) as f2 : f o r index2, l i n e 2 i n enumerate ( f2, s t a r t =5) : v a l u e 2 = f l o a t ( l i n e 2. s p l i t ( ) [ 2]) d a t a 2. append ( ( index2, v a l u e 2 ) ) with open ( s y s. a rgv [ 3 ], rb ) as f3 : f o r index3, l i n e 3 i n enumerate ( f3, s t a r t =5) : v a l u e 3 = f l o a t ( l i n e 3. s p l i t ( ) [ 2]) d a t a 3. append ( ( index3, v a l u e 3 ) ) f i g = f i g u r e ( ) ax = f i g. a d d s u b p l o t ( ) # ax. s e t y l i m ( 0, ) ax. s e t x l a b e l ( p a y l o a d s i z e i n b y t e s ) ax. s e t y l a b e l ( b y t e s p e r seconds ) ax. p l o t ( [ s [ 0 ] f o r s i n d a t a ], [ s [ 1 ] f o r s i n d a t a ] ) ax. p l o t ( [ s2 [ 0 ] f o r s2 i n d a t a 2 ], [ s2 [ 1 ] f o r s2 i n d a t a 2 ] ) ax. p l o t ( [ s3 [ 0 ] f o r s3 i n d a t a 3 ], [ s3 [ 1 ] f o r s3 i n d a t a 3 ] ) ax. l e g e n d ( ( AACK enabled, ARET enabled, AACK d i s a b l e d, ARET d i s a b l e d, AACK enabled, ARET d i s a b l e d ), l o c =4, shadow=true ) s a v e f i g ( ( average %s. pdf % ( name, ) ). r e p l a c e (, ). r e p l a c e ( /, ), t r a n s p a r e n t =True ) c l o s e ( ) d e f d t n r t t a v g ( ) : d a t a = [ ] with open ( s y s. a rgv [ 1 ], rb ) as f : f o r l i n e i n f : i n d e x = l i n e. s p l i t ( ) [ 0 ] u n i t = l i n e. s p l i t ( ) [ 1] time = l i n e. s p l i t ( ) [ 2] v a l u e = f l o a t ( time [ 5 : ] ) i f u n i t == s : v a l u e = 1000 # p r i n t index, v a l u e d a t a. append ( ( index, v a l u e ) ) s i z e s = 73

88 A. Tools f o r x i n d a t a : s i z e s. s e t d e f a u l t ( i n t ( x [ 0 ] ), [ ] ). append ( x [ 1 ] ) s i z e s = s i z e s. i t e m s ( ) s i z e s. s o r t ( ) f i g = f i g u r e ( ) ax = f i g. a d d s u b p l o t ( ) ax. s e t y l i m ( 2 0 0, 400) # ax. s e t y l i m ( 2 0 0, 2200) ax. s e t x l a b e l ( d t n p i n g p a y l o a d s i z e i n b y t e s ) ax. s e t y l a b e l ( round t r i p time i n m i l l i s e c o n d s ) ax. p l o t ( [ s [ 0 ] f o r s i n s i z e s ], [ numpy. a v e r a g e ( s [ 1 ] ) f o r s i n s i z e s ] ) s a v e f i g ( ( average %s. pdf % ( name, ) ). r e p l a c e (, ). r e p l a c e ( /, ), t r a n s p a r e n t =True ) c l o s e ( ) d t n t h r o u g h p u t ( ) # i e e e r t t m e d i a n ( ) # i e e e r t t a v g ( ) # i e e e t h r o u g h p u t ( ) # d t n r t t a v g ( ) show ( ) 74

89 B. Code B.1. Lowpan-perf Code Listing B.1: lowpan-perf.c / Linux IEEE p e r f o r m a n c e measurement C o p y r i g h t (C) 2011 S t e f a n Schmidt <s t e f a d a t e n f r e i h a f e n. org> This program i s f r e e s o f t w a r e ; you can r e d i s t r i b u t e i t and / or modify i t under t h e t e r m s of t h e GNU G e n e r a l P u b l i c L i c e n s e as p u b l i s h e d by t h e F ree S o f t w a r e F o u n d a t i o n ; v e r s i o n 2 of t h e L i c e n s e. This program i s d i s t r i b u t e d i n t h e hope t h a t i t w i l l be u s e f u l, b u t WITHOUT ANY WARRANTY; w i t h o u t even t h e i m p l i e d w a r r a n t y of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See t h e GNU G e n e r a l P u b l i c L i c e n s e f o r more d e t a i l s. You s h o u l d have r e c e i v e d a copy of t h e GNU G e n e r a l P u b l i c L i c e n s e a l o n g with t h i s program ; i f not, w r i t e t o t h e F r ee S o f t w a r e Foundation, I n c., 51 F r a n k l i n S t r e e t, F i f t h Floor, Boston, MA USA. / # i f d e f HAVE CONFIG H # i n c l u d e <c o n f i g. h> # e n d i f # i n c l u d e <n e t / i f. h> # i n c l u d e <s y s / i o c t l. h> # i n c l u d e <s y s / s o c k e t. h> # i n c l u d e <s y s / time. h> # i n c l u d e <s t d i o. h> # i n c l u d e <s t d i n t. h> # i n c l u d e <s t d l i b. h> # i n c l u d e <s t r i n g. h> # i n c l u d e <u n i s t d. h> # i n c l u d e <g e t o p t. h> # i n c l u d e i e e e h # i n c l u d e <n e t l i n k / n e t l i n k. h> # i n c l u d e <n e t l i n k / a t t r. h> # i n c l u d e <n e t l i n k / g e n l / g e n l. h> # i n c l u d e <n e t l i n k / g e n l / c t r l. h> # i n c l u d e nl h e x t e r n s t r u c t n l a p o l i c y i e e e p o l i c y [ IEEE ATTR MAX + 1 ] ; # d e f i n e MAX PAYLOAD LEN

90 B. Code # d e f i n e MODE ROUNDTRIP 1 # d e f i n e MODE THROUGHPUT 2 # d e f i n e MODE BOTH 3 # d e f i n e PACKET CONFIG 1 # d e f i n e PACKET ROUNDTRIP 2 # d e f i n e PACKET THROUGHPUT 3 # i f d e f HAVE GETOPT LONG s t a t i c c o n s t s t r u c t o p t i o n p e r f l o n g o p t s [ ] = s e r v e r, r e q u i r e d a r g u m e n t, NULL, s, c l i e n t, r e q u i r e d a r g u m e n t, NULL, c, p a c k e t s, r e q u i r e d a r g u m e n t, NULL, p, l e n g t h, r e q u i r e d a r g u m e n t, NULL, l, r o u n d t r i p, no argument, NULL, r, t h r o u g h p u t, no argument, NULL, t, v e r s i o n, no argument, NULL, v, h e l p, no argument, NULL, h, NULL, 0, NULL, 0, ; # e n d i f s t r u c t c o n f i g c h a r p a c k e t l e n ; c h a r p a c k e t l e n r o t a t i o n ; u n s i g n e d s h o r t p a c k e t s ; long d s t a d d r ; long s r c a d d r ; i n t p a n i d ; c h a r s e r v e r ; c h a r mode ; c h a r p a c k e t t y p e ; ; e x t e r n c h a r o p t a r g ; void usage ( c o n s t c h a r name ) p r i n t f ( Usage : %s OPTIONS\n OPTIONS : \ n s e r v e r s c l i e n t a d d r e s s \n c l i e n t c s e r v e r a d d r e s s \n p a c k e t s p number of p a c k e t s \n l e n g t h l p a c k e t l e n g t h \n r o u n d t r i p r s t a r t i n r o u n d t r i p mode\n t h r o u g h p u t t s t a r t i n t h r o u g h p u t mode\n v e r s i o n v p r i n t o u t v e r s i o n \n h e l p This usage t e x t \n, name ) ; i n t g e t i n t e r f a c e i n f o ( s t r u c t c o n f i g c onf ) s t r u c t n l h a n d l e n l = n l h a n d l e a l l o c ( ) ; u n s i g n e d c h a r buf = NULL; s t r u c t s o c k a d d r n l n l a ; s t r u c t n l a t t r a t t r s [ IEEE ATTR MAX + 1 ] ; s t r u c t nlmsghdr n l h ; s t r u c t nl msg msg ; i n t f a m i l y ; i f (! n l ) r e t u r n 1 ; g e n l c o n n e c t ( n l ) ; 76

91 B.1. Lowpan-perf Code / B u i l d and send message / msg = n l m s g a l l o c ( ) ; f a m i l y = g e n l c t r l r e s o l v e ( nl, MAC ) ; g e n l m s g p u t ( msg, NL AUTO PID, NL AUTO SEQ, family, 0, NLM F ECHO, IEEE LIST IFACE, 1) ; n l a p u t s t r i n g ( msg, IEEE ATTR DEV NAME, wpan0 ) ; n l s e n d a u t o c o m p l e t e ( nl, msg ) ; n l m s g f r e e ( msg ) ; / Receive and p a r s e answer / n l r e c v ( nl, &nla, &buf, NULL) ; n l h = ( s t r u c t nlmsghdr ) buf ; g e n l m s g p a r s e ( nlh, 0, a t t r s, IEEE ATTR MAX, i e e e p o l i c y ) ; n l m s g d a t a ( n l h ) ; i f (! a t t r s [ IEEE ATTR SHORT ADDR ]! a t t r s [ IEEE ATTR SHORT ADDR ] ) r e t u r n 1 ; / We only h a n d l e s h o r t a d d r e s s e s r i g h t now / conf >p a n i d = n l a g e t u 1 6 ( a t t r s [ IEEE ATTR PAN ID ] ) ; conf >s r c a d d r = n l a g e t u 1 6 ( a t t r s [ IEEE ATTR SHORT ADDR ] ) ; f r e e ( buf ) ; n l c l o s e ( n l ) ; r e t u r n 0 ; void dump packet ( u n s i g n e d c h a r buf, i n t l e n ) i n t i ; f p r i n t f ( s t d o u t, P a c k e t p a y l o a d : ) ; f o r ( i = 0 ; i < l e n ; i ++) p r i n t f ( %x, buf [ i ] ) ; p r i n t f ( \ n ) ; i n t g e n e r a t e p a c k e t ( u n s i g n e d c h a r buf, s t r u c t c o n f i g conf, u n s i g n e d s h o r t seq num ) i n t i ; / Max p a y l o a d s i z e 115 b y t e / i f ( conf >p a c k e t l e n >= MAX PAYLOAD LEN) conf >p a c k e t l e n = MAX PAYLOAD LEN; / We have a t l e a s t 5 b y t e p a y l o a d f o r l e n g t h, f l a g s, e t c / i f ( conf >p a c k e t l e n < 5) conf >p a c k e t l e n = 5 ; / Use r o t a t i o n f o r p a c k e t l e n g t h f i e l d / i f ( conf >p a c k e t l e n r o t a t i o n ) conf >p a c k e t l e n = conf >p a c k e t l e n r o t a t i o n ; conf >p a c k e t l e n r o t a t i o n ++; i f ( conf >p a c k e t l e n r o t a t i o n == MAX PAYLOAD LEN + 1) conf >p a c k e t l e n r o t a t i o n = 5 ; buf [ 0 ] = conf >p a c k e t l e n ; buf [ 1 ] = conf >p a c k e t t y p e ; buf [ 2 ] = seq num >> 8 ; / Upper b y t e / buf [ 3 ] = seq num & 0xFF ; / Lower b y t e / f o r ( i = 4 ; i < conf >p a c k e t l e n ; i ++) 77

92 B. Code buf [ i ] = 0xAB ; r e t u r n 0 ; i n t p a r s e f l a g s ( s t r u c t c o n f i g conf, u n s i g n e d c h a r buf ) conf >p a c k e t l e n = buf [ 0 ] ; conf >p a c k e t t y p e = buf [ 1 ] ; i f ( conf >p a c k e t t y p e == PACKET CONFIG && buf [ 4 ]!= 0xAB && buf [ 5 ]!= 0xAB) conf >p a c k e t s = ( buf [ 4 ] << 8) buf [ 5 ] ; r e t u r n 0 ; i n t f i r e t h r o u g h p u t p a c k e t s ( s t r u c t c o n f i g conf, i n t sd ) u n s i g n e d c h a r buf ; i n t i ; conf >p a c k e t t y p e = PACKET THROUGHPUT; buf = ( u n s i g n e d c h a r ) m a l l oc (MAX PAYLOAD LEN) ; f o r ( i = 0 ; i < conf >p a c k e t s ; i ++) memset ( buf, 0, MAX PAYLOAD LEN) ; g e n e r a t e p a c k e t ( buf, conf, i ) ; send ( sd, buf, conf >p a c k e t l e n, 0) ; / / p r i n t f ( P a c k e t %i f i r e d \n, i ) ; f r e e ( buf ) ; r e t u r n 0 ; i n t m e a s u r e t h r o u g h p u t ( s t r u c t c o n f i g conf, i n t sd ) i n t i ; s s i z e t len, len sum = 1 ; u n s i g n e d c h a r buf ; s t r u c t t i m e v a l s t a r t t i m e, end time, t i m e o u t ; long sec, usec ; i n t c o u n t ; f l o a t t h r o u g h p u t ; u n s i g n e d s h o r t seq num ; f p r i n t f ( s t d o u t, S t a r t t h r o u g h p u t measurement... \ n ) ; f p r i n t f ( s t d o u t, CSV f o r m a t : Sequence number, p a c k e t l e n g t h \n ) ; conf >p a c k e t t y p e = PACKET CONFIG ; buf = ( u n s i g n e d c h a r ) m a l l o c (MAX PAYLOAD LEN) ; g e n e r a t e p a c k e t ( buf, conf, 1) ; buf [ 4 ] = conf >p a c k e t s >> 8 ; / Upper b y t e / buf [ 5 ] = conf >p a c k e t s & 0xFF ; / Lower b y t e / / / dump packet ( buf, 6) ; send ( sd, buf, 6, 0) ; / 2 s e c o n d s p a c k e t r e c e i v e t i m e o u t / t i m e o u t. t v s e c = 2 ; t i m e o u t. t v u s e c = 0 ; s e t s o c k o p t ( sd, SOL SOCKET, SO RCVTIMEO, ( s t r u c t t i m e v a l )&t i m e o u t, s i z e o f ( s t r u c t t i m e v a l ) ) ; 78

93 B.1. Lowpan-perf Code l e n = c o u n t = 0 ; f o r ( i = 0 ; i < conf >p a c k e t s ; i ++) l e n = r e c v ( sd, buf, MAX PAYLOAD LEN, 0) ; len sum += l e n ; seq num = ( buf [ 2 ] << 8) buf [ 3 ] ; / / p r i n t f ( P a c k e t with s e q u e n z e numer %i a r r i v e d \n, seq num ) ; i f ( c o u n t > seq num ) / / p r i n t f ( Sequenze number d i d n o t match. \ n ) ; / / c o n t i n u e ; / / p r i n t f ( Got %i, e x p e c t e d %i \n, seq num, c o u n t ) ; i f ( l e n > 0) i f ( i == 0) g e t t i m e o f d a y (& s t a r t t i m e, NULL) ; / / p r i n t f ( P a c k e t %i a r r i v e d \n, c o u n t ) ; c o u n t ++; e l s e f p r i n t f ( s t d e r r, H i t p a c k e t t i m e o u t \n ) ; f p r i n t f ( s t d o u t, %i,% i \n, ( ( buf [ 2 ] << 8) buf [ 3 ] ), ( i n t ) l e n ) ; memset ( buf, 0, MAX PAYLOAD LEN) ; g e t t i m e o f d a y (& end time, NULL) ; s e c = e n d t i m e. t v s e c s t a r t t i m e. t v s e c ; u sec = e n d t i m e. t v u s e c s t a r t t i m e. t v u s e c ; i f ( u sec < 0) u sec += ; sec ; t h r o u g h p u t = len sum / ( ( f l o a t ) s e c + ( f l o a t ) u s ec / ) ; f p r i n t f ( s t d o u t, Received %i b y t e s i n %l i s e c o n d s and %l i u s ec => %f Bytes / second \n, ( i n t ) len sum, sec, usec, t h r o u g h p u t ) ; f r e e ( buf ) ; r e t u r n 0 ; i n t m e a s u r e r o u n d t r i p ( s t r u c t c o n f i g conf, i n t sd ) u n s i g n e d c h a r buf ; s t r u c t t i m e v a l s t a r t t i m e, end time, t i m e o u t ; long s e c = 0, u sec = 0 ; long sec max = 0, usec max = 0 ; long s e c m i n = , usec min = ; long sum sec = 0, sum usec = 0 ; i n t i, r e t, c o u n t ; u n s i g n e d s h o r t seq num ; f p r i n t f ( s t d e r r, S t a r t r o u n d t r i p time measurement... \ n ) ; f p r i n t f ( s t d e r r, CSV f o r m a t : Sequence number, p a c k e t l e n g t h, r t t secs, r t t u s e c s \n ) ; conf >p a c k e t t y p e = PACKET ROUNDTRIP ; buf = ( u n s i g n e d c h a r ) m a l l o c (MAX PAYLOAD LEN) ; / 2 s e c o n d s p a c k e t r e c e i v e t i m e o u t / t i m e o u t. t v s e c = 2 ; t i m e o u t. t v u s e c = 0 ; s e t s o c k o p t ( sd, SOL SOCKET, SO RCVTIMEO, ( s t r u c t t i m e v a l )&t i m e o u t, s i z e o f ( s t r u c t t i m e v a l ) ) ; c o u n t = 0 ; f o r ( i = 0 ; i < conf >p a c k e t s ; i ++) 79

94 B. Code g e n e r a t e p a c k e t ( buf, conf, i ) ; seq num = ( buf [ 2 ] << 8) buf [ 3 ] ; send ( sd, buf, conf >p a c k e t l e n, 0) ; g e t t i m e o f d a y (& s t a r t t i m e, NULL) ; r e t = r e c v ( sd, buf, conf >p a c k e t l e n, 0) ; i f ( seq num!= ( ( buf [ 2 ] << 8) buf [ 3 ] ) ) / / p r i n t f ( Sequenze number d i d n o t match \n ) ; c o n t i n u e ; i f ( r e t > 0) g e t t i m e o f d a y (& end time, NULL) ; c o u n t ++; s e c = e n d t i m e. t v s e c s t a r t t i m e. t v s e c ; sum sec += s e c ; u sec = e n d t i m e. t v u s e c s t a r t t i m e. t v u s e c ; i f ( u s ec < 0) u sec += ; sec ; sum sec ; f p r i n t f ( s t d o u t, %i,% i,% l i,% l i \n, ( i n t ) seq num, r e t, sec, usec ) ; sum usec += usec ; i f ( s e c > sec max ) sec max = s e c ; e l s e i f ( s e c < s e c m i n ) s e c m i n = s e c ; i f ( u s ec > usec max ) usec max = usec ; e l s e i f ( usec < usec min ) usec min = usec ; / / p r i n t f ( Pong i n %l i s e c o n d s and %l i u s e c s \n, sec, usec ) ; e l s e f p r i n t f ( s t d o u t, %i,% i,, \ n, ( i n t ) seq num, conf >p a c k e t l e n ) ; / / f p r i n t f ( s t d e r r, H i t p a c k e t t i m e o u t \n ) ; f p r i n t f ( s t d e r r, Received %i from %i p a c k e t s \n, count, conf >p a c k e t s ) ; f p r i n t f ( s t d e r r, A r i t h m e t i c mean r o u n t r i p time : %f s e c o n d s and %f u s e c s \n, ( f l o a t ) sum sec / ( f l o a t ) count, ( f l o a t ) sum usec / ( f l o a t ) c o u n t ) ; f p r i n t f ( s t d e r r, Minimal time %f s e c o n d s and %f u s e c s \n, ( f l o a t ) sec min, ( f l o a t ) usec min ) ; f p r i n t f ( s t d e r r, Maximal time %f s e c o n d s and %f u s e c s \n, ( f l o a t ) sec max, ( f l o a t ) usec max ) ; f r e e ( buf ) ; r e t u r n 0 ; void i n i t s e r v e r ( s t r u c t c o n f i g conf, i n t sd ) s s i z e t l e n ; u n s i g n e d c h a r buf ; / / s t r u c t s o c k a d d r i e e e s r c ; / / s o c k l e n t a d d r l e n ; / / a d d r l e n = s i z e o f ( s r c ) ; l e n = 0 ; f p r i n t f ( s t d o u t, S e r v e r mode. Waiting f o r p a c k e t s... \ n ) ; buf = ( u n s i g n e d c h a r ) m a l l o c (MAX PAYLOAD LEN) ; w h i l e ( 1 ) / / l e n = r e c v f r o m ( sd, buf, MAX PAYLOAD LEN, 0, ( s t r u c t s o c k a d d r )&s r c, &a d d r l e n ) ; 80

95 B.1. Lowpan-perf Code l e n = r e c v ( sd, buf, MAX PAYLOAD LEN, 0) ; / / p r i n t f ( Received %zd b y t e s, l e n ) ; / / dump packet ( buf, l e n ) ; p a r s e f l a g s ( conf, buf ) ; / R o u n d t r i p mode : send same p a c k e t back / i f ( conf >p a c k e t t y p e == PACKET ROUNDTRIP) / / p r i n t f ( E n t e r e d r o u n d t r i p mode\n ) ; send ( sd, buf, len, 0) ; i f ( conf >p a c k e t t y p e == PACKET CONFIG) / / p r i n t f ( E n t e r e d t h r o u g h p u t mode\n ) ; f i r e t h r o u g h p u t p a c k e t s ( conf, sd ) ; f r e e ( buf ) ; i n t i n i t n e t w o r k ( s t r u c t c o n f i g c onf ) i n t sd ; i n t r e t ; s t r u c t s o c k a d d r i e e e a ; sd = s o c k e t ( PF IEEE802154, SOCK DGRAM, 0) ; i f ( sd < 0) p e r r o r ( s o c k e t ) ; r e t u r n 1 ; g e t i n t e r f a c e i n f o ( conf ) ; a. f a m i l y = AF IEEE ; a. addr. a d d r t y p e = IEEE ADDR SHORT ; a. addr. p a n i d = conf >p a n i d ; / Bind s o c k e t on t h i s s i d e / a. addr. s h o r t a d d r = conf >s r c a d d r ; r e t = b ind ( sd, ( s t r u c t s o c k a d d r )&a, s i z e o f ( a ) ) ; i f ( r e t ) p e r r o r ( b ind ) ; r e t u r n 1 ; / Connect t o o t h e r s i d e / a. addr. s h o r t a d d r = conf >d s t a d d r ; r e t = c o n n e c t ( sd, ( s t r u c t s o c k a d d r )&a, s i z e o f ( a ) ) ; i f ( r e t ) p e r r o r ( c o n n e c t ) ; r e t u r n 1 ; i f ( conf >s e r v e r ) i n i t s e r v e r ( conf, sd ) ; i f ( conf >mode == MODE ROUNDTRIP conf >mode == MODE BOTH) m e a s u r e r o u n d t r i p ( conf, sd ) ; i f ( conf >mode == MODE THROUGHPUT conf >mode == MODE BOTH) m e a s u r e t h r o u g h p u t ( conf, sd ) ; f r e e ( c o nf ) ; shutdown ( sd, SHUT RDWR) ; c l o s e ( sd ) ; 81

96 B. Code r e t u r n 0 ; i n t main ( i n t argc, c h a r a rgv [ ] ) i n t c ; s t r u c t c o n f i g c onf ; c onf = ( s t r u c t c o n f i g ) m a l l o c ( s i z e o f ( s t r u c t c o n f i g ) ) ; / D e a f u l t s t a t e i f n o t s e t on t h e c l i / conf >p a c k e t l e n r o t a t i o n = 5 ; i f ( a r g c < 2) usage ( a rgv [ 0 ] ) ; e x i t ( 1 ) ; w h i l e ( 1 ) # i f d e f HAVE GETOPT LONG i n t o p t i d x = 1; c = g e t o p t l o n g ( argc, argv, c : p : l : s : r t v h, p e r f l o n g o p t s, &o p t i d x ) ; # e l s e c = g e t o p t ( argc, argv, c : p : l : s : r t v h ) ; # e n d i f i f ( c == 1) b r e a k ; s w i t c h ( c ) c a s e c : conf >d s t a d d r = s t r t o l ( o p t a r g, NULL, 16) ; b r e a k ; c a s e s : conf >s e r v e r = 1 ; conf >d s t a d d r = s t r t o l ( o p t a r g, NULL, 16) ; b r e a k ; b r e a k ; c a s e p : conf >p a c k e t s = a t o i ( o p t a r g ) ; b r e a k ; c a s e l : conf >p a c k e t l e n = a t o i ( o p t a r g ) ; conf >p a c k e t l e n r o t a t i o n = 0 ; b r e a k ; c a s e r : i f ( conf >mode == MODE THROUGHPUT) conf >mode = MODE BOTH; e l s e conf >mode = MODE ROUNDTRIP; b r e a k ; c a s e t : i f ( conf >mode == MODE ROUNDTRIP) conf >mode = MODE BOTH; e l s e conf >mode = MODE THROUGHPUT; b r e a k ; c a s e v : f p r i n t f ( s t d o u t, lowpan p e r f 0.1\ n ) ; r e t u r n 1 ; c a s e h : usage ( a rgv [ 0 ] ) ; r e t u r n 1 ; d e f a u l t : usage ( a rgv [ 0 ] ) ; 82

97 B.2. IBR-DTN Convergence Layer Code r e t u r n 1 ; i n i t n e t w o r k ( c onf ) ; r e t u r n 0 ; B.2. IBR-DTN Convergence Layer Code Listing B.2: LOWPANConvergenceLayer.cpp # i n c l u d e n e t / LOWPANConvergenceLayer. h # i n c l u d e n e t / LOWPANConnection. h # i n c l u d e n e t / lowpanstream. h # i n c l u d e c o r e / BundleCore. h # i n c l u d e c o r e / TimeEvent. h # i n c l u d e <ibrcommon / n e t / UnicastSocketLowpan. h> # i n c l u d e <ibrcommon / n e t / l o w p a n s o c k e t. h> # i n c l u d e <ibrcommon / Logger. h> # i n c l u d e <ibrcommon / t h r e a d / MutexLock. h> # i n c l u d e <ibrcommon / TimeMeasurement. h> # i n c l u d e <i b r d t n / d a t a / ScopeControlHopLimitBlock. h> # i n c l u d e <s y s / s o c k e t. h> # i n c l u d e <s t d l i b. h> # i n c l u d e <s t d i o. h> # i n c l u d e <s t r i n g. h> # i n c l u d e <i o s t r e a m > # i n c l u d e <l i s t > # d e f i n e EXTENDED MASK 0 x08 # d e f i n e SEQ NUM MASK 0 x07 u s i n g namespace d t n : : d a t a ; namespace d t n namespace n e t LOWPANConvergenceLayer : : LOWPANConvergenceLayer ( ibrcommon : : v i n t e r f a c e net, i n t panid, u n s i g n e d i n t mtu ) : DiscoveryAgent ( d t n : : daemon : : C o n f i g u r a t i o n : : g e t I n s t a n c e ( ). g e t D i s c o v e r y ( ) ), s o c k e t (NULL), n e t ( n e t ), p a n i d ( p a n i d ), m maxmsgsize ( mtu ), r u n n i n g ( f a l s e ), i p n d b u f l e n ( 0 ), i p n d b u f ( new c h a r [ BUFF SIZE + 2 ] ) s o c k e t = new ibrcommon : : UnicastSocketLowpan ( ) ; LOWPANConvergenceLayer : : LOWPANConvergenceLayer ( ) componentdown ( ) ; d e l e t e s o c k e t ; 83

98 B. Code d t n : : c o r e : : Node : : P r o t o c o l LOWPANConvergenceLayer : : g e t D i s c o v e r y P r o t o c o l ( ) c o n s t r e t u r n d t n : : c o r e : : Node : : CONN LOWPAN; void LOWPANConvergenceLayer : : u p d a t e ( c o n s t ibrcommon : : v i n t e r f a c e &i f a c e, s t d : : s t r i n g &name, s t d : : s t r i n g &params ) throw ( d t n : : n e t : : D i s c o v e r y S e r v i c e P r o v i d e r : : N o S e r v i c e H e r e E x c e p t i o n ) i f ( i f a c e == n e t ) name = l o w p a n c l ; s t r i n g s t r e a m s e r v i c e ; s t r u c t s o c k a d d r i e e e a d d r e s s ; a d d r e s s. addr. a d d r t y p e = IEEE ADDR SHORT ; a d d r e s s. addr. p a n i d = p a n i d ; / / Get a d d r e s s v i a n e t l i n k ibrcommon : : l o w p a n s o c k e t : : g e t A d d r e s s (& a d d r e s s. addr, i f a c e ) ; / / FIXME b e t t e r naming f o r a d d r e s s and p a n i d. This w i l l need u p d a t e s t o t h e s e r v i c e p a r s e r. s e r v i c e << i p = << a d d r e s s. a d dr. s h o r t a d d r << ; p o r t = << p a n i d << ; ; params = s e r v i c e. s t r ( ) ; e l s e throw d t n : : n e t : : D i s c o v e r y S e r v i c e P r o v i d e r : : N o S e r v i c e H e r e E x c e p t i o n ( ) ; void LOWPANConvergenceLayer : : s e n d c b ( c h a r buf, i n t len, u n s i g n e d i n t a d d r e s s ) / / Add own a d d r e s s a t t h e end s t r u c t s o c k a d d r i e e e s o c k a d d r ; u n s i g n e d s h o r t l o c a l a d d r ; s o c k e t >g e t A d d r e s s (& s o c k a d d r. addr, n e t ) ; l o c a l a d d r = s o c k a d d r. addr. s h o r t a d d r ; / / g e t a lowpan p e e r ibrcommon : : l o w p a n s o c k e t : : p e e r p = s o c k e t >g e t P e e r ( a d d r e s s, s o c k a d d r. addr. p a n i d ) ; i f ( l e n > 113) IBRCOMMON LOGGER( e r r o r ) << LOWPANConvergenceLayer : : s e n d c b b u f f e r t o b i g t o be t r a n s f e r r e d ( << l e n << ). << IBRCOMMON LOGGER ENDL; / / Add own a d d r e s s a t t h e end memcpy(& buf [ l e n ], &l o c a l a d d r, 2) ; / / s e t w r i t e l o c k ibrcommon : : MutexLock l ( m w r i t e l o c k ) ; / / send c o n v e r t e d l i n e i n t r e t = p. send ( buf, l e n + 2) ; i f ( r e t == 1) / / CL i s busy 84

99 B.2. IBR-DTN Convergence Layer Code throw ( ibrcommon : : E x c e p t i o n ( Send on s o c k e t f a i l e d ) ) ; void LOWPANConvergenceLayer : : queue ( c o n s t d t n : : c o r e : : Node &node, c o n s t ConvergenceLayer : : Job &j o b ) c o n s t s t d : : l i s t <d t n : : c o r e : : Node : : URI> u r i l i s t = node. g e t ( d t n : : c o r e : : Node : : CONN LOWPAN) ; i f ( u r i l i s t. empty ( ) ) r e t u r n ; c o n s t d t n : : c o r e : : Node : : URI &u r i = u r i l i s t. f r o n t ( ) ; s t d : : s t r i n g a d d r e s s ; u n s i g n e d i n t pan ; u r i. decode ( a d d r e s s, pan ) ; IBRCOMMON LOGGER DEBUG( 1 0 ) << LOWPANConvergenceLayer : : queue << IBRCOMMON LOGGER ENDL; ibrcommon : : MutexLock l c ( c o n n e c t i o n l o c k ) ; g e t C o n n e c t i o n ( a t o i ( a d d r e s s. c s t r ( ) ) ) > s e n d e r. queue ( j o b ) ; LOWPANConnection LOWPANConvergenceLayer : : g e t C o n n e c t i o n ( u n s i g n e d s h o r t a d d r e s s ) / / T e s t i f c o n n e c t i o n f o r t h i s a d d r e s s a l r e a d y e x i s t s t d : : l i s t <LOWPANConnection >:: i t e r a t o r i ; f o r ( i = C o n n e c t i o n L i s t. b e g i n ( ) ; i!= C o n n e c t i o n L i s t. end ( ) ; ++ i ) IBRCOMMON LOGGER DEBUG( 1 0 ) << C o n n e c t i o n a d d r e s s : << hex << ( i ) > a d d r e s s << IBRCOMMON LOGGER ENDL; i f ( ( i ) > a d d r e s s == a d d r e s s ) r e t u r n ( i ) ; / / C o n n e c t i o n does n o t e x i s t, c r e a t e one and p u t i t i n t o t h e l i s t LOWPANConnection c o n n e c t i o n = new LOWPANConnection ( a d d r e s s, ( t h i s ) ) ; C o n n e c t i o n L i s t. p u s h b a c k ( c o n n e c t i o n ) ; IBRCOMMON LOGGER DEBUG( 1 0 ) << LOWPANConvergenceLayer : : g e t C o n n e c t i o n << c o n n e c t i o n > a d d r e s s << IBRCOMMON LOGGER ENDL; c o n n e c t i o n >s t a r t ( ) ; r e t u r n c o n n e c t i o n ; void LOWPANConvergenceLayer : : remove ( c o n s t LOWPANConnection conn ) ibrcommon : : MutexLock l c ( c o n n e c t i o n l o c k ) ; s t d : : l i s t <LOWPANConnection >:: i t e r a t o r i ; f o r ( i = C o n n e c t i o n L i s t. b e g i n ( ) ; i!= C o n n e c t i o n L i s t. end ( ) ; ++ i ) i f ( ( i ) == conn ) C o n n e c t i o n L i s t. e r a s e ( i ) ; r e t u r n ; 85

100 B. Code void LOWPANConvergenceLayer : : componentup ( ) b i n d E v e n t ( d t n : : c o r e : : TimeEvent : : classname ) ; t r y t r y ibrcommon : : UnicastSocketLowpan &sock = d y n a m i c c a s t <ibrcommon : : UnicastSocketLowpan&>( s o c k e t ) ; sock. bind ( p a n i d, n e t ) ; a d d S e r v i c e ( t h i s ) ; c a t c h ( c o n s t s t d : : b a d c a s t &) c a t c h ( c o n s t ibrcommon : : l o w p a n s o c k e t : : S o c k e t E x c e p t i o n &ex ) IBRCOMMON LOGGER DEBUG( 1 0 ) << F a i l e d t o add LOWPAN ConvergenceLayer on << n e t. t o S t r i n g ( ) << : << p a n i d << IBRCOMMON LOGGER ENDL; IBRCOMMON LOGGER DEBUG( 1 0 ) << E x c e p t i o n : << ex. what ( ) << IBRCOMMON LOGGER ENDL; void LOWPANConvergenceLayer : : componentdown ( ) unbindevent ( d t n : : c o r e : : TimeEvent : : classname ) ; r u n n i n g = f a l s e ; s o c k e t >shutdown ( ) ; j o i n ( ) ; void LOWPANConvergenceLayer : : sendannoucement ( c o n s t u i n t 1 6 t &sn, s t d : : l i s t <d t n : : n e t : : D i s c o v e r y S e r v i c e > &s e r v i c e s ) IBRCOMMON LOGGER( n o t i c e ) << LOWPAN IPND beacon send s t a r t e d << IBRCOMMON LOGGER ENDL; DiscoveryAnnouncement announcement ( DiscoveryAnnouncement : : DISCO VERSION 01, d t n : : c o r e : : BundleCore : : l o c a l ) ; / / s e t sequencenumber announcement. setsequencenumber ( sn ) ; / / c l e a r a l l s e r v i c e s announcement. c l e a r S e r v i c e s ( ) ; / / add s e r v i c e s f o r ( s t d : : l i s t <D i s c o v e r y S e r v i c e >:: i t e r a t o r i t e r = s e r v i c e s. b e g i n ( ) ; i t e r!= s e r v i c e s. end ( ) ; i t e r ++) D i s c o v e r y S e r v i c e &s e r v i c e = ( i t e r ) ; t r y / / u p d a t e s e r v i c e i n f o r m a t i o n s e r v i c e. u p d a t e ( n e t ) ; / / add s e r v i c e t o d i s c o v e r y message announcement. a d d S e r v i c e ( s e r v i c e ) ; c a t c h ( c o n s t d t n : : n e t : : D i s c o v e r y S e r v i c e P r o v i d e r : : N o S e r v i c e H e r e E x c e p t i o n &) / / S e t e x t e n d e d h e a d e r b i t. E v e r y t h i n g e l s e 0 i p n d b u f [ 0 ] = 0 x08 ; 86

101 B.2. IBR-DTN Convergence Layer Code / / S e t d i s c o v e r y b i t i n e x t e n d e d h e a d e r i p n d b u f [ 1 ] = 0 x80 ; / / s e r i a l i z e announcement s t r i n g s t r e a m s s ; s s << announcement ; i n t l e n = s s. s t r ( ). s i z e ( ) ; i f ( l e n > 111) IBRCOMMON LOGGER( e r r o r ) << D i s c o v e r y announcement t o b i g ( << l e n << ) << IBRCOMMON LOGGER ENDL; memcpy ( i p n d b u f +2, s s. s t r ( ). c s t r ( ), l e n ) ; s e n d c b ( i p n d b u f, l e n + 2, 0 x f f f f ) ; void LOWPANConvergenceLayer : : componentrun ( ) r u n n i n g = t r u e ; w h i l e ( r u n n i n g ) ibrcommon : : MutexLock l ( m r e a d l o c k ) ; c h a r d a t a [ m maxmsgsize ] ; c h a r h e a d e r ; u n s i g n e d s h o r t a d d r e s s ; IBRCOMMON LOGGER DEBUG( 1 0 ) << LOWPANConvergenceLayer : : componentrun e a r l y << IBRCOMMON LOGGER ENDL; / / Receive f u l l frame from s o c k e t i n t l e n = s o c k e t >r e c e i v e ( data, m maxmsgsize ) ; IBRCOMMON LOGGER DEBUG( 1 0 ) << LOWPANConvergenceLayer : : componentrun << IBRCOMMON LOGGER ENDL; / / We g o t n o t h i n g from t h e s o c k e t, keep r e a d i n g i f ( l e n <= 0) c o n t i n u e ; / / R e t r i e v e h e a d e r of frame h e a d e r = d a t a [ 0 ] ; / / R e t r i e v e s e n d e r a d d r e s s from t h e end of t h e frame a d d r e s s = ( ( c h a r ) d a t a [ len 1] << 8) d a t a [ len 2]; IBRCOMMON LOGGER( n o t i c e ) << ADDRESS << a d d r e s s << IBRCOMMON LOGGER ENDL; / / Check f o r e x t e n d e d h e a d e r and r e t r i e v e i f a v a i l a b l e i f ( ( h e a d e r & EXTENDED MASK) && ( d a t a [ 1 ] & 0 x80 ) ) IBRCOMMON LOGGER( n o t i c e ) << Received announcement f o r LoWPAN d i s c o v e r y << IBRCOMMON LOGGER ENDL; DiscoveryAnnouncement announce ; s t r i n g s t r e a m s s ; s s. w r i t e ( d a t a +2, len 4) ; s s >> announce ; DiscoveryAgent : : r e c e i v e d ( announce, 30) ; c o n t i n u e ; ibrcommon : : MutexLock l c ( c o n n e c t i o n l o c k ) ; 87

102 B. Code / / C o n n e c t i o n i n s t a n c e f o r t h i s a d d r e s s LOWPANConnection c o n n e c t i o n = g e t C o n n e c t i o n ( a d d r e s s ) ; / / Decide i n which queue t o w r i t e based on t h e s r c a d d r e s s c o n n e c t i o n >g e t S t r e a m ( ). queue ( data, len 2) ; / / Cut o f f a d d r e s s from end y i e l d ( ) ; void LOWPANConvergenceLayer : : r a i s e E v e n t ( c o n s t Event e v t ) t r y c o n s t TimeEvent &t ime = d y n a m i c c a s t <c o n s t TimeEvent&>( e v t ) ; i f ( time. g e t A c t i o n ( ) == TIME SECOND TICK ) i f ( time. gettimestamp ( ) %5 == 0) t i m e o u t ( ) ; c a t c h ( c o n s t s t d : : b a d c a s t &) bool LOWPANConvergenceLayer : : c a n c e l l a t i o n ( ) / / s i n c e t h i s i s an r e c e i v i n g t h r e a d we have t o c a n c e l t h e h a rd way r e t u r n f a l s e ; c o n s t s t d : : s t r i n g LOWPANConvergenceLayer : : getname ( ) c o n s t r e t u r n LOWPANConvergenceLayer ; # i f n d e f LOWPANCONVERGENCELAYER H # d e f i n e LOWPANCONVERGENCELAYER H # i n c l u d e Component. h # i n c l u d e n e t / ConvergenceLayer. h # i n c l u d e n e t / LOWPANConnection. h # i n c l u d e n e t / DiscoveryAgent. h # i n c l u d e n e t / D i s c o v e r y S e r v i c e P r o v i d e r. h # i n c l u d e <ibrcommon / n e t / v i n t e r f a c e. h> # i n c l u d e <ibrcommon / n e t / l o w p a n s o c k e t. h> # i n c l u d e <l i s t > u s i n g namespace d t n : : d a t a ; Listing B.3: LOWPANConvergenceLayer.h namespace d t n namespace n e t c l a s s LOWPANConnection ; / This c l a s s implements a ConvergenceLayer f o r LOWPAN. / c l a s s LOWPANConvergenceLayer : p u b l i c DiscoveryAgent, p u b l i c ConvergenceLayer, 88

103 B.2. IBR-DTN Convergence Layer Code p u b l i c d t n : : daemon : : IndependentComponent, p u b l i c l o w p a n s t r e a m c a l l b a c k, p u b l i c E v e n t R e c e i v e r, p u b l i c D i s c o v e r y S e r v i c e P r o v i d e r p u b l i c : LOWPANConvergenceLayer ( ibrcommon : : v i n t e r f a c e net, i n t panid, u n s i g n e d i n t mtu = 115) ; / /MTU i s a c t u a l l y v i r t u a l LOWPANConvergenceLayer ( ) ; / t h i s method u p d a t e s t h e g i v e n v a l u e s / void u p d a t e ( c o n s t ibrcommon : : v i n t e r f a c e &i f a c e, s t d : : s t r i n g &name, s t d : : s t r i n g & d a t a ) throw ( d t n : : n e t : : D i s c o v e r y S e r v i c e P r o v i d e r : : N o S e r v i c e H e r e E x c e p t i o n ) ; d t n : : c o r e : : Node : : P r o t o c o l g e t D i s c o v e r y P r o t o c o l ( ) c o n s t ; / Queueing a j o b f o r a s p e c i f i c node. S t a r t i n g p o i n t f o r t h e DTN c o r e t o s u b m i t b u n d l e s t o nodes b e h i n d t h e LoWPAN n Node r e f e r e n c j o b Job r e f e r e n c e / void queue ( c o n s t d t n : : c o r e : : Node &n, c o n s t ConvergenceLayer : : Job &j o b ) ; Component : : getname ( ) / v i r t u a l c o n s t s t d : : s t r i n g getname ( ) c o n s t ; v i r t u a l void r a i s e E v e n t ( c o n s t Event e v t ) ; / C a l l b a c k i n t e r f a c e f o r s e n d i n g d a t a back from t h e lowpanstream t o t h e buf B u f f e r with a d a t a l e n Length of t h e b u f f e a d d r e s s IEEE s h o r t a d d r e s s of t h e d e s t i n a t i o n / v i r t u a l void s e n d c b ( c h a r buf, i n t len, u n s i g n e d i n t a d d r e s s ) ; s t a t i c c o n s t s i z e t BUFF SIZE = 113; void remove ( c o n s t LOWPANConnection conn ) ; p r o t e c t e d : v i r t u a l void componentup ( ) ; v i r t u a l void componentrun ( ) ; v i r t u a l void componentdown ( ) ; bool c a n c e l l a t i o n ( ) ; v i r t u a l void sendannoucement ( c o n s t u i n t 1 6 t &sn, s t d : : l i s t <d t n : : n e t : : D i s c o v e r y S e r v i c e > &s e r v i c e s ) ; p r i v a t e : ibrcommon : : l o w p a n s o c k e t s o c k e t ; ibrcommon : : v i n t e r f a c e n e t ; i n t p a n i d ; c h a r i p n d b u f ; i n t i p n d b u f l e n ; ibrcommon : : Mutex c o n n e c t i o n l o c k ; 89

104 B. Code s t d : : l i s t <LOWPANConnection > C o n n e c t i o n L i s t ; LOWPANConnection g e t C o n n e c t i o n ( u n s i g n e d s h o r t a d d r e s s ) ; u n s i g n e d i n t m maxmsgsize ; ibrcommon : : Mutex m w r i t e l o c k ; ibrcommon : : Mutex m r e a d l o c k ; bool r u n n i n g ; ; # e n d i f / LOWPANCONVERGENCELAYER H / # i n c l u d e n e t / LOWPANConnection. h # i n c l u d e n e t / BundleReceivedEvent. h # i n c l u d e c o r e / BundleEvent. h # i n c l u d e n e t / T r a n s f e r C o m p l e t e d E v e n t. h # i n c l u d e n e t / T r a n s f e r A b o r t e d E v e n t. h # i n c l u d e r o u t i n g / RequeueBundleEvent. h # i n c l u d e c o r e / BundleCore. h # i n c l u d e <ibrcommon / Logger. h> # i n c l u d e <ibrcommon / t h r e a d / MutexLock. h> Listing B.4: LOWPANConnection.cpp # i n c l u d e <i b r d t n / d a t a / ScopeControlHopLimitBlock. h> # i n c l u d e <i b r d t n / u t i l s / U t i l s. h> # i n c l u d e <i b r d t n / d a t a / S e r i a l i z e r. h> # i n c l u d e <u n i s t d. h> # i n c l u d e <s t d l i b. h> # i n c l u d e <s t d i o. h> # i n c l u d e <s t r i n g. h> # i n c l u d e <i o s t r e a m > # i n c l u d e <l i s t > u s i n g namespace d t n : : d a t a ; namespace d t n namespace n e t LOWPANConnection : : LOWPANConnection ( u n s i g n e d s h o r t a d d r e s s, LOWPANConvergenceLayer & c l ) : a d d r e s s ( a d d r e s s ), s t r e a m ( cl, a d d r e s s ), s e n d e r ( s t r e a m ), c l ( c l ) LOWPANConnection : : LOWPANConnection ( ) lowpanstream& LOWPANConnection : : g e t S t r e a m ( ) r e t u r n s t r e a m ; void LOWPANConnection : : s e t u p ( ) 90

105 B.2. IBR-DTN Convergence Layer Code s e n d e r. s t a r t ( ) ; void LOWPANConnection : : f i n a l l y ( ) s e n d e r. s t o p ( ) ; s e n d e r. j o i n ( ) ; / / remove t h i s c o n n e c t i o n from t h e c o n n e c t i o n l i s t c l. remove ( t h i s ) ; void LOWPANConnection : : run ( ) t r y w h i l e ( s t r e a m. good ( ) ) t r y d t n : : d a t a : : D e f a u l t D e s e r i a l i z e r d e s e r i a l i z e r ( s t r e a m ) ; d t n : : d a t a : : Bundle b u n d l e ; d e s e r i a l i z e r >> b u n d l e ; IBRCOMMON LOGGER DEBUG( 1 0 ) << LOWPANConnection : : run << IBRCOMMON LOGGER ENDL; / / d e t e r m i n e s e n d e r EID s e n d e r ; / / i n c r e m e n t v a l u e i n t h e scope c o n t r o l hop l i m i t b l o c k t r y d t n : : d a t a : : ScopeControlHopLimitBlock &s c h l = b u n d l e. getblock<d t n : : d a t a : : ScopeControlHopLimitBlock >() ; s c h l. i n c r e m e n t ( ) ; c a t c h ( c o n s t d t n : : d a t a : : Bundle : : NoSuchBlockFoundException &) ; / / r a i s e d e f a u l t b u n d l e r e c e i v e d e v e n t d t n : : n e t : : BundleReceivedEvent : : r a i s e ( sender, b u n d l e ) ; c a t c h ( c o n s t d t n : : I n v a l i d D a t a E x c e p t i o n &ex ) IBRCOMMON LOGGER DEBUG( 1 0 ) << Received a i n v a l i d b u n d l e : << ex. what ( ) << IBRCOMMON LOGGER ENDL; c a t c h ( c o n s t ibrcommon : : IOException &) c a t c h ( s t d : : e x c e p t i o n &ex ) IBRCOMMON LOGGER DEBUG( 1 0 ) << Thread d i e d : << ex. what ( ) << IBRCOMMON LOGGER ENDL; / / c l a s s LOWPANConnectionSender LOWPANConnectionSender : : LOWPANConnectionSender ( lowpanstream &s t r e a m ) : s t r e a m ( s t r e a m ) LOWPANConnectionSender : : LOWPANConnectionSender ( ) 91

106 B. Code void LOWPANConnectionSender : : queue ( c o n s t ConvergenceLayer : : Job &j o b ) IBRCOMMON LOGGER DEBUG( 1 0 ) << : LOWPANConnectionSender : : queue << IBRCOMMON LOGGER ENDL; queue. push ( j o b ) ; void LOWPANConnectionSender : : run ( ) t r y w h i l e ( s t r e a m. good ( ) ) ConvergenceLayer : : Job j o b = queue. getnpop ( t r u e ) ; d t n : : d a t a : : D e f a u l t S e r i a l i z e r s e r i a l i z e r ( s t r e a m ) ; IBRCOMMON LOGGER DEBUG( 1 0 ) << : LOWPANConnectionSender : : run << IBRCOMMON LOGGER ENDL; d t n : : c o r e : : B u n d l e S t o r a g e &s t o r a g e = d t n : : c o r e : : BundleCore : : g e t I n s t a n c e ( ). g e t S t o r a g e ( ) ; / / r e a d t h e b u n d l e o u t of t h e s t o r a g e c o n s t d t n : : d a t a : : Bundle b u n d l e = s t o r a g e. g e t ( j o b. b u n d l e ) ; / / Put b u n d l e i n t o s t r i n g s t r e a m s e r i a l i z e r << b u n d l e ; s t r e a m. f l u s h ( ) ; / / r a i s e b u n d l e e v e n t d t n : : n e t : : T r a n s f e r C o m p l e t e d E v e n t : : r a i s e ( j o b. d e s t i n a t i o n, b u n d l e ) ; d t n : : c o r e : : BundleEvent : : r a i s e ( bundle, d t n : : c o r e : : BUNDLE FORWARDED) ; / / FIXME : E x i t s t r a t e g y when s e n d i n g on s o c k e t f a i l e d. Like d e s t r o y i n g t h e c o n n e c t i o n o b j e c t / / Also check what needs t o be done when t h e node i s n o t r e a c h a b l e ( t r a n s f e r r e q u e u e... ) IBRCOMMON LOGGER DEBUG( 1 0 ) << : LOWPANConnectionSender : : run s t r e a m d e s t r o y e d << IBRCOMMON LOGGER ENDL; c a t c h ( s t d : : e x c e p t i o n &ex ) IBRCOMMON LOGGER DEBUG( 1 0 ) << Thread d i e d : << ex. what ( ) << IBRCOMMON LOGGER ENDL; bool LOWPANConnectionSender : : c a n c e l l a t i o n ( ) queue. a b o r t ( ) ; r e t u r n t r u e ; # i f n d e f LOWPANCONNECTION H # d e f i n e LOWPANCONNECTION H # i n c l u d e ibrcommon / t h r e a d / Queue. h # i n c l u d e lowpanstream. h # i n c l u d e n e t / LOWPANConvergenceLayer. h u s i n g namespace d t n : : d a t a ; Listing B.5: LOWPANConnection.h 92

107 B.2. IBR-DTN Convergence Layer Code namespace d t n namespace n e t c l a s s lowpanstream ; c l a s s LOWPANConnectionSender : p u b l i c ibrcommon : : J o i n a b l e T h r e a d p u b l i c : LOWPANConnectionSender ( lowpanstream & s t r e a m ) ; v i r t u a l LOWPANConnectionSender ( ) ; / Queueing a job, o r i g i n a l l y coming from t h e DTN core, t o work i t s way down t o t h e CL t h r o u g h t h e j o b R e f e r e n c e t o t h e j o b c o n a t i n i n g EID and b u n d l e / void queue ( c o n s t ConvergenceLayer : : Job &j o b ) ; void run ( ) ; bool c a n c e l l a t i o n ( ) ; p r i v a t e : lowpanstream & s t r e a m ; ibrcommon : : Queue<ConvergenceLayer : : Job> queue ; ; c l a s s LOWPANConvergenceLayer ; c l a s s LOWPANConnection : p u b l i c ibrcommon : : DetachedThread p u b l i c : / LOWPANConnection c o n s t r u c t o a d d r e s s IEEE s h o r t a d d r e s s t o i d e n t f y t h e c o n n e c t i o LOWPANConvergenceLayer r e f e r e n c e / LOWPANConnection ( u n s i g n e d s h o r t a d d r e s s, LOWPANConvergenceLayer &c l ) ; v i r t u a l LOWPANConnection ( ) ; / IEEE s h o r t a d d r e s s of t h e node t h i s c o n n e c t i o n h a n d l e s d a t a f o r. / u n s i g n e d s h o r t a d d r e s s ; / G e t t i n g t h e lowpanstream c o n n e c t e d with t h e lowpanstream r e f e r e n c e / lowpanstream& g e t S t r e a m ( ) ; void run ( ) ; void s e t u p ( ) ; void f i n a l l y ( ) ; / I n s t a n c e of t h e LOWPANConnectionSender / LOWPANConnectionSender s e n d e r ; p r i v a t e : bool r u n n i n g ; lowpanstream s t r e a m ; 93

108 B. Code LOWPANConvergenceLayer & c l ; ; # e n d i f / LOWPANCONNECTION H / # i n c l u d e ibrcommon / c o n f i g. h # i n c l u d e ibrcommon / Logger. h # i n c l u d e lowpanstream. h # i n c l u d e n e t / LOWPANConvergenceLayer. h # i n c l u d e ibrcommon / t h r e a d / MutexLock. h # i n c l u d e <s i g n a l. h> # i n c l u d e <s t r i n g. h> # i n c l u d e <math. h> Listing B.6: lowpanstream.cpp / Header : B i t 7 6: 00 t o be c o m p a t i b l e with 6LoWPAN B i t 5 4: 00 Middle segment 01 L a s t segment 10 F i r s t segment 11 F i r s t and l a s t segment B i t 3 : 0 Extended frame n o t a v a i l a b l e 1 Extended frame a v a i l a b l e B i t 2 0: s e q u e n c e number (0 7) Extended h e a d e r ( only i f e x t e n d e d frame a v a i l a b l e ) B i t 7 : 0 No d i s c o v e r y frame 1 D i s c o v e r y frame B i t 6 0: Reserved Two b y t e s a t t h e end of t h e frame a r e r e s e r v e d f o r t h e s h o r t a d d r e s s of t h e s e n d e r. This i s a workaround u n t i l r e c v f r o m ( ) g e t s f i x e d. / # d e f i n e SEGMENT FIRST 0 x20 # d e f i n e SEGMENT LAST 0 x10 / / # d e f i n e SEGMENT BOTH 0 x30 # d e f i n e SEGMENT MIDDLE 0 x00 # d e f i n e SEQ NUM MASK 0 x07 namespace d t n namespace n e t lowpanstream : : lowpanstream ( l o w p a n s t r e a m c a l l b a c k &c a l l b a c k, u n s i g n e d i n t a d d r e s s ) : s t d : : i o s t r e a m ( t h i s ), i n f i r s t s e g m e n t ( t r u e ), o u t s t a t ( SEGMENT FIRST ), i n b u f ( new c h a r [ BUFF SIZE ] ), o u t b u f ( new c h a r [ BUFF SIZE + 2 ] ), o u t 2 b u f ( new c h a r [ BUFF SIZE ] ), a d d r e s s ( a d d r e s s ), c a l l b a c k ( c a l l b a c k ), i n s e q n u m ( 0 ), o u t s e q n u m ( 0 ), i n b u f f r e e ( t r u e ), o u t s e q n u m g l o b a l ( 0 ), a b o r t ( f a l s e ) / / I n i t i a l i z e g e t p o i n t e r. This s h o u l d be z e r o so t h a t u n d e r f l o w i s c a l l e d upon f i r s t r e a d. s e t g ( 0, 0, 0) ; 94

109 B.2. IBR-DTN Convergence Layer Code s e t p ( o u t b u f + 1, o u t b u f + BUFF SIZE 1) ; lowpanstream : : lowpanstream ( ) d e l e t e [ ] i n b u f ; d e l e t e [ ] o u t b u f ; d e l e t e [ ] o u t 2 b u f ; void lowpanstream : : a b o r t ( ) ibrcommon : : MutexLock l ( i n b u f c o n d ) ; w h i l e (! i n b u f f r e e ) i n b u f c o n d. w a i t ( ) ; a b o r t = t r u e ; i n b u f c o n d. s i g n a l ( ) ; void lowpanstream : : queue ( c h a r buf, i n t l e n ) ibrcommon : : MutexLock l ( i n b u f c o n d ) ; IBRCOMMON LOGGER DEBUG( 1 0 ) << lowpanstream : : queue << IBRCOMMON LOGGER ENDL; / / R e t r i e v e s e q u e n c e number of frame u n s i g n e d i n t seq num = buf [ 0 ] & SEQ NUM MASK; IBRCOMMON LOGGER DEBUG( 4 5 ) << Received frame s e q u e n c e number : << seq num << IBRCOMMON LOGGER ENDL; / / Check i f t h e s e q u e n c e number i s what we e x p e c t i f ( i n s e q n u m!= seq num ) IBRCOMMON LOGGER( e r r o r ) << Received frame w ith o u t of bound s e q u e n c e number ( << seq num << e x p e c t e d << ( i n t ) i n s e q n u m << ) << IBRCOMMON LOGGER ENDL; a b o r t = t r u e ; i n b u f c o n d. s i g n a l ( ) ; r e t u r n ; / / i n c r e m e n t t h e s e q u e n c e number i n s e q n u m = ( i n s e q n u m + 1) % 8 ; / / check i f t h i s i s t h e r i g h t segment i f ( i n f i r s t s e g m e n t ) i f (! ( buf [ 0 ] & SEGMENT FIRST ) ) r e t u r n ; i f (! ( buf [ 0 ] & SEGMENT LAST) ) i n f i r s t s e g m e n t = f a l s e ; w h i l e (! i n b u f f r e e ) i n b u f c o n d. w a i t ( ) ; memcpy ( i n b u f, buf + 1, l e n 1) ; 95

110 B. Code i n b u f l e n = l e n 1 ; i n b u f f r e e = f a l s e ; i n b u f c o n d. s i g n a l ( ) ; / / c l o s e woth a b o r t f o r c o n d i t i o n a l i n t lowpanstream : : sync ( ) / / Here we know we g e t t h e l a s t segment. Mark i t so. o u t s t a t = SEGMENT LAST ; i n t r e t = s t d : : c h a r t r a i t s <char >:: e q i n t t y p e ( t h i s >o v e r f l o w ( s t d : : c h a r t r a i t s <char >:: e o f ( ) ), s t d : : c h a r t r a i t s <char >:: e o f ( ) )? 1 : 0 ; IBRCOMMON LOGGER DEBUG( 1 0 ) << lowpanstream : : sync << IBRCOMMON LOGGER ENDL; r e t u r n r e t ; i n t lowpanstream : : o v e r f l o w ( i n t c ) c h a r i b e g i n = o u t b u f ; c h a r i e n d = p p t r ( ) ; IBRCOMMON LOGGER DEBUG( 1 0 ) << lowpanstream : : o v e r f l o w << IBRCOMMON LOGGER ENDL; / / mark t h e b u f f e r as f r e e s e t p ( o u t b u f + 1, o u t b u f + BUFF SIZE 1) ; i f (! s t d : : c h a r t r a i t s <char >:: e q i n t t y p e ( c, s t d : : c h a r t r a i t s <char >:: e o f ( ) ) ) i e n d ++ = s t d : : c h a r t r a i t s <char >:: t o c h a r t y p e ( c ) ; / / b y t e s t o send s i z e t b y t e s = ( i e n d i b e g i n ) ; / / i f t h e r e i s n o t h i n g t o send, j u s t r e t u r n i f ( b y t e s == 0) IBRCOMMON LOGGER DEBUG( 1 0 ) << lowpanstream : : o v e r f l o w ( ) n o t h i n g t o s e n t << IBRCOMMON LOGGER ENDL; r e t u r n s t d : : c h a r t r a i t s <char >:: n o t e o f ( c ) ; / / FIXME : Should we w r i t e i n t h e segment p o s i t i o n h e r e? o u t b u f [ 0 ] = 0 x07 & o u t s e q n u m ; o u t b u f [ 0 ] = o u t s t a t ; o u t s e q n u m g l o b a l ++; o u t s e q n u m = ( o u t s e q n u m + 1) % 8 ; IBRCOMMON LOGGER DEBUG( 1 0 ) << lowpanstream send segment << ( i n t ) o u t s e q n u m << / << o u t s e q n u m g l o b a l << IBRCOMMON LOGGER ENDL; / / Send segment t o CL, use c a l l b a c k i n t e r f a c e c a l l b a c k. s e n d c b ( o u t b u f, b y t e s, a d d r e s s ) ; i f ( o u t s t a t & SEGMENT LAST) 96

111 B.2. IBR-DTN Convergence Layer Code / / r e s e t o u t g o i n g s t a t u s b y t e o u t s t a t = SEGMENT FIRST ; e l s e o u t s t a t = SEGMENT MIDDLE; r e t u r n s t d : : c h a r t r a i t s <char >:: n o t e o f ( c ) ; i n t lowpanstream : : u n d e r f l o w ( ) ibrcommon : : MutexLock l ( i n b u f c o n d ) ; IBRCOMMON LOGGER DEBUG( 1 0 ) << lowpanstream : : u n d e r f l o w << IBRCOMMON LOGGER ENDL; w h i l e ( i n b u f f r e e ) i f ( a b o r t ) throw ibrcommon : : E x c e p t i o n ( s t r e a m a b o r t e d ) ; i n b u f c o n d. w a i t ( ) ; memcpy ( o u t 2 b u f, i n b u f, i n b u f l e n ) ; i n b u f f r e e = t r u e ; IBRCOMMON LOGGER DEBUG( 1 0 ) << lowpanstream : : u n d e r f l o w i n b u f f r e e : << i n b u f f r e e << IBRCOMMON LOGGER ENDL; i n b u f c o n d. s i g n a l ( ) ; / / S i n c e t h e i n p u t b u f f e r c o n t e n t i s now v a l i d ( or i s new ) / / t h e g e t p o i n t e r s h o u l d be i n i t i a l i z e d ( or r e s e t ). s e t g ( o u t 2 b u f, o u t 2 b u f, o u t 2 b u f + i n b u f l e n ) ; r e t u r n s t d : : c h a r t r a i t s <char >:: n o t e o f ( o u t 2 b u f [ 0 ] ) ; # i f n d e f LOWPANSTREAM H # d e f i n e LOWPANSTREAM H Listing B.7: lowpanstream.h # i n c l u d e ibrcommon / t h r e a d / C o n d i t i o n a l. h # i n c l u d e <s t r e a m b u f > # i n c l u d e <s t d i n t. h> namespace d t n namespace n e t c l a s s l o w p a n s t r e a m c a l l b a c k p u b l i c : / C a l l b a c k i n t e r f a c e i m p l e m e n t a t i o n from LoWPAN LOWPANConvergenceLayer / v i r t u a l void s e n d c b ( c h a r buf, i n t len, u n s i g n e d i n t a d d r e s s ) = 0 ; ; c l a s s lowpanstream : p u b l i c s t d : : b a s i c s t r e a m b u f <char, s t d : : c h a r t r a i t s <char> >, 97

112 B. Code p u b l i c s t d : : i o s t r e a m p u b l i c : / S i z e of t h e i n p u t and o u t p u t b u f f e r s. / s t a t i c c o n s t s i z e t BUFF SIZE = 113; lowpanstream ( l o w p a n s t r e a m c a l l b a c k &c a l l b a c k, u n s i g n e d i n t a d d r e s s ) ; v i r t u a l lowpanstream ( ) ; / Queueing d a t a r e c e i v e d from t h e CL worker t h r e a d f o r t h e buf B u f f e r with r e c e i v e d d a t l e n Length of t h e b u f f e r / void queue ( c h a r buf, i n t l e n ) ; void a b o r t ( ) ; void c l o s e ( ) ; p r o t e c t e d : v i r t u a l i n t sync ( ) ; v i r t u a l i n t o v e r f l o w ( i n t = s t d : : c h a r t r a i t s <char >:: e o f ( ) ) ; v i r t u a l i n t u n d e r f l o w ( ) ; u n s i g n e d i n t a d d r e s s ; p r i v a t e : bool i n f i r s t s e g m e n t ; c h a r o u t s t a t ; / / I n p u t b u f f e r c h a r i n b u f ; i n t i n b u f l e n ; bool i n b u f f r e e ; / / Output b u f f e r c h a r o u t b u f ; c h a r o u t 2 b u f ; / / s e q u e n c e number u i n t 8 t i n s e q n u m ; u i n t 8 t o u t s e q n u m ; long o u t s e q n u m g l o b a l ; bool a b o r t ; l o w p a n s t r e a m c a l l b a c k &c a l l b a c k ; ibrcommon : : C o n d i t i o n a l i n b u f c o n d ; ; # e n d i f / LOWPANSTREAM H / 98

113 Acronyms 6LoWPAN ACK BP CBHE CCA CL CRC CSMA/CA CSV DMA DTN EID FCS FFD FIFO GPIO GPL I2C ION IP IPND ISM LR-WPAN MAC MTU IPv6 over Low power Wireless Personal Area Networks. Acknowledgement. Bundle Protocol. Compressed Bundle Header Encoding. Clear Channel Assessment. Convergence Layer. Cyclic redundancy check. Carrier Sense Multiple Access with Collision Avoidance. Comma Separated Value. Direct Memory Access. Delay Tolerant Networking. Endpoint Identifier. Frame Check Sequence. Full Function Device. First In First Out. General Purpose Input Output. General Public License. Inter-Integrated Circuit. Interplanetary Overlay Network. Internet Protocol. IP Neighbor Discovery. Industrial Scientific and Medical. Low-Rate Wireless Personal Area Networks. Media Access Control. Maximum Transmission Unit. 99

114 Acronyms PAN PAN ID PCB PIO RF RFCOMM RFD RTT SD SDIO SDNV SFD SoC SPI USB WSN Personal Area Network. Personal Area Network Identifier. Printed Circuit Board. Programmed Input/Output. Radio frequency. Radio Frequency Communication. Reduced Function Device. Round Trip Time. Secure Digital. Secure Digital Input Output. Self-Delimiting Numeric Values. Start Frame Delimiter. System on Chip. Serial Peripheral Interface. Universal Serial Bus. Wireless Sensor Network. 100