IEEE s Wireless Mesh Networks for Last-mile Internet Access: An Open-source Real-world Indoor Testbed Implementation

Transcription

1 IEEE 802.s Wireless Mesh Networks for Last-mile Internet Access: An Open-source Real-world Indoor Testbed Implementation ABSTRACT Due to their easy-to-deploy and self-healing features, WMNs (Wireless Mesh Networks) are emerging as a new promising technology with a rich set of applications. While the standardization of this new technology is still in progress, its main traits are already set in the IEEE 802.s standard: WMN architecture and MAC Routing. WMNs are attracting considerable research in Academia (and industry as well). However, the lack of open-source testbeds is restricting such a research by limiting it to simulation tools. This paper presents an open-source implementation of a real-world indoor IEEE 802.s WMN testbed: the implementation is transparent and easy-to-deploy as it uses off-the-shelf commodity hardware and software. The source code and deployment instructions are made available online. The current testbed source code can serve as a blueprint for the WMNs research community to deploy their own testbeds. By delving into the testbed implementation subtleties, the paper is also shading further light into the still-on-process IEEE 802.s standard. Major encountered implementation problems (e.g., Clients Association, Internetworking and addressing multiple gateways) are highlighted and addressed. To ascertain the functionality of the testbed, both UDP and TCP trafc are supported and operational. The testbed uses the default IEEE 802.s HWMP (Hybrid Wireless Mesh Protocol) routing protocol along with the default IEEE 802.s Airtime routing metric.. INTRODUCTION Due to their easy-to-deploy and self-healing features, WMNs [9] are emerging as a promising technology. WMNs are easy-to-deploy as setting a WMN involves minimal wiring and conguration overhead: Settling down the WMN nodes and powering On is all what is required to operate a WMN. On the other hand, WMNs are selfhealing thanks to the redundancy of wireless links whereby a failure Technical Report# CSSE0-02 Riduan M Abid, Taha Benbrahim, Saâd Biaz Computer Science and Software Engineering Department Auburn University Shelby Center for Engineering Technology, Suite 30 Auburn University, AL, , USA {abidmoh, benbrah, biazsaa}@auburn.edu of a wireless link incites the network to choose an alternative operational link, thus healing the link failure and continuously maintaining the network operational. WMNs may serve a rich set of applications, e.g., wireless community networks, wireless enterprise networks, transportation systems, home networking and last-mile wireless Internet access. Providing last-mile wireless Internet access is one of the most promising application as WMNs tremendously reduce the cost and conguration overhead when compared with current solutions. e.g. Wi-Fi (802.) WLANs. The proliferation of Wi-Fi LANs played a tremendous role in the promising success of WMNs. Even though very successful, 802. WLANs still suffer from the cost and overhead of setting the wired backbone that connects the different 802. APs (Access Points). In 802. WLANs, covering a cell requires the deployment of an AP which must be wired to the Internet through a backbone network, e.g., a LAN. When another adjacent cell is to be connected, another AP has to be settled and wired to the Internet.This way, the settlement of further APs becomes more and more costly when the cells to cover are quite far from the wired backbone network. Furthermore, in case of dense regions, a parallel dense deployment of APs is needed to meet customers needs, thus resulting in more wiring and settlement overhead. Even though APs are relatively cheap, wiring the APs and nding the appropriate canals for the wires remains expensive. With WMNs, no wiring is required except for the AP which will serve as a gateway towards the Internet (Even the gateway can be wirelessly connected to the Internet). All other APs serve as routers and route data on behalf of each other towards/from the gateway AP connected to the wired network. Therefore, covering a cell reduces to adding an AP without wiring it to the backbone network. Easing wireless coverage facilitates increasing wireless coverage: A fact which is of tremendous importance to industry since more coverage involves more user connections and hence more return to industry. In this context, IETF (Internet Engineering Task Force) is actively working to nalize the IEEE 802.s standard which will pave the way towards a successful worldwide industrialization of this promising technology. In 2005, IETF set a mesh networking TG to standardize IEEE 802.s. The work is still in progress and the last IEEE 802.s TG meeting was held in January, 200 [24]. Even though the standard is not nal yet, its main traits have are set, e.g. IEEE 802.s Architecture and Routing. The IEEE 802.s TG did set HWMP (Hybrid Wireless Mesh Protocol) [] as a default routing proto-

2 col for 802.s compliant devices. HWMP uses MAC addresses for routing. Besides, Airtime [0] has been set as a default routing metric for HWMP. These two amendments aim to maintain a minimum compatibility between the IEEE 802.s devices to be manufactured by different companies. Albeit a new technology, several WMNs solutions are already commercialized (e.g., by Strix Systems, Cisco, Nortel, Tropos, BelAir). Besides, valuable research real-world deployments are in place too, e.g., The MIT Roofnet [2] and the Microsoft Mesh Networking project [22]. However, these deployments are not compliant to the new IEEE 802.s standard and are proprietary. WMNs are attracting considerable research efforts in academia and industry. However, the absence of real-world open-source testbeds is restricting the proliferation of such a research by forcing researchers to use simulation tools, e.g., ns-2 [6] and Qualnet [8]. Eventhough simulators are relatively good at approximating realworld scenarios, they are still quite far from a good approximation of wireless indoor environments, which is the case with IEEE 802.s technology: In wireless indoor environments, RF propagation is very crucial in determining the interferences level, this later has been proved to be of great impact on multi-hop wireless mesh networks [3,5]. Indeed, simulators can not catch the complex aspects of RF propagation such as multi-path fading, modeling mobile obstacles, modeling external interferences, etc. In fact, when dealing with RF propagation, most simulation tools remain simplistic: for instance, ns-2, which is the most widely used tool in academia, uses a simplied version of a physical model (the capture threshold model) that accounts only for one interferer at a time [4], which is not the case in real-world interferences where interference can be caused by different interferers. However, simulators are still very common in research as they remain the only research experimentation alternative when the deployment costs of real-world testbed are not affordable. Besides, and depending on the kind of application, simulators can be a good alternative as they are close to reality, e.g., in simulating TCP, IP, 802.3, etc. However, when it comes to wireless technology and especially when it is in indoor environments, simulators are just far from accurately presenting the real world. In this context, we are presenting an open-source real-world IEEE 802.s WMN testbed implementation that can be easily deployed as it uses off-the-shelf commodity hardware and software. The linux-based testbed code is open-source and written in C language. The code is made available online at abidmoh/ [7]. Thus, the academia WMN research community can easily migrate to the use of real-world testbeds, instead of simulators, by deploying the proposed testbed or by using it as a blueprint for deploying their own testbed. Besides serving the purpose of allowing the WMN researches to easily build their own WMN testbed, the testbed can serve as a building block for a future widely-used open-source WMN. By delving into the implementation subtleties of the testbed, the paper shades further light into the new IEEE 802.s standard. Major encountered implementation issues such as clients association, Internetworking and multiple gateways are highlighted and addressed. Both TCP and UDP trafc are supported in order to ascertain the functionality of the testbed: Real-world web browsing sessions from Wi-Fi enabled stations, using the testbed as a wireless backhaul, are operational. The current testbed uses 5 nodes and spans two separate locations that are connected through the Internet.The testbed uses HWMP as a routing protocol along with the IEEE 802.s default Airtime routing metric. However, It is very important to mention that this work is not meant to evaluate the performance of the new IEEE 802.s standard but to present an easy-to-deploy and open-source real-world in-door IEEE 802.s WMN testbed that can serve as a seed for other WMN testbeds, hence enriching the WMN research in academia. The primary contributions of this work are: Present an easy-to-deploy and open-source real-world IEEE 802.s WMN testbed implementation. Encourage Academic researchers on IEEE 802.s WMNs to use real-world testbeds. Present a building block for a future academic widely-used open-source WMN testbed. Highlight and solving major implementation issues, e.g.,: Clients association problem Internetworking WMNs Supporting multiple gateways Highlight the main traits of the new IEEE 802.s standard. Highlight and implement the new IEEE 802.s HWMP routing protocol and the IEEE 802.s Airtime routing metric. The rest of the paper is organized as follows: Section 2 overviews the main specications of the IEEE 802.standard. Section 3 highlights the details of the HWMP routing protocol and the Airtime routing metric. The implementation details and the encountered problems are respectively covered in Sections 4 and 5. In Section 6, we outline the necessary steps to deploy the testbed. In Section 7, we present the experimental settings of the testbed, the topologies and the experimental results. Finally, Section 8 concludes and presents future work. 2. IEEE 802.S OVERVIEW The IEEE 802.s standard started initially as a study group in 2003, then it became a Task Group in July, The rst draft was accepted in March, 2006 and the last TG meeting was held in January, 200 [24]. The standard is concerned with ve main areas:. Architecture, 2. Routing in MAC layer, 3. MAC enhancements, 4. Internetworking and 5. Security. This work pertains to areas, 2 and 4. Areas and 2 are introduced in next paragraphs, while Area 4 is deferred to Section s Architecture IEEE 802.s denes three types of stations:. MPs (Mesh Points): MPs are wireless stations that perform only routing. 2. MAPs (Mesh Access Points): MAPs are MPs with additional access point capabilities. Besides performing routing, MAPs aggregate trafc from/towards legacy 802. stations. A MAP can be thought of as a legacy access point which performs also routing. 3. MPPs (Mesh Portal Points): MPPs are MPs that serve as gateways to other types of networks such as networks. MPPs aggregate trafc from/towards non-mesh networks, e.g., Internet.

3 IEEE 80.2.s set HWMP [] as a mandatory routing protocol for all 802.s compliant devices. Furthermore, Airtime [0] was set as a mandatory routing metric too. This is in order to ensure compatibility between different 802.s devices manufacturers. HWMP and Airtime are detailed in next sections. 3. MAC ROUTING IN 802.S HWMP is a hybrid protocol as it is reactive and a proactive. The two behaviors can be used separately or at the same time depending on the application. Figure : WMN Architecture To illustrate the functionality of every mesh node type, Figure depicts a scenario where an end user, e.g., a Wi-Fi enabled station, is browsing the Internet. The end-user, at station sta, is connected to a legacy 802. network through a mesh access point (MAP). The MAP is forwarding/routing user data towards/from the WMN and has mesh point (MP2) as a next hop. Mesh point (MP2) has mesh point (MP4) as its next hop and MP7 afterwards. This later forwards data towards the Mesh Portal Point (MPP) which serves as a gateway towards the Internet where the web server is. The role of the routing protocol is to determine the best sequence of next hops for a data frame to get to its nal destination. In 802.s, routing is performed using MAC addresses and uses Airtime [0] as a routing metric. 2.2 Routing at MAC Layer Multi-hop routing is a key element in 802.s. 802.s performs routing using MAC addresses and uses either four or six MAC addresses depending on the nature of the trafc. When both the source and destination are mesh points in the WMN, only four addresses are used. When at least the source or the destination is outside the WMN, i.e., a non-mesh point, e.g., a 802.s legacy station, six addresses are used. 802.s frame header bear an AE (Address Extension) mode bit that discriminates between the two modes. In the case where WMNs are used for last-mile Internet access, the AE bit is set to as both the destination and the source are non-mesh points. Adding two extra MAC addresses, to the ordinary four 802. addresses, is meant to track the MAC addresses of the non-mesh source and non-mesh destination as they would be lost once the frame enters/quits the mesh network. The six MAC addresses are:. Destination MAC Address: It corresponds to the next hop MAC address. 2. Source MAC Address: MAC address of the source. 3. Destination Proxy Address: The MAC address of the MPP/MAP from where the frame leaves the mesh network. 4. Source Proxy Address: The MAC address of the MPP/MAP from where the frame enters to the mesh network. 5. Final Destination Address: The MAC address of the nal non-mesh destination. 6. Originator Address: The MAC address of the non-mesh station that originated the frame. 3. Reactive Routing in HWMP RM-AODV (Radio-Metric Ad hoc On Demand Distance Vector) [4] is the reactive protocol in HWMP. RM-AODV is an adaptation of the AODV [9] protocol that uses Airtime [0] as link quality metric. Reactive routing protocols initiate route discovery requests only when needed, e.g., route failure. In RM-AODV, when a node needs a path to a certain destination, it broadcasts a PREQ (Path Request) message that contains the MAC address of the destination. Every PREQ message bears a sequence number that uniquely distinguishes it from other PREQ messages. When an intermediate MP receives a PREQ message, it rst checks the freshness of the received message by comparing its sequence number with the last locally stored sequence number: A PREQ message is processed only when it has a greater sequence number or when it has the same sequence number but exhibiting a better route. The sequence numbers are used to keep the network loop-free. After receiving a fresher PREQ message, intermediate nodes update their reverse path towards the source, update the routing metric by including the weight of the last hop then re-broadcast the updated PREQ message. Subsequent MPs proceed the same way until the PREQ reaches the destination. When the destination MP receives the PREQ message, it checks its freshness the same way as other intermediate MPs did, then it updates its reverse path towards the source. The destination then formulates a RREP (Route Reply) message which is afterwards uni-casted towards the source. Intermediate MPs receiving the RREP message update their forward path towards the destination, update the routing metric then forward the updated RREP towards the source. In case of node failure, the intermediate MPs detecting the failure send a RERR (Route Error) message towards the source to inform it about the breakage of the link. The source can then re-initiate a new route discovery using a new PREQ message. 3.2 Proactive Routing in HWMP HMWP has two proactive modes []: Proactive PREQ and Proactive RANN. The two protocols are variations of tree based routing [20,2] 3.2. Proactive PREQ Mode In the proactive PREQ mode, every root MP periodically broadcasts PREQ messages bearing unique sequence numbers. The protocol is similar to reactive HWMP, presented in Section 3., in how intermediate nodes react and process PREQ messages. The only difference is that in reactive HWMP, PERQ messages are sent in a reactive way, i.e. when needed, whereas in proactive PREQ, PREQ messages are sent proactively, i.e., in advance and in a periodic basis.

4 3.2.2 Proactive RANN Mode Like in proactive PREQ mode, MPPs periodically broadcast RANN (Route Announcement) messages with unique sequence numbers. However, RANN messages differ from PREQ messages in that they are not meant for requesting paths, they are instead meant only to communicate routing metrics in the network. Upon the reception of a RANN, the intermediate MPs re-broadcast the RANN message only if it corresponds to a newer RANN message or to a RANN message with same sequence number but with better routing metric. Furthermore, if an intermediate MP nds the new RANN advising a better route to the root, it then sends a unicast PREQ message which sets the forward path to the root MP. Upon the reception of the PREQ message, the root MP replies with a unicast PREP (Path Reply) which sets the reverse path to the root MP Which protocol to use in case of WMNs for last-mile Internet access? As illustrated in previous sections, HWMP advises three protocols in total. The question is which protocol to use and can they be used together? In fact, the use of these different protocols is for the purpose of coping with the wide range of applications that WMNs may support. Hence, the protocol to use depends on the kind of application. For instance, in case of wireless enterprise networks, where enterprise stations are mostly communicating between each other, the reactive HWMP is the appropriate protocol to use, and this is due basically to the random pattern of trafc in the network which involves high dynamism in the network in terms of link quality. Furthermore, if the enterprise network is to be connected to the Internet, which is usually the case, then the reactive AODV must be combined with proactive PREQ or proactive RANN, since a good part of the trafc will be always directed towards/from the gateways connecting the enterprise to Internet. If the WMN is solely used for last-mile Internet access, the very specicity of the trafc which is always directed from/towards the MPPs and which shapes a tree topology advocates the use of a treebased routing protocol where tree roots, i.e., Internet gateways, would disseminate broadcast messages for root selection. Both proactive PREQ and proactive RANN are tree-based routing variations. The remaining point is which protocol to use? i.e., proactive PREQ or proactive RANN? The proactive RANN and proactive PREQ protocols share the same principle of periodical broadcast of root advertisement messages. However, and to our understanding, the proactive PREQ can be viewed as a lesser overhead version of the proactive RANN since this later broadcasts RANN messages in addition to PREQ and PREP messages. Proactive RANN would be more appropriate for networks exhibiting a dense trafc and/or more dynamism in trafc pattern. We would advise, for instance, the usage of proactive RANN instead of proactive PREQ in the case where the WMN is witnessing a dense trafc or when the WMN has many gateways to Internet. In case of a unique gateway or even a couple of gateways, proactive PREQ is quite sufcient and exhibits less overhead in terms of minimal control frames. To conclude, there is a trade-off deal between the adoption of proactive PREQ or proactive RANN. The rst one exhibits less overhead while the other one has more overhead but allows more adaptation to network dynamism: The kind of the application is to decide s Radio-Aware Airtime Metric 802.s set Airtime as a default routing metric for all 802.s compliant devices. Airtime metric is a radio-aware metric which is meant to measure the amount of consumed channel resources when transmitting a frame over a particular wireless link [0]. Airtime is computed as follows: Airtime =O ca + O p + B t r e fr () where O ca, O p and B t are constants quantifying respectively the Channel Access Overhead, the Protocol Overhead and the number of Bits in a probe frame. r is the transmission rate (in Mbps) for a frame of size B t and frame error rate e fr. IEEE 802.s did not set a specic way to measure e fr, it is left as an implementation issue [8]. Unlike ETX (Expected Transmission Count) [2] which accounts solely for frame error rate, Airtime accounts for both frame error rate and link bandwidth as well, which is the case with ETT (Expected Transmission Time) [22] as well. However, Airtime does further account for channel and protocol overhead. The next sections highlight ETX and ETT main traits ETX (Expected Transmission Count) De Couto et. al proposed ETX [2]. ETX accounts solely for frame loss rate and does so by measuring the expected number of transmissions needed to successfully transmit a frame. To compute ETX, every node periodically broadcasts a constant number of probe frames N over a certain constant time period T. The receiver counts the number of received frames and computes the forward delivery ratio as follows: d f = R f N where R f is the number of received frames in the forward direction. When the receiver broadcasts its N probe frames over the same period T, it does piggyback in its probe frames the computed d f values for all its neighbors. Upon receiving a probe frame, every node counts the received probe frames in order to compute the reverse delivery ratio d r - in a similar way to computing d f, equation (2) - and does at the same time extract the corresponding embedded d f value that was computed at the sender. This way every node has both the forward and reverse delivery ratios. These later are used to compute ETX as follows: ET X = (3) d f d r However, ETX suffers from the major shortcoming of not accounting for link bandwidth. To cope with this problem ETT has been proposed ETT (Expected Transmission Time) R. Draves et al. proposed ETT [22]. ETT is meant to cope with the major shortcoming of ETX in not accounting for link bandwidth. In [22] authors are aliasing ETT as Bandwidth-adjusted ETX. ETT accounts for both frame loss rate and bandwidth as well. ETT measures the needed time for a frame to be successfully transmitted. ETT is computed as follows: ET T = ET X t (4) where t is the average time for a single frame to be transmitted regardless of the transmission being it successful or not. (2)

5 t = S B where S is the size of the probe frame and B is the link bandwidth. To compute t, R. Draves et al. are using the wellknown technique of packet-pairs [6]. 4. IMPLEMENTATION 4. System Design As mentioned earlier, IEEE 802.s performs routing at the MAC layer. We tried an implementation that can be deployed using offthe-shelf 802. commodity hardware, i.e., it does not require any change to the 802. driver. In fact, an optimal implementation would require a thorough change of the MAC driver as 802.s uses six MAC addresses instead of the four ones which are used in Furthermore, the code should be implemented at the Kernel level in order to minimize the context switchings involved with user processes, especially when accessing network data. Still, for our implementation to approximate as much as possible an optimal MAC protocol implementation, we deployed two small kernel modules that drop all trafc going to/from the TCP-IP stack. These kernel modules are placed at the PRE-ROUTING and LOCAL- OUT hooks of Netlter [5], see Figure 3. Besides, by using Datalink Raw Sockets, the user-space generated 802.s frames bypasses the TCP-IP stack. This way, the TCP-IP stack becomes transparent to our user-space implementation, hence ideally approximating a kernel implementation. The system is made of three modules, see Figure 2, that communicate between each other using IPC (Inter Process Communication) shared memories:. Routing, 2. Data forwarding and 3. Link quality measurements. Only the last module (link quality measurement) is identical in all WMN nodes, i.e., MAPs, MPs and MPPs. For the two other modules, their implementation depends on the type of the node. Next sections detail implementations for each module and highlights how the implementations differ according to the type of the mesh point. 4.2 Data Forwarding Module The Data Forwarding module implementation depends on the type of the mesh node Data Forwarding in MPPs MPPs have two network interfaces: an interface and an 802. ad-hoc interface, a fact which results in processing two different types of frames. Upon reception of an frame, the MPP-Data-Forwarding module extracts the destination IP address from the frame then fetches the corresponding MAC address and the associated MAP MAC address from the MPP Association Table (Association tables will be further highlighted in in Section 5 when addressing implementation problems). Once the destination MAP MAC address is retrieved, the corresponding next hop MAC address is retrieved too from the MPP proxying table and the appropriate 802.s header frame is built. This later contains basically the six MAC addresses that will be used to get the frame, through the 802.s adhoc network, to its nal destination. The six MAC addresses are built as follows:. Destination MAC address: corresponds to the next-hop MAC address retrieved from the MPP proxying table 2. Source MAC address: The MAC address of the sender, i.e., the MPP MAC address in this case 3. Destination Proxy MAC address: The MAC address of the corresponding MAP. 4. Source Proxy MAC address: The MAC address of the MPP 5. Final Destination MAC address: The destination MAC address which is retrieved from the local association table. 6. Originator MAC address: The source MAC address in the received frame The MPP then strips off the frame header and replaces it with the 802.s frame header while maintaining intact the payload of the frame. The resulting 802.s frame is then forwarded using the MPP 802. interface. Upon reception of an 802.s frame in its ad-hoc interface, the MPP reads the 802.s header frame, gets the originator MAC address and its IP address and creates an entry for it in the MPP association table. The entry contains the originator IP address, the originator MAC address and the associated MAP MAC address. Afterwards, the MPP strips off the 802.s frame header and replaces it with a frame header containing as a source address the local MPP MAC address Data Forwarding in MPs In contrast to MPPs, Data Forwarding in MPs is quite straightforward as it deals only with one type of frames, i.e., 802.s frames. However, the MP still receives two types of frames that differ in terms of their nal destination which can be either in an network (e.g., frame destined to Internet) or an 802. network (frame destined to a Wi-Fi station). In other words, the two frames can differ in terms of whether their nal destination, within the WMN, is an MPP or a MAP? The proxy type, i.e., a MPP or an MPP, is made transparent in our implementation by having the same type of entry in the MP forwarding table for all proxies. Upon reception of an 802.s frame, the MP extracts the destination proxy address, i.e., the MPP or MAP MAC address, consults its forwarding table and retrieves the corresponding entry which contains the next hop MAC address. The 802.s frame is then updated by changing only two of the six MAC addresses in the 802.s frame header: The remaining four ones are kept intact. The two updated MAC addresses are the destination MAC address and the source MAC address which become respectively the retrieved next hop MAC address and the MP MAC address Data Forwarding in MAPs MAPs have two network interfaces, a managed mode (Access point) 802. interface and an ad-hoc mode 802. interface, which results in processing two different types of frames. Upon reception on an 802. frame, from its Access Point interface, the MAP rst reads the source MAC address, and formulates the six MAC addresses in the 802.s header frame as follows:. Destination MAC address: It corresponds to the next hop MAC address, towards the best MPP, which is retrieved from the MAP Proxying-Table. 2. Source MAC address: the MAC address of the local MAP.

6 Figure 2: System Design 3. Destination Proxy Address: the MAC address of the MPP to which to forward the frame. It is retrieved from the MAP proxying table 4. Source Proxy Address: the MAC address of Local MAP 5. Final Destination: Unknown at this stage. A NULL address is put in. 6. Originator: the source MAC address in the received 802. frame. The MAP then strips off the 802. header, replaces it with the just shaped 802.s header and forwards the frame through its ad-hoc 802. interface. On the other side, upon reception of a 802.s frame in its ad-hoc interface, the 802.s header frame is stripped off and replaced by a broadcast 802. header frame. Thereafter, the nal 802. legacy destination will look up its IP address in the broadcasted 802. frame and thus process the frame. 4.3 Routing Module The routing module implementation depends on the type of the mesh node Routing in MPPs In MPPs, the routing module is made of two sub-modules:. MPP-PREQ-Broadcast Module: periodically broadcasts PREQ messages which contain basically the MAC address of the local MPP, an incremental sequence number and a routing metric eld. 2. The MPP-RREP-Processing Module: processes RREP messages forwarded by intermediate MPs and originated from different MAPs. Upon reception of a RREP message, the MPP gets the MAC address of the sender MAP, looks up its entry it the MPP proxying table and updates the corresponding routing metric and the corresponding next hop. The RREP messages do set the forward path towards MAPs Routing in MPs In MPs, the MP-PREQ-Processing module processes two types of messages: PREQ and RREP messages. Figure 3: 802.s MAC Routing and the OSI Model Upon reception of a PREQ message, the module checks rst the freshness of the message in order to process it as only fresher messages must be considered. The module extracts rst the MAC address of the sender MPP, from the PREQ message, then uses this address to look up its corresponding entry in the MP forwarding table. The MP forwarding table entries store, among other values, the last fresh sequence numbers of the concerned MPPs. Once the last fresh sequence number is retrieved, the PREQ message is discarded in case it corresponds to a non-fresh message (i.e., of a smaller sequence number). If the PREQ message is a fresher one then the routing module updates its reverse path entry towards the concerned MPP. Afterwards, the PREQ message is updated by adjusting the routing metric value: The module retrieves the corresponding routing metric value, e.g. ETX or Airtime, for the link from where it received the PREQ message and adds it to the current PREQ routing metric then rebroadcast the message. If the PREQ message is of equal freshness as the current sequence number, the PREQ message is then processed only and only if it corresponds to a better route. If it is the case, the PREQ is processed as if it was a fresh PREQ message. Upon reception of a RREP message, the MP gets the MAC address of the sending MAP, looks up its entry in the forwarding table, and then updates the corresponding forward path to the MAP. To forward the RREP message, the MP looks up the destination MPP MAC address and retrieves its entry from the forwarding table. The retrieved entry contains the MAC address of the next hop in the route towards the concerned MPP. Afterwards, the routing metric in the RREP message is updated and then forwarded to the next hop Routing in MAPs Upon reception of a PREQ message, the MAP-PREQ-Processing module retrieves the MAC address of the MPP which issued the PREQ message, retrieves the entry, from the proxying table, that corresponds to the sender MPP and then checks the freshness of the current PREQ message against the already stored freshness. If the PREQ message corresponds to a fresher route or corresponds to an equal freshness route with better routing metric, the corresponding entry in the MAP proxying table is updated and the routing metric value is updated too the same way as with MPs. However, the processed PREQ message can correspond to only a local better route, i.e., the best route leading to a specic MPP.

7 802.a 802.b/g O ca 75 µs 335 µs O p 0 µs 364 µs Table : Airtime metric constants Hence, there may be better routes to other MPPs. Selecting the MPP that has the best routing metric would fail in case the entry was updated a long time ago or in case the MPP is off (This problem will be further highlighted in Section 5). To address this problem, we timestamp all MPP routing entries in the proxying table. The timestamps correspond to the time at which the MPP routing entry was last updated. This way we select the MPP that has the better route as well as a fresher route by comparing its timestamp to the current time. We set a time threshold which is twice the MPP broadcast periods. We choose a twice period to account for the case when a PREQ broadcast has been accidentally lost. The best MPP is marked in the proxying table. Once the proxying table is updated, a RREP message is formed and sent back to the selected MPP. The RREP message acknowledges the receipt of the MPP PREQ message and it plays also the role of an association message telling the MPP to associate the MAP with it. The RREP message bears mainly the MAC address of the MAP as well as the corresponding routing metric. 4.4 Airtime Measurement Module As presented in Section 3.3, Airtime is computed as follows: Airtime =O ca + O p + Bt (5) r e fr Based on Equation 5, airtime depends on two basic variables: The frame error rate e fr and the bit rate r. The other constants are given in Table. To compute the frame error rate, we used the techniques highlighted in ETX [2] in order to compute the reverse and the forward delivery ratios. The ETX module implementation is highlighted in next section. Once the forward and reverse delivery ratios d f and d r are available, the frame error rate is approximated as follows: e fr = ( d f ) ( d d ) To compute the bit rates we used the "/Proc" system les: The rate adaptation algorithm is SampleRate [3] ETX Module To compute ETX, we need both the forward and the reverse delivery ratios. Our implementation of the ETX module consists on two sub-modules: The ETX-Broadcast module and the ETX- Collect module. The ETX-Broadcast module periodically broadcasts probe frames embedding the number of probe frames received in the reverse path from neighboring MPs during a certain time period T. The ETX-Broadcast gets these values from the ETX-Collect module. The ETX-Collect module is the module that computes the ETX values towards every neighbor. To get the reverse delivery ratios, the module counts the probe frames broadcasted by every neighboring MP during a certain period of time T. Once computed, the delivery ratios are sent to the ETX- Broadcast module which embeds them in the broadcasted probe frames. While reading the received probe frames, every ETX-module extracts his corresponding forward delivery ratios that were embedded by the ETX-Broadcast modules of the neighboring MPs. 5. IMPLEMENTATION PROBLEMS During implementation, we faced three basic problems:. clients association, 2. internetworking and 3. addressing multiple gateways. The next sections highlight these problems and suggest solutions. 5. The Clients Association Problem The clients association problem arises when an 802.s frame leaves the WMN, i.e., passes through the MPP gateway. In such a scenario, the MAC address of the source 802. legacy station as well as the MAC address of the MAP who originated the frame are lost forever since the IEEE 802.s frame headers would be stripped off at the level of the gateway. The problem is that when the MPP will receive back a frame as a response to the already sent frame, the MPP cannot remember which MAP sent the frame. As a consequence, the MPP cannot forward the frame to the right destination as it does not know the MAC address of the corresponding MAP. To solve this problem, we propose two alternatives:. Forcing every MAP to periodically broadcast the IP and the MAC addresses of its associated legacy 802. stations, 2. Having the MPP extract the needed information from forwarded frames and store them in a table. Both alternatives would meet the goal of having the MPP remember the MAP from which it received the concerned frame. However, we opted for the second alternative as the rst one would induce more overhead in the network because of the broadcast nature of the approach. Besides, the rst approach becomes delicate in the case where the legacy 802. stations are mobile. In this later case, a MAP may broadcast that a station is associated with it while the station did already moved to another MAP coverage area. In other words, the rst approach does not handle hand-offs. In the adopted approach, the MPP extracts the following data: Source IP address, source MAC address and the MAC address of the associated MAP. The extracted data is stored in a table. To cope with the look up time of the table which is of 0(n), we implemented the association table as a hash table with a chained list as a collision resolution strategy. We simplied the hash function to allow for further alleviation in terms of computational speed: Hash(IP addr ) = IP addr [3] % 256 This way, when the MPP receives an incoming packet, the corresponding MAC addresses of both the 802. legacy station and the associating MAP can be easily retrieved, from the association table, to be used in formulating the appropriate 802.s MAC header. This later will be used to forward the received packet till the intended MAP. The MAP, in its turn, will deliver the packet to the right destination using the MAC address of the 802. legacy station which was retrieved from the association table too. 5.2 Interconnecting WMNs The previously addressed problem, i.e., clients association, is meant basically to store/retrieve the MAP where a legacy 802. station resides. When trying to Interconnect WMNs, another problem arises and it is due this time to the irrelevance of the IP addresses of the legacy 802. stations outside the MWN, i.e., in external networks. This irrelevance stems from the fact that the

8 802. legacy stations would have IP addresses that are unknown to external networks as they are not directly connected to these networks, and hence not contributing in the external networks routing protocols. To solve this problem we implemented a NAT-like (Network Address Translation) mechanism at the level of the WMN Gateways (i.e., MPPs). This later have two interfaces, a WMN interface and an interface. Accordingly, when a frame is about to leave the WMN towards another network, e.g., Internet, the MPP gateway reads the IP address of the originator, i.e., the 802. legacy station, and replaces it by its own IP address. The packet is then sent. However, since there will be other frames originating from different WMN clients, replacing only the IP address is not sufcient as there should be a way for a unique mapping back to the original IP address. To solve this problem, we implemented another hash table, that has as a hash key the combination between the source port, the destination port and the destination IP address. This way, when a connection is created, e.g., TCP or UDP, an entry in the hash table is created constituting of the following quadruplet (Destination IP, Original IP, Destination Port, Source Port). When a response packet is received back, the gateway maps to the right entry in the hash table using the destination IP, the source port and the destination port. This way the original IP is retrieved. The original IP is then used to map into the association table in order to get its corresponding MAC address as well as the MAC address of the MAP where the original legacy station resides. Even if the presented NATing framework seems logic, this approach did not work at rst: We realized that the gateway (MPP), when receiving a response packet, especially in TCP, it does reply to the packet and establishes a TCP connection for itself as well. Hence resulting in sending duplicate packets (one from the MPP and one from the legacy station) as a response to a single TCP connection establishment. The problem was easily solved when we deployed the Netlter [5] modules, presented in Section 4., as they drop all IP packets destined to the gateway. However, the received packets, can still be forwarded to the user since the framework uses Raw Datalink sockets. These later read frames at the MAC layer, i.e., before the Netlter module drops the packets. 5.3 Addressing Multiple Gateways (MPPs) When processing HWMP PREQ (Path REQuest) messages, a MAP has to process only fresher PREQ messages, i.e., the ones that bear a larger or equal sequence number than the last fresher route. However, when a fresher PREQ message is processed and the corresponding routing entry is updated, a problem arises when the wireless mesh network has multiple MPPs as there will be different sets of PREQ sequence numbers. Each set pertains to a single MPP: a PREQ message is uniquely broadcasted by one MPP. Accordingly, when selecting a fresh sequence number, this later would correspond only to a better route toward a specic MPP, i.e., the MPP which originated the concerned PREQ message. The situation becomes quite critical because of the likeliness of having alternate routes to other MPPs with a better routing metric. Besides, since PREQ messages bear sequence numbers that cannot be synchronized as they are sent from different MPPs, comparing the freshness of sequence numbers issued from different MPPs is nonsense. Furthermore, if we just select the MPP that has the best route, this may not work since the selected route may shortly become an old route and hence invalid. To cope with this problem, we timestamp every routing entry with the time when the entry was created. This way, we select the MPP that has the best route as well as a still-valid route. To determine the validity of a route we heuristically set a threshold which depends on the MPP broadcast period. The experimental value of the threshold was presented in Section TESTBED DEPLOYMENT INSTRUCTIONS All the les referred to in this section are available online [7] and use the same names. 6. Deploying the Netlter Modules Two Netlter kernel modules must placed at the PRE-ROUTING and LOCAL-OUT hooks of Netlter as explained in Section 4.. To hook the two modules, do "make" and "insmod" the following two kernel les: "premodule.c" and "localoutmodule.c". 6.2 Deploying Mesh Access Points (MAPs) Deploying the Data Forwarding module: Compile and run the "MAP_ Data_Forwarding.c" le. Deploying the Routing module: The routing module depends on the underlying "Link Quality Measurement" module as the two modules do exchange link quality values through the IPC (Inter process communication) mechanism. Hence, depending on whether the WMN is using ETX or Airtime as a routing metric, you need to compile and run one of the following les:"map_preq _Processing_ETX.c" or "MAP_PREQ_ Processing_Airtime.c". Deploying the Link Quality Measurement module: The deployment of this module is the same for all of the three types of mesh node, i.e., MAPs, MPPs and MPs. Hence, it will be separately covered in section Deploying Mesh Portal Points (MPPs) Deploying the Data Forwarding module: Compile and run the "MPP_ Data_Forwarding.c" le. Deploying the Routing module: Compile and run the following two les: "MPP_PREQ_Broadcast.c" and "MPP_RREP _Processing.c". Deploying the Link Quality Measurement module: See Section Deploying Mesh Points (MPs) Deploying the Data Forwarding module: Compile and run the "MP_ Data_Forwarding.c" le. Deploying the Routing module: As with MAPs, you need to compile one of the following two les depending on whether the WMN is using ETX or Airtime as a routing metric: "MP_ PREQ_Processing_ETX.c" or "MP_PREQ_Processing_Airtime.c". Deploying the Link Quality Measurement module: See Section Link Quality Measurement This module is the same for all WMN nodes and it must be deployed in all WMN nodes. The deployment of this module depends on whether the WMN is using ETX or Airtime as a routing metric: Deploying ETX: Compile and run the following two les: "ETX_ Broadcast.c" and "ETX_Collect.c".

9 Figure 4: WMN Testbed, Topology Figure 5: WMN Testbed, Topology 2 Deploying Airtime: Compile and run the "Airtime.c" le. However, you need to compile and run also the "ETX_Broadcast.c" and the "ETX_Collect.c" les as the Airtime module uses ETX for error frame computation. See Section XXX 7. EXPERIMENTS It is very important to remind that the purpose of this work is not to evaluate the performance of IEEE 802.s WMNs. Instead, it is presenting a real-world IEEE 802.s WMN testbed implementation. However, we do go through some basic experiments for the purpose of implementation validation. In the following paragraphs, we describe the topological and experimental settings of the testbed and we compare the performance of IEEE 802.s with Airtime as a routing metric against ETX. 7. Topologies We used two different topologies for running experiments:. In the rst topology, see Figure 4, we deployed a wireless mesh network composed of mesh nodes. This later is located in the same building and is connected to the Internet. 2. In the second topology, we connected two WMNs, see Figure 5, through Internet. The rst WMN is composed of 7 nodes, and the other WMN is composed of 8 nodes. The two WMNs are located in different buildings. Both topologies were deployed at the second oor of the Shelby center at Auburn university. Besides the WMN nodes, we deployed an 802. legacy station (which serves as a client and is connected to the WMN through a MAP) and a station (which serves as a server). Using Iperf [2], we created UDP and TCP connections between the client and the server. In topology, the server resides on the Internet while in topology 2 the server lies on a WMN. 7.2 Settings In both topologies, the mesh nodes network interface cards (NICs) are operating in the 802.g channel while the 802. clients NICs are operating in the 802.g channel. The selection of different channels for client stations and mesh nodes is for the purpose of minimizing interferences between the client and the mesh nodes. The mesh nodes NICs run the open-source Madwi driver [4]: For a list of NICs that are compatible with Madwi, please refer to []. The HWMP PREQ messages were periodically sent every 4 seconds and the ETX probe frames are 024 Bytes long and are periodically sent every second [2]. During experiments, the TCP and UDP sessions were repeatedly run over periods of 20 seconds and averaged to plot the network throughput in function of client bandwidth. 7.3 Results First, during experimentation, we noticed a "ping-pong" effect in the behavior of the Airtime metric. This behavior is basically due to Airtime implicitly accounting for the load by means of accounting for the bit rate: The bit rate is dynamic as Madwi uses the default SampleRate rate adaptation algorithm [3] which adapts the transmission bit rate according to the observed link quality. Thus, when Airtime advises a less-loaded path, this later will soon become loaded, a fact that incites Airtime to favor another less-loaded path which will become soon loaded too, thus resulting in a "pingpong" effect. Hence, we strongly advise for a more stable routing metric. Besides, we do clearly notice that, in both topologies, Airtime and ETX have a quite similar performance for both TCP and UDP, see Figures [6, 7, 8, 9] : We depicted the graphs throughput in a scale of [0, 5] Mbps in order to relatively match the large scale of bandwidth which is [0, 0] Mbps, and this in order to give also an idea about the network efciency: Efficiency = T hroughput/bandwidth In fact, we cannot denitively state that Airtime and ETX have quite similar performance simply because the topologies we deployed have a quite limited set of alternative paths: In other words, Airtime and ETX are likely to pick the same hops as there are not many alternatives. Deploying larger topologies, with a larger set of alternative paths, is crucial to ascertain the performances of Airtime and ETX with regard to each other. The actual performance is quite poor when comparing the network throughput to client bandwidth, i.e., very low network efciency. We explain this poor performance by the following two main reasons: The testbed is using only a single channel: Performance degradation is a very known fact in single-channel single-radio multi-hop wireless networks [5], e.g., WMNs. This is mainly due to the fact that neighboring nodes, which are in the carrier sense range of each other, cannot transmit simultane-

10 0 UDP Throughput: Topology TCP Throughput: Topology ETX Airtime ETX Airtime UDP Throughput (Mbps) TCP Throughput (Mbps) Bandwidth (Mbps) Bandwidth (Mbps) Figure 6: Topology - UDP Throughput Figure 7: Topology - TCP Throughput UDP Throughput: Topology 2 TCP Throughput: Topology ETX Airtime ETX Airtime UDP Throughput (Mbps) TCP Throughput (Mbps) Bandwidth (Mbps) Bandwidth (Mbps) Figure 8: Topology 2 - UDP Throughput Figure 9: Topology 2 - TCP Throughput ously. This results in having only one active hop, in a route, at a time while the two other adjacent hops (always in the same route) cannot be active, a fact which noticeably affects network throughput. Furthermore, the nodes that are in the interferences range of each other would noticeably affect the network throughput as they would cause additional further bit errors. In this context, the multi-hop wireless researchers have been extensively investigating the use of multi-channels and multi-radios [7, 2, 23] in order to cope for the limitations of using single-radios. Actually, we are intending to have this open-source testbed support the use of multichannels and multi-radios in our future work. In the deployed topologies, the WMN nodes have been deployed in a quite dense region (See Figures 4, 5), a fact that renders links quite close to each others. In fact, this was limited by the available space. Ideally, the WMN nodes should be geometrically scattered and hence relatively far from each other in order to minimize the Interferences as well as to favor simultaneous transmissions between links. Still, we need to remind that evaluating the performance of IEEE 802.s is out of the scope of this work. 8. CONCLUSION AND FUTURE WORK We presented a real-world IEEE 802.s WMN testbed implementation which is an open-source and an easy-to-deploy one. We highlighted most important implementation problems and suggested suitable solutions. The implementation is an easy-to-deploy one as it does not involve any 802. MAC driver alteration, thus it can be deployed using using off-the-shelf 802. wireless cards. The implementation was made open-source and transparent by making the full code available online. The implemented testbed uses nodes in a rst topology and 5 nodes in a second topology where two WMNs, located in separate locations, are interconnected through Internet. The testbed is operational and supports real-world trafc, e.g., supporting web browsing sessions initiated from legacy 802. clients, that are connected to the Internet through the WMN. We have a strong belief that the presented implementation will be of great input to the WMN research community, especially in academia, as it easily provides a way for deploying a testbed and hence being able to perform more concise and real-world experimental research. The experiments we lead with HWMP, and with Airtime as a routing metric, showed a clear "ping-pong" behavior. On the other hand, ETX and Airtime exhibited quite similar performances. As a future work, and after getting feedback from interested WMN research community about the testbed, we aim to continuously maintain the current web site about the testbed and work for further versions with further enhancements, thus paving the way towards a widely used open-source WMN testbed. We are, also, intending to add the support for the multi-channel multi-radio technology and implement a more stable routing metric: A more stable routing metric might involve "looking" for a more stable link quality metric, e.g., Interferences. 9. REFERENCES [] Hardware supported by madwi. [2] The iperf project. [3] Madwi bit-rate selection algorithms. [4] The madwi project. [5] The netler project. [6] The ns-2 manual. [7] A pre-ieee 802.s wirless mesh network testbed. abidmoh/. [8] Doc ieee /063r0, May 2007.

11 [9] W. Wang Akyildiz, X. Wang. Wireless mesh networks: a survey. Elsevier Sience, [0] M. Bahr. Proposed routing for ieee 802.s wlan mesh networks. WICON, [] M. Barh. Update on the hybrid wireless mesh protocol of ieee 802.s. IEEE Conference on Mobile Adhoc and Sensor Systems, [2] J. Bicket D. De Couto, D. Aguayo and R. Morris. High-throughput path metric for multi-hop wireless routing. MOBICOM, [3] P. Gupta and P. R. Kumar. The capacity of wireless networks. IEEE Transactions on Information Theory, [4] Y. Kengo H. Aoki, T. Shinji and Y. Akira. Ieee 802.s wireless mesh network technology. IEEE NTT DoCoMo Technical Journal, [5] V. Padmanabhan L. Qiu K. Jain, J. Padhye. Impact of interference on multi-hop wireless network performance. MOBICM, [6] S. Keshav. A control-theoretic approach to ow control. SIGCOMM, 99. [7] Y. Liu and E. Knightly. Opportunistic fair scheduling over multiple wireless channels. INFOCOM, [8] Scalable Networks. Qualnet. [9] C. E. Perkins and E. M. Royer. Ad-hoc on-demand distance vector routing. WMCSA, 999. [20] M. Lewis R. Orgier, F. Templin. Topology dissemination based on reverse-path forwarding (tbrpf). RFC IETF, [2] A. Raniwala and T. Chiueh. Architecture and algorithms for an 802.-based multi-channel wireless mesh network. INFOCOM, [22] J. Padhye R.Draves and B. Zill. Routing in multi-radio, multi-hop wireless mesh networks. MOBICOM, [23] J. So and N. Vaidya. Multi-channel mac for ad hoc networks : Handling multi-channel hidden terminals using a single transiever. MOBIHOC, [24] IEEE TGs. Status of project ieee 802.s. January 200.