Performance Evaluation of OpenFlow Devices

Performance Evaluation of OpenFlow Devices Mariusz Żal, Janusz Kleban Poznan University of Technology Faculty of Electronic and Telecommunications Chair of Communication and Computer Networks Email: mariusz.zal@put.poznan.pl, janusz.kleban@put.poznan.pl Abstract The paper presents an environment for performance evaluation of OpenFlow capable devices. This environment was configured for performance evaluation of hardware employing the HAL (Hardware Abstraction Layer), proposed within the ALIEN FP7 project. The HAL layer implemented at non- OpenFlow devices convert them to OpenFlow capable devices. The proposed testing environment is based on OFLOPS application. In this paper experiment scenarios as well as selected results of tests obtained for Open VSwitch are presented. Index Terms - OpenFlow, Software Defined Networking, OFLOPS. I. INTRODUCTION The complexity of current IP networks is growing very fast. It is difficult to configure the network according to predefined policies, as well as to reconfigure it to respond to faults, load and changes. Each individual network device must be configured by network operators separately using vendor specific software. Two planes are implemented in networking devices and are responsible for traffic forwarding, namely a control plane and data plane. The control plane decides how to deal with network traffic. The data plane forwards traffic according to the control plane decisions. Implementation of these two planes in each network node reduces the flexibility of networks and its evolution. This state of affairs may be changed by Software Defined Networking (SDN) technology [1]. The SDN separates the network s control logic from routers and switches and employs a logically centralized controller simplifying policy enforcement, network configuration and evolution. The routers and switches become simple forwarding devices with implemented data plane rules. A well-defined application programming interface is needed to allow the SDN controller to exercise direct control on the data plane elements. The well known example of such an API is OpenFlow. OpenFlow is an open standard managed by the Open Networking Foundation. It separates the control plane implemented in an OpenFlow controller and the data plane performed by an OpenFlow switches. The behavior of the OpenFlow switch may be modified through a well-defined forwarding instruction set. According to instructions sent by the controller, the OpenFlow switch can behave like a router, switch, firewall, load balancer, traffic shaper etc. The OpenFlow protocol specifies a set of instructions that can be used by the controller to program the data plane of network devices. This paper presents performance tests which can be used to evaluate how fast selected actions can be performed by an OpenFlow switch implemented on different hardware platform. These tests were prepared for evaluation of an OpenFlow switch implementation within the ALIEN project. The ALIEN project aimed at delivering innovative Network abstraction layer to connect non-openflow capable equipment (called also alien hardware ) to OpenFlow environment. It was achieved by providing the Hardware Abstraction Layer (HAL) for non-openflow capable network devices. Alien hardware is any type of network device that does not support natively an OpenFlow. The following equipment was considered in the ALIEN project: NetFPGA cards, EZChip NP-3 network processors (in EZappliance platform), Cavium OCTEON Plus AMC Network processor (module in an ATCA systems and DELL switch), optical switches, GEPON OLT and ONU units, and DOCSIS hardware. The performance tests are designed in order to show that ALIEN platforms with implemented HAL have the performance comparable with existing commercial and open source OpenFlow switch implementation. We want to show, that ALIEN platforms are capable to act as native OpenFlow devices. The paper is organized as follows: Section II presents main ideas of OpenFlow switch, Section III describes testing environment, and Section IV includes description of tests scenarios. Section V presents selected results obtained for Open VSwitch. The paper is concluded in Section VI. II. OPENFLOW SWITCH The main components of an OpenFlow switch are shown in Figure 1 [2]. It consists of one or more flow tables and an OpenFlow channel to an external controller. The flow tables are used to manage flows and perform packet lookups, and forwarding. The OpenFlow channel is used by the controller to manage the switch via the OpenFlow protocol. Using this communication channel the controller can add, update, and delete flow entries in flow tables. Each flow table contains a set of flow entries with match fields, counters, and a set of instructions to apply to matching packets. Matching starts at the first flow table and continues to additional flow tables until a matching entry is found. In the case of matching, the instructions associated with the matching entry are executed. If no match is found in the flow table the packet may be forwarded to the controller over the OpenFlow channel, may 42 XVIII Poznańskie Warsztaty Telekomunikacyjne - Poznań, 12 grudnia 2014

Fig. 1. Main components of the OpenFlow switch be dropped or may continue to the next flow table. Actions performed on packets and pipeline processing are defined by a set of instructions associated with each flow entry. Packet forwarding, packet modification and group table processing are described by actions. Pipeline processing instructions define how subsequent tables process packets and what kind of information, in the form of metadata, is sent between tables. If a next table is not specified by the instruction set associated with a matching flow entry the table pipeline processing stops and the actions in the pre-defined action set of the packet are executed. Packets may be forwarded to a physical or logical port. The logical port may be defined by the switch or by the OpenFlow switch specification. Ports defined by the OpenFlow switch specification are called reserved ports. These ports may specify generic forwarding actions such as: sending to the controller, flooding, or forwarding using non-openflow methods. The switch-defined logical ports may specify link aggregation groups, tunnels or loopback interfaces. Additional processing is specified by a group table. The group table consists of group entries and represents sets of actions for flooding, as well as more complex forwarding semantics (e.g. multipath, fast reroute, and link aggregation). Each group entry contains a list of actions buckets with specific semantics dependent on group type. The actions in one or more action buckets are applied to packets sent to the group. OpenFlow ports are the logical network interfaces made by an OpenFlow switch for OpenFlow processing. Using these ports OpenFlow switches connect logically to each other. The number of OpenFlow ports may not be identical to the number of network interfaces provided by the switch hardware. Some physical network interfaces may be disabled for OpenFlow, and some additional OpenFlow ports may be defined by the switch. OpenFlow packets arrive to an ingress port, next they are processed by the OpenFlow pipeline, and finally they may be forwarded to an output port. An OpenFlow switch must support three types of OpenFlow ports: physical ports, logical Fig. 2. Packet flow through the processing pipeline ports and reserved ports. OpenFlow switches may be classified into two types: OpenFlow-only, and OpenFlow-hybrid. OpenFlow-only switches support only OpenFlow operations, and all packets are processed by the OpenFlow pipeline. OpenFlow-hybrid switches support both OpenFlow operations and normal Ethernet (or any Layer 2 techniques) operations. These switches must classify traffic and route it to either the OpenFlow pipeline or the normal pipeline. In this kind of switch packets may go from the OpenFlow pipeline to the normal pipeline through reserved ports. The main components of the OpenFlow pipeline are shown in Figure 2 [2]. The OpenFlow pipeline processing defines interactions between packets and flow tables containing multiple flow entries. One or more flow tables may be implemented in the OpenFlow switch. The pipeline processing is greatly simplified when only a single flow table is used. In the case of multiple flow tables they are sequentially numbered, starting at 0. During pipeline processing the OpenFlow packet is first matched against flow entries of flow table 0. Depending on the outcome of the match in the first table, other flow tables may be used. If the flow entry is found, the instruction set included in that flow entry is executed. The packet may be directed to another flow table, where the same process is repeated again. If the packet is not directed to another flow entry, pipeline processing stops at this table. Next, the packet is processed according to the associated action set and forwarded. Unmatched packets are: drooped, passed to another table or to the controller over the control channel. Each flow table contains flow entries. Each flow table entry consists of the following fields: Match Fields, Priority, Counters, Instructions, Timeouts, and Cookie: Match Fields - to match against packets. These fields comprise the ingress port and packet headers, and optionally metadata specified by a previous table. Priority - matching precedence of the flow entry. Counters - are updated when packets are matched. Instructions - to modify the action set or pipeline processing. Timeouts - maximum amount of time or idle time before flow is expired by the switch. Cookie - opaque data value chosen by the controller. May be used by the controller to filter flow statistics, flow modification and flow deletion. A flow table entry is identified by its match fields and priority: the match fields and priority taken together identify a unique flow entry in the flow table. The flow entry that XVIII Poznańskie Warsztaty Telekomunikacyjne - Poznań, 12 grudnia 2014 43

wildcards all fields (all fields omitted) and has priority equal to 0 is called the table-miss flow entry. Flow entries may be removed from flow tables in two ways: at the request of the controller or by the switch flow expiry mechanism. This mechanism is based on the two parameters: idle_timeout and hard_timeout. These parameters are associated with each flow entry. If the hard_timeout_field is non-zero, the entry s arrival time must be noted by the switch and the flow entry has to be removed after the given number of seconds. If the idle_time_field is non-zero, the arrival time of the last packet of the flow must be noted by the switch, and the flow entry has to be removed when it has matched no packets in the given number of seconds. When one of the timeouts is exceeded the associated flow entries must be removed from the flow table. Flow entries may be also removed by the controller using delete flow table modification messages. After deletion of a flow entry the switch must check the flow entry s flag. If the flag is set, the switch must send a flow removed message to the controller. Each flow removed message consists of the following: a complete description of the flow entry, the reason for removal (expiry or deletion), the flow duration at the time of removal, and the flow statistics at the time of removal. The OpenFlow channel is the interface that connects each OpenFlow switch to a controller. It is usually encrypted and may be run over the TCP/IP protocol stack. Using the OpenFlow channel the controller configures and manages the switch, receives messages about events from the switch, and sends packets out the switch. A typical OpenFlow controller manages multiple OpenFlow channels, each one to a different OpenFlow switch. An OpenFlow switch may have one Open- Flow channel to a single controller, or multiple channels for reliability, each to a different controller. The OpenFlow switch always initiates connections to an OpenFlow controller. All OpenFlow Channel messages must be formatted according to the OpenFlow protocol, which supports three message types: controller-to-switch, asynchronous, and symmetric, each with multiple sub-types. Controller-to-switch messages are initiated by the controller and used to directly manage or inspect the state of the switch. Asynchronous messages are initiated by the switch and used to update the controller about network events and changes to the switch state. Symmetric messages are initiated by either the switch or the controller and sent without solicitation. The OpenFlow protocol provides reliable message delivery and processing. It does not automatically provide acknowledgements or ensure ordered message processing. If the OpenFlow channel fails, the switch may have gone into fail standalone mode. Switches must process every message received from a controller, and possibly generate a reply. If a switch cannot completely process a message received from a controller, it must send back an error message. Packets may be silently dropped after OpenFlow processing due to congestion at the switch, QoS policy, or if sent to a blocked or invalid port. Switches must send to the controller all asynchronous messages generated by the OpenFlow state changes, such as flow-removed, port-status or packet-in messages to ensure that the controller has the actual knowledge about the switch state. Packets may be dropped also when they fail to match any entries in a flow table, and that table s default action is to send to the controller. Controllers are free to ignore messages they receive, but must respond to echo messages to prevent the switch from terminating the connection. The detailed information about each component of the OpenFlow switch as well as the OpenFlow protocol may be found in [2]. III. TESTING ENVIRONMENT OVERVIEW ALIEN platforms performance tests are based on OFLOPS application [3], [4]. The OFLOPS is a tool that allows developing use-case tests for both hardware and software OpenFlow implementations. Using this tool, developers can define and simulate specific usage scenario and understand the impact of switch implementation on its performance. The OFLOPS platform is implemented as multi-thread application without any concurrent access control. It results in reduced latency. The OFLOPS consists of the following five threads, each one serving specific type of events (Figure 3): 1) Packet generator - controls data plane traffic generators, 2) Packet capture/analyzer - captures and pushes data plane traffic to modules, 3) Message generator - translate OpenFlow packets to control events or generates OpenFlow messages, 4) SNMP channel - perform asynchronous SNMP polling (because selected HAL implementation does not support SNMP protocol, in prepared tests this module is switched off), 5) Action scheduler - manages time events scheduled by measurement modules The OFLOPS application offers many ready to use experiment scenarios. Some of these scenarios are adopted to performance evaluation of the OpenFlow implementation on ALIEN platforms. These scenarios and modified application will be referred to later in this document as ALIEN OFLOPS. In order to avoid any additional delay caused by other network devices, the host with installed ALIEN OFLOPS application is connected directly with tested platform. The minimal number of required interfaces is equal to two, one for control channel and one for data path. Unfortunately, in this configuration only two test scenarios can be realized. In order to realize full set of prepared test scenarios two interfaces in data plane are required. The OFLOPS application allows to use also the third data plane interface, but for our purpose, this interface is superfluous. The test scenarios are implemented as runtime linked modules written in C++ code. The parameters of each experiment are configured using the configuration script presented in Listing 1. 44 XVIII Poznańskie Warsztaty Telekomunikacyjne - Poznań, 12 grudnia 2014

Fig. 3. Hardware configuration oflops : { control : { control_dev = "eth0 "; control_port = 6633; snmp_addr = "10.1.0.2"; cpu_mib="1.3.6.1.2.1.25.3.3.1.2.768"; in_mib="1.3.6.1.2.1.2.2.1.11.2"; out_mib ="1.3.6.1.2.1.2.2.1.17.2"; snmp_community = "public "; }; data = ({ dev="eth1 "; port_num=8; in_snmp_mib="1.3.6.1.2.1.2.2.1.11.3"; out_snmp_mib ="1.3.6.1.2.1.2.2.1.17.3"; },{ dev="eth2 "; port_num=7; in_snmp_mib="1.3.6.1.2.1.2.2.1.11.4"; out_snmp_mib ="1.3.6.1.2.1.2.2.1.17.4"; },{ dev="eth3 "; port_num=6; in_snmp_mib="1.3.6.1.2.1.2.2.1.11.5"; out_snmp_mib ="1.3.6.1.2.1.2.2.1.17.5"; }); traffic_generator = 1; dump_control_channel = 0; module : ({ path="/home/ alien / oflops / example_modules / openflow_add_flow /. libs /libopenflow_add_flow.so"; }); }; param="flows=1 data_rate=500/ probe_rate=500 pkt_size=500 table=0/ probe_time =120 echo_rate =4"; Listing 1. OFLOPS configuration file IV. DESCRIPTION OF EXPERIMENT SCENARIOS A. PacketIn test For every packet that does not have a matching flow entry or if a packet matches an entry with a forward to In_Port action or when flow table is empty, an asynchronous packet-in event is sent to the controller. If the switch has sufficient memory to buffer packets that are sent to the controller, the packet-in events contain some fraction of the packet header (by default 128 bytes) and a buffer ID to be used by the controller, when it is ready for the switch to forward the packet. Switches that do not support internal buffering (or have run out of internal buffering) must send the full packet to the controller as part of the event. ALIEN platforms are based on existing network devices and don t support such packet buffering. In In_Port action the OpenFlow controller receives whole data packets. It is very important to check, how many In_Packet actions can be served by Alien platforms and how length of data packets affect the delay of packet transfer. In order to measure In_Packet action throughput and packet delay, the following scenario, presented in Figure 4, was prepared. When ALIEN-OFLOPS application receives OFPT_HELLO messages, OFPT_FLOW_MOD message is sent. OFPT_FLOW_MOD contains OFPFC_DELETE action, which is applied to all installed flows. Simultaneously, through dev1 ALIEN-OFLOPS starts generate UDP packets, which contain (in payload field) sequence number and generation time. Since the OpenFlow switch flow table is empty, switch sends OFPT_PACKET_IN messages containing UDP packets. Basing on reception of OFPT_PACKET_IN message and generation time of conveyed UDP packet we can calculate delay of In_Packet action. Additionally, basing on sequence numbers loss probability is calculated. Test is repeated for different UDP packets sizes and different time interval between packets. B. PacketOut test This test scenario, presented in Figure 5, is similar to previous one. In this test ALIEN-OFLOPS application generates sequence of OFPT_PACKET_OUT messages, which convey UDP packets. As previously, UDP packets are marked by sequence numbers and OFPT_PACKET_OUT message generation time. The OFPT_PACKET_OUT message contains also of_port number determining the output port of the action. Basing on reception of UDP packets and generation time conveyed in this packet we can calculate delay of Out_Packet action. Moreover basing on sequence numbers loss probability is calculated. Test is repeated for different UDP packet sizes and different time interval between packets. C. Path_delay test The Path_delay test allows determining path delay in control and data plane. In case of control plane both, OFPT_ECHO_REQUEST and OFPT_ECHO_REPLY messages are used. The data field in echo request message must be copied into echo replay data field. When OFPT_ECHO_REQUEUST is generated into this field the XVIII Poznańskie Warsztaty Telekomunikacyjne - Poznań, 12 grudnia 2014 45

Fig. 6. PathDelay test scenario Fig. 4. PacketIn test scenario diagram for this experiment is presented in Figure 6. Fig. 5. PacketOut test scenario current system time is written. The difference between OFPT_ECHO_REPLY message reception time and the time included in data field of this message, indicates the control path delay. In the case of data plane path delay UDP packets are marked by the sequence numbers and packet generation time. In order to reduce packet processing time the flow table stores only one defined flow entry matches for incoming UDP packets and directs them into specific output port. Path delay in control plane is determined without any parameters. Path delay in data plane is studied for different packet sizes and different time intervals between packets. The message flow D. AddFlow test The aim of this test is to determine time required to add the given number of flows. Only exact match flows entries are considered. This test procedure is repeated several times in order to achieve reliable test results. During whole test time ALIEN-OFLOPS application generates UDP packets containing in data field packet generation time and send these messages through dev1 port. In the first step Alien platform flow table is cleared. Next, the determined number of ADD_FLOW operation is generated and sent. It starts time counter. Only the last added flow entry matches generated UDP packets. According to action bound with this flow, the generated packets which appear on sdev1 port are directed to sdev2 port. When some packet occurson dev2 port triggers the test procedure repetition and stops time counter. Time counter shows how long flows are installed. The message flow diagram for this experiment is presented in Figure 7. V. TESTING ENVIRONMENT AND RESULTS ALIEN OFLOPS application was used for performance evaluation of OpenFlow functionality implementation on ALIEN platforms. The source code and installation procedure of presented application are available on [5]. The results of performance tests for ALIEN platforms were compared with the results obtained for the software OpenFlow switch OVS (Open VSwitch) [6]. Both, ALIEN OFLOPS application and OVS switch, were configured as two virtual machines connected directly through VirtualBox internal networks (see Figure 8). In Figures 9 and 10 results for PacketIn, PacketOut and AddFlow tests are presented. In Figure 10 we can observe, that OFTP_PACKET_OUT execution time is smaller by two orders of magnitude than time consumed by OFPT_PACKET_IN actions. In case of AddFlow test, presented in Figure 9, we 46 XVIII Poznańskie Warsztaty Telekomunikacyjne - Poznań, 12 grudnia 2014

μ Fig. 7. AddFlow test scenario μμ μμ Fig. 9. Time required by modify flow entry message Fig. 10. Delays of OFPT_PACKET_OUT and OFPT_PACKET_IN Open- Flow command executions of HAL. The detailed description of test results obtained for different platforms considered within the ALIEN project may be found in D5.3 report, which is available on the project website in section Deliverables (http://www.fp7-alien.eu). ACKNOWLEDGEMENTS The work described in this paper was partially financed from funds of Ministry of Science and Higher Education for year 2014. Fig. 8. Implementation of OpenFlow reference switch performance testing environment can see, that time required to add given number of entries to flow table increases exponentially. VI. CONCLUSIONS This paper presents basic issues related to performance tests of OpenFlow capable devices. The presented testing environment was used in ALIEN project for performance evaluation of non-openflow devices which were converted to OpenFlow capable devices using HAL layer. The tests allow us also to draw some conclusions concerning prototype implementation REFERENCES [1] D. Kreutz, F. M. V. Ramos, P. Verissimo, Ch. E. Rothenberg, S. Azodolmolky, S. Uhlig. (2014, October). Software-Defined Networking: A Comprehensive Survey. To appear in Proceedings of the IEEE, 2015, current version 2.01, pp. 1-61, [On-line]. Available: http://arxiv.org/pdf/1406.0440v3.pdf [2] OpenFlow Switch Specification v. 1.3.4. (2014, March). Open Networking Foundation [Online]. Available: https://www.opennetworking.org/images/stories/downloads/sdnresources/onf-specifications/openflow/openflow-switch-v1.3.4.pdf [3] OFLOPS webpage, http://archive.openflow.org/wk/index.php/oflops [4] Charalampos Rotsos, Nadi Sarrar, Steve Uhlig, Rob Sherwood, Andrew W. Moore, "OFLOPS: An Open Framework for OpenFlow Switch Evaluation", in Passive and Active Measurement, Lecture Notes in Computer Science Volume 7192, 2012, pp 85-95, Springer 2012 [5] FP7 ALIEN performance test webpage, https://github.com/fp7- alien/performance-tests, [6] Open VSwitch webpage, http://http://openvswitch.org/ XVIII Poznańskie Warsztaty Telekomunikacyjne - Poznań, 12 grudnia 2014 47