NoEncap: Overlay Network Virtualization with no Encapsulation Overheads

Transcription

1 : Overlay Network Virtualization with no Encapsulation Overheads Sergey Guenender, Katherine Barabash, Yaniv Ben-Itzhak, Anna Levin, Eran Raichstein, and Liran Schour IBM Research Lab Haifa, Israel {guenen, kathy, yanivb, lanna, eranra, ABSTRACT Overlay network virtualization quickly gains traction in today s multi-tenant data centers due to its ability to provide independent virtual networks, at scale, along with complete isolation from the underlying physical network. Despite the benefits, performance degradation due to the imposed perpacket encapsulation overhead is a serious impediment. Mitigation approaches are mostly hardware based and thus depend on costly networking gear upgrades and suffer from lesser flexibility and longer times to market, compared to software solutions. Software optimizations proposed so far are limited in scope, applicability, and interoperability. In this paper we present, a software-only optimization, capable of eliminating almost completely the overheads, while fully preserving the benefits of an overlay-based network virtualization. Categories and Subject Descriptors C.2.0[Computer Communication Networks]: General Data communications General Terms Design, Experimentation, Optimization Keywords Network Virtualization, Overlay Network Virtualization, Data- Center Virtualization, SDN 1. INTRODUCTION Modern cloud-scale data centers sustain increasingly large amounts of independent, highly dynamic, and communication intensive workloads, consolidated over common physical infrastructure. Workload consolidation is enabled by advances in server virtualization technology which allows running multiple independent execution environments, e.g. Virtual Machines (VMs) or containers, on a single physical Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. SOSR2015 June 17-18, 2015, Santa Clara, CA, USA Copyright 2015 ACM. ISBN /15/06...$ host, with additional benefits of automatic VM provisioning, VM migration between the physical servers, and greater independence from the host hardware. Traditionally, VM communication is achieved by bridging virtualized network interfaces to the underlying physical data center network (DCN), through a thin layer of host software (e.g. Linux bridge). As the consolidation levels and the amount and the diversity of workloads and their communication needs increase, virtualizing the network becomes crucial to the success of cloud-scale data centers [12]. Although acute in the multi-tenant cloud-scale environments, network virtualization has been around for over a decade [7] in traditional, non-virtualized, data centers, e.g. for workload isolation. Best known traditional network virtualization technology is VLAN with its 12-bit identifier located between the Ethernet and the IP headers. Being widely adopted and ubiquitously supported in networking gear, both hardware and software, VLAN was natural first choice for isolating VM networks. It has quickly become apparent, however, that some of the inherent limitations of VLAN technology make it unsuitable for efficient, scalable, and flexible network virtualization in large-scale multitenant virtualized environments, e.g. clouds. First, 12-bit VLAN identifier limits the scalability of cloud networks to K isolation domains. Second, VLAN segmentation is restricted to a single L2 network and therefore cannot by itself support extension of a single virtual network across L3 boundaries. Third, dynamic nature of today s workloads calls for high event frequency in VMs life cycle, while VLAN configuration, traditionally being fairly static and hardwarebound, does not lend itself easily to efficient automation at scale [12]. One of the solutions for the aforementioned and other limitations of the VLAN technology is overlay network virtualization, capable of complete separation of virtual networks not only from each other but from the underlying physical network. Examples of overlay-based network virtualization solutions existing today include VMware NSX, IBM DOVE and SDN VE, Midokura Midonet, Google Andromeda, Microsoft Azure, and more. Solutions differ in approaches to major building blocks and compete in offered features and capabilities, while having the same common structure with the following required constituents: encapsulation protocol, tunnel termination devices (SW or HW), and a mechanism for VM location management(collection and dissemination). In the data plane, different encapsulation protocols are used, e.g. VXLAN [17], NVGRE [18], STT [10], and Geneve [13]. Tunnel termination can be implemented in edge switches,

2 in virtual switches, or in add-on appliances. Location management can be centralized or distributed, SDN-based, or even delegated to the underlying network services, e.g. multicast groups or evpn can be used for managing VxLAN networks. Although highly beneficial for its capability to virtualize networks at scale, overlay approach has its limitations, the most prominent being the need to encapsulate all the crossserver virtual networks data. When done in host software, encapsulation drains server resources and reduces the communication speed, impacting the end to end performance of the deployed applications. In addition to the direct overhead of additional headers handling, overlay approach stresses the network by increasing the total amount of data sent over the physical links and sometimes requires MTU reconfiguration along the path of the encapsulated traffic. More significantly, encapsulated data is opaque to existing in-network devices and services, including optimizers and offload engines like TSO, LRO, etc. In this paper, we introduce a completely software approach for mitigating encapsulation overheads. allows to continue enjoying all the benefits of overlay virtualization while avoiding the performance degradation for a significant subset of communication flows. In a nutshell, trades off major improvement in data plane performance for minor increase in control plane complexity. The rest of the paper is structured as follows: solution architecture is described in Section 2; solution implementation based on IBM DOVE prototype is described in Section 3; evaluation results are presented and analysed in Section ; Section 5 surveys the related work and Section 6 concludes the paper with discussion and future work directions. 2. ARCHITECTURE Before describing architecture, we first outline the generic overlay based network virtualization solution, using the framework and the terminology suggested by the NVO3 standardization group [16]. The solution provides isolated Virtual Networks (VNs) identified by their Virtual Network Contexts and comprises multiple Network Virtualization Edge (NVE) modules and a Network Virtualization Authority (NVA). NVE modules intercept traffic generated by VN clients and use encapsulation tunnels to send it over the physical network, while NVA manages the VNs, the tunnels, and, to some extent, the NVE modules. Figure 1(a) illustrates data flow between two VN clients that can be VMs, bare metal servers, or containers. Data sent by VN Client s is intercepted by its hosting NVE module, NVE s, which inspects the packet headers and, in consultation with the NVA, resolves the VN Context and the location of the destination VN client, VN Client d, and its hosting NVE module, NVE d. NVE s encapsulates the flow s packets and sends them over the physical network towards the NVE d which, in consultation with the NVA, decapsulates the packets and delivers them to VN Client d. To minimize the amount of VN Context and location resolutions, all NVE modules cache the resolved information, preferably for the entire flow duration time. Cache expiration and invalidation due to virtual or physical events, e.g. VM migration, policy updates, port failover, etc, must be seamlessly supported by every specific implementation. With overlay network virtualization, data exchanged between VN clients is delivered over the physical network in fragments carrying multiple headers: inner, or virtualization level, headers created by VN clients and outer, or physical level headers, appended by NVEs. The format and the contents of the inner headers depend on the communication protocol in use by the VN clients, while the format and the contents of the outer headers depend both on the communication protocol in use by the NVEs and on the overlay encapsulation format. Figure 1(b) presents an example of inner and outer headers for a case of VN clients communicating over TCP/IP/Ethernet stack and outer headers dictated by the VxLAN[17] encapsulation format. In this case, outer headers obscure the VN-level TCP headers from TCP Segmentation Offload (TSO) and Large Recieve Offload (LRO) engines ubiquitously deployed in host network adapters (NICs). As a result, VN-level TCP segmentation and reassembly must happen in software, leading to major performance degradation and failure to leverage the available NIC capability. To regain the benefits of NIC-based TCP optimizations, we build upon two observations. First, IP over Ethernet is the most widespread communication service today, deployed in majority of DCNs and expected by majority of network clients. Second, when both the VN clients and the NVEs communicate through the TCP/IP/Eth, inner packet headers are structured in exactly the same way as outer headers. With, we exploit this property to offload entire TCP sessions from VN clients to NVEs, by replacing the contents of inner header fields with the NVE-level data. Actual communication payload of VN clients is therefore transmitted as if it was data communicated between NVEs. Original VN-level header fields are entrusted to the NVA data store in exchange for a numeric key that we denote Flow Identification Tag or, shortly, FIT. In architecture FIT is assigned at flow initiation time and included in all the packets of its flow. The mapping between the original inner headers content and the FIT must be unique in the context of a pair of communicating NVEs and can be generated either centrally, e.g. by SDN controller, or distributively, e.g. through direct communication between the source and the target NVEs. Small integer range is sufficient to support all the possible simultaneous TCP sessions served by a pair of NVEs, therefore FIT can be easily accommodated in an unused outer header field of TCP packet, e.g. in the TCP source port. Figure 1(c) presents control and data plane information for TCP processing under, to accompany generic series of flow processing events in Figure 1(a). As before, data packets sent by VN Client s are intercepted by NVE s. In addition to resolving VN Context and destination location, NVE s makes (or receives) a decision whether the flow is eligible to processing or should be handled with the encapsulation approach. Flows not chosen for processing are treated as dictated by the overlay solution at hand. Each flow, selected for processing, is assigned a FIT, currently unused in the scope of communicating NVEs pair. A mapping between the original header fields and the FIT is stored in NVA and is cached in the involved NVEs. NVE s rewrites the original header fields to contain NVE-level addresses and to include FIT (in implementation specific way, e.g. as a TCP source port) and sends the modified packet over the physical network towards NVE d. Upon receiving packet, NVE d retrieves the FIT from the source TCP port field, obtains flow s original

3 Data and Control Path Encapsulation-Based Solutions VN Client S (vip S ; vmac S ) 1 VNE S (pip S ; pmac S ) 2 Virtual Environment Physical Environment 3 Control Environment VNA VN Client D (vip D ; vmac D ) 5 VNE D (pip D ; pmac D ) Controller DB: pip pmac VNID UDP Header vmac vmac vip vip Src Src Encapsulation Header (e.g., VxLAN) Dst Dst vip S vip D VNID S pip D VNID D ( ( ( Data Path pip S pip D Proto FIT vip S vip D SrcP * DstP * VNID S Control Path * SrcP/DstP= Source/Destination a) b) c) vmac pmac vip pip Src FIT Dst No Encap vip S vip D VNID S pip D VNID D Figure 1: Overlay Network Virtualization Architecture: (a) - generalized flow of packet processing events; (b) - example data plane and control plane contents for VxLAN encapsulation; (c) - example data plane and control plane contents for optimization header fields together with the VN Context from NVA or from its cache, restores the original packet headers, and delivers the packet to its ultimate destination, VN Client d. Architecture requires implementing a method for classifying traffic as such at receiving NVE. This can be done, for example, by designating a TCP destination port to communications. Decision to apply depends on current network state, on whether flow s SLA is met, or on whether the flow is large and long (elephant), communication intensive, or mission critical. Architecturally, the decision maker is a separate component, while implementations may vary, entrusting the decision making to the centralized or distributed NVA, to the higher level management/orchestration engines, or even to end users e.g. all TCP flows of tenant paying for platinum service are eligible. The decision making can be automated, e.g. based on continuous monitoring of performance and state. In addition to selective applicability, decision to subject a flow to processing can be dynamically reconsidered during flow lifetime, so that flows can initially be handled with a regular encapsulation and later upgraded to handling and vice versa. As with regular overlay processing, all NVEs cache resolved flows data, to be applied to further packets of already active flows. Cache expiration and invalidation rules apply to flows similarly to regular overlay flows, with the addition of dynamic reconfiguration of flow handling method. 3. IMPLEMENTATION To validate architecture, we have prototyped it as an extension over IBM Distributed Overlay Virtual Network (DOVE) [9]. In this section, we briefly describe an extensible prototype of the underlying overlay network virtualization solution, based on open components. We then present extensions implementing the processing and ensuring co-existence of the and the regular VxLAN processing in the same forwarding plane. IBM DOVE is an overlay network virtualization technology combining VxLAN encapsulation in the data plane, logically centralized controllers cluster in the control plane, and a unique policy-based virtual network abstraction in the management and configuration plane. DOVE is integrated with OpenStack Neutron [5] for data center wide orchestration and management [8]. For easier extensibility and experimentation, we maintain a light weight version of DOVE, created with open source components for KVM [1] based x86-6 virtualization environments. In our implementation, an unmodified Open vswitch (OVS) [] is used as NVE module and Ryu Open Flow (OF) controller framework [6] as a base for a simplified version of DOVE Policy Service (DPS). This simple DPS implements both the configuration/management and the control functionality, i.e. it is responsible for maintaining the VN semantics DOVE networks context and policies, for distributing VM location and VN policy information among the DOVE NVE modules OVS instances, and for controlling OVS data processing rules. In DOVE, virtual network clients are partitioned into virtual groups. Special policies allow or forbid communication between members of any pair of groups, or force their communication to pass through one or more network services. In our minimalistic DOVE prototype, every virtual group is allocated a unique VxLAN ID. 3.1 Extending DOVE for WehaveextendedDPStohandleFITallocationandmanagement and to program OVS switches to overwrite and restore packet headers using OpenFlow(OF) protocol. Rewritten packets carry FIT from the source to the destination OVS in the source TCP port field, while designated TCP destination port number is used to identify packets as such. Distinguishing between the regular TCP traffic targeted tovmhostsfrom traffictargetedtohostedvmsat destination is accomplished through OVS configuration and static OF rules. As shown in Figure 2, OVS is configured in a dual datapath 1 setup with static OF rules in additional Mux datapath, forcing traffic to bypass the host IP stack. Design and implementation of component responsible for selective application of is out of the scope of this work. Initial prototype described here allows for plugging such a component into a DPS in the future and ensures that both the and the encapsulated flows can co-exist and be forwarded simultaneously. 1 The overhead of using the second datapath was tested and found to be negligible.

4 0 Dynamic Configuration Initial Configuration VM1 A (vip1 VM1 A ) B vnic (vip1 B ) vnic MTU vnic MTU vnic. VM2.. B (vip2 B ) DOVE OVS Data-Path Traffic Traffic DOVE OVS Data-Path Mux OVS Data-Path VM A (vip A ) VxLAN-UDP-Socket VxLAN Traffic Throughput [Gbps] Control Path Data Path Physical NIC (pip A ) MTU Phy Host A Host B Figure 2: OVS setup for. EVALUATION We evaluate implementation described in Section 3 through a series of experiments. All experiments share the same environment, consisting of a pair of IBM System x3550 M servers connected back-to-back with 0GbE link. Each server is equipped with 197GB of RAM, two Xeon(R) CPUs 2, and Mellanox ConnectX-3 0GbE network card capable of VxLAN offload. Physical NICs are pinned to dedicated CPU cores (3 first cores on each server) and configured with 1500 bytes MTU Phy. Both servers have BIOS settings set for maximum performance, irqbalance service turned off, vhost-net module loaded at boot time, and TCP/IP stacks fine-tuned for achieving the highest network performance. For virtualization, each server runs KVM and libvirt [2] and hosts four VMs, provisioned with two dedicated cores, 1GB RAM, a virtual NIC connected to management network, and a virtual NIC connected to DOVE OVS datapath. Schematic representation of experimental environment is shown on Figure 2. We have configured the experimental environment into four distinct setups. For each setup, we have run a series of tests to collect statistically solid numbers for key network performance indicators (KPIs) throughput, latency, and efficiency. The following four experiment setups are defined and compared: SwVxLAN setup represents unoptimized overlay network vitalization solution where all the VMs communications are subject to VxLAN processing. setup represents the most active usage of where all the VMs TCP communications are subject to processing, while rest of the traffic is processed with VxLAN. HwVxLAN setup represents the HW offload solution, were all the VxLAN hardware acceleration capabilities are enabled in Mellanox ConnectX-3 PRO network cards. setup is used as the upper performance bound. Here, VMs are bridged to the physical network and experience no encapsulation overhead. In order to accommodate for encapsulation headers, guest machines have their MTU vnic reduced to 150 bytes in 2 12 cores, HyperThreading disabled. Figure 3: Aggregated host throughput SwVxLAN, and HwVxLAN setups. For the remaining setup, 1500 MTU vnic is used (to this setup relative advantage). For all the KPIs we evaluated, the differences between the HwVxLAN and setups were negligible, therefore in what follows we, for a sake of brevity, present and omit HwVxLAN results..1 Evaluation results and analysis For all the experiments, we generate traffic between pairs of VMs such that the source and the destination VM reside on different host, e.g., VM1 A in Figure 2 talks to VM1 B. The results are obtained for runs with one to four communicating simultaneously. Throughput. In order to evaluate the achieved aggregated throughput, we run four 3 TCP sessions between each pair of VMs, using iperf, and measure the results over the physical host s NIC with ifstat. As shown in Figure 3, achieves approximately the same aggregated throughput as. In addition, while both and reach the physical line rate (0Gbps) for three and four, SwVxLAN is not capable of saturating the link or scale with the amount of communication flows. Efficiency. To evaluate efficiency, we normalize achieved aggregated throughput by the total CPU load measured by vmstat running on hosts. Figures a and b present sender and receiver efficiency, indicating that SwVxLAN is significantly less efficient than and for both sides. It can be seen that is more efficient at sending while at receiving, with position improving with load. Having run a separate set of tests with mpstat collecting detailed CPU load statistics for irq, sys and guest CPU time per core, we attribute this tendency to hasing algorithm used by the network cards favoring higher entropy in packet headers for growing number of simultaneous streams. Latency. To measure latency, we run five consecutive netperf TCP_RR tests, each for 120 seconds, trimming the first and the last 20 seconds to eliminate the measurement inaccuracies of TCP connection establishment and termination. The round-trip-time (RTT) results, averaged over all the VMs, are presented in Figure 5. It can be seen that achieves the lowest RTT. The RTT difference between and varies from 1.5µseconds 3 Beyond this number CPU becomes the bottleneck.

5 [Gbps]/CPU Load[%] [Gbps]/CPU Load[%] (a) Sender (b) Receiver Figure : Sender and receiver efficiency for a single VM pair to 2.3 µseconds for four. RTT is the highest for SwVxLAN as in that setup the packets are processed by the hosts IP stack before being forwarded to DOVE OVS. In, on the other hand, the packets are processed only by the host MAC layer, resulting in on averge 13.2 µseconds shorter RTT than in SwVxLAN. In summary, our evaluation demonstrates that offers significantly better performance as compared to Software VxLAN, in terms of throughput, efficiency, and latency. performance is very close to both the and the HwVxLAN, while not having their drawbacks, e.g. HW dependency for HwVxLAN and lack of virtualization for. 5. RELATED WORK is not the first solution for eliminating performance degradation of overlay virtualization while retaining its highly prized benefits. All the proposed solutions can be roughly subdivided into the hardware based (HW) and the software based (SW) camps. As overlay network virtualization gains popularity, more and more HW vendors offer virtualization support in their gear, both in NICs and in edge switches. In the simplest case, hardware forwarding pipeline is upgraded to recognise the encapsulation, skip it, and apply optimizations like TSO. In more advanced cases, HW can fully take over the tunnel termination functionality. For instance, both Intel and Mellanox have introduced server adapters capable of VXLAN and NVGRE offloading [3], while all the major switch vendors support advanced VxLAN processing in their switches. Although promising, HW-based approach has its drawbacks, namely: the need to replace existing gear with potentially more expensive; longer development cycles for new features, e.g. even small changes in encapsulation format; more complex HW often requires more tweaking and tuning to achieve optimal performance; increased risk of vendor dependency and lock-in. Software solutions are more flexible and do not have most of the drawbacks associated with hardware. For most situations, SW flexibility and extensibility poses serious competition to the additional performance improvement offered by special purpose HW. In what follows, we overview the most prominent software optimization approaches and compare them to. Like, STT protocol [10] was designed to regain the TCP segmentation offload benefits. STT is an IP-based encapsulation utilizing a TCP-like header over IP which does not include any TCP connection state associated with the tunnel, so that loss of a single segment can potentially lead to retransmission of the whole 6K packet. With, the whole TCP session is offloaded to the host and packets are sent with real, slightly modified, TCP headers, for a significant advantage of over STT in lossy environments. Like with almost any encapsulation approach, additional headers imposed by STT complicate and sometimes neutralize traffic processing by middleboxes, such as firewalls, SSL accelerators, etc. Without implementing encapsulation specific capabilities or deploying translation gateways, middleboxes are likely to drop encapsulated traffic. is more middlebox-friendly as it is only required to obtain (and cache) the original header fields in the TCP header from a pre-configured database accessible by every switch, hypervisor, or middlebox. Additional benefit can come from combining with novel middlebox technologies like FlowTags [11], e.g. by reusing FIT to deliver session specific information to FlowTags enabled middleboxes. Additional SW optimization approach for overlay network virtualization was proposed by Kawashima et al. in[1], and evaluated in [15]. Based on address translation, this solution maps VM MAC addresses to MAC addresses of the physical hosts, and performs MAC address replacement using Open- Flow protocol and centralized OpenFlow controller. The authors report performance improvement of almost four times, as compared to VxLAN and GRE approaches. Compared to, this approach is limited as it is scoped to a single L2 domain, while is based on L3 address translation providing more flexibility. Additional advantages of over other known software solutions are its selective and dynamic applicability and its ability to coexist seamlessly with overlay processing. These capabilities open up additional opportunities for network wide decision making and optimization, and ease the adoption.

6 Latency [µsec] Figure 5: Latency 6. DISCUSSION AND FUTURE WORK In this section we briefly survey features that open up further research questions or have a potential to strengthen the solution itself and the environment where it is deployed, if thoroughly investigated. Control plane complexity. Due to space limitations, we did not discuss the control plane aspects of at length and only have mentioned that in, the improvement in dataplane performance comes at the cost of added control plane complexity. Acknowledging this, we envision extension in two complimentary directions. First, improve scale by smart distributed data management techniques. Second, extend to interact with DC-wide operations to scope the application of to where it is most beneficial. Security. Security implications of must be further investigated, even if its DCN scope limits the potential attack surface. DoS attack on FIT allocator seems as an immediate threat requiring attention. IP fragmentation support. Although rare in DC environments, non-uniform MTU settings along the communication path can potentially cause the packets to be fragmented in transfer. This can break the communications since, in the current design, destination NVE might not always be able to retrieve the FIT from the destination TCP port and will have to drop VM packets. Adding fragmentation support (or avoidance) is an important improvement direction. Service chaining support. The ability of network virtualization solution to co-exist with different types of service appliances and ways they are used, is one of the strongest metrics. This includes dedicated hardware appliances, virtual appliances (NFV), distributed network services, and service chains. Although we did not explore this fully yet (see also Section 5), we observe that has relative advantage as it avoids decapsulating and re-encapsulating packets at each hop along the service chain. Cost vs usability trade-off. is a SW only, highly efficient add-on to edge based overlay network virtualization. A possible alternative to edge based overlay network virtualization is to design, build, and deploy the infrastructure based network virtualization (differentiate from offloading or accelerating SW-based approached with HW as described in Section 5). While highly attractive in terms of performance, infrastructure based solutions are tightly coupled with HW, leading to higher costs, slower innovation cycles, and a risk of vendor lock-in. It is therefore worth exploring HW-assists for edge-based, often SW solutions, where presents strong immediate business value for data centers and allows, as prices drop, to smooth the transition from current, overlay unaware, networking gear to a new equipment generation. 7. ACKNOWLEDGEMENTS The research leading to results published in this paper is partially supported by the European Communitys Seventh Framework Programme (FP7/ ), grant agreement number , in the context of the COSIGN Project.

7 8. REFERENCES [1] KVM: Kernel Virtual Machine. [2] libvirt: The virtualization API. [3] Mellanox ConnectX-3 Pro. adapter_cards/pb_connectx-3_pro_card_en.pdf. [] Open vswitch. [5] Openstack neutron. [6] Ryu. [7] N. M. K. Chowdhury and R. Boutaba. A survey of network virtualization. Comput. Netw., 5(5): , Apr [8] R. Cohen, K. Barabash, and L. Schour. Distributed overlay virtual ethernet (dove) integration with openstack. In IM, pages , [9] R. Cohen and et al. An intent-based approach for network virtualization. In 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), Ghent, Belgium, May 27-31, 2013, pages 2 50, [10] B. Davie and J. Gross. A stateless transport tunneling protocol for network virtualization (stt). IETF draft, [11] S. K. Fayazbakhsh, V. Sekar, M. Yu, and J. C. Mogul. Flowtags: Enforcing network-wide policies in the presence of dynamic middlebox actions. In Proceedings of the Second ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking, HotSDN 13, pages 19 2, New York, NY, USA, ACM. [12] A. Greenberg, J. Hamilton, D. A. Maltz, and P. Patel. The cost of a cloud: Research problems in data center networks. SIGCOMM Comput. Commun. Rev., 39(1):68 73, Dec [13] J. Gross, T. Sridhar, P. Garg, C. Wright, and I. Ganga. Geneve: Generic network virtualization encapsulation. IETF draft, 201. [1] R. Kawashima and H. Matsuo. Non-tunneling edge-overlay model using openflow for cloud datacenter networks. In Cloud Computing Technology and Science (CloudCom), 2013 IEEE 5th International Conference on, volume 2, pages IEEE, [15] R. Kawashima and H. Matsuo. Performance evaluation of non-tunneling edge-overlay model on 0gbe environment. In Network Cloud Computing and Applications (NCCA), 201 IEEE 3rd Symposium on, pages IEEE, 201. [16] M. Lasserre, F. Balus, T. Morin, N. Bitar, and Y. Rekhter. Framework for dc network virtualization. IETF draft, 201. [17] T. Sridhar, L. Kreeger, D. Dutt, C. Wright, M. Bursell, M. Mahalingam, P. Agarwal, and K. Duda. Vxlan: A framework for overlaying virtualized layer 2 networks over layer 3 networks. IETF draft, 201. [18] M. Sridharan, K. Duda, I. Ganga, A. Greenberg, G. Lin, M. Pearson, P. Thaler, C. Tumuluri, N. Venkataramiah, and Y. Wang. Nvgre: Network virtualization using generic routing encapsulation. IETF draft, 2011.