STATE OF THE ART OF DATA CENTRE NETWORK TECHNOLOGIES CASE: COMPARISON BETWEEN ETHERNET FABRIC SOLUTIONS

Transcription

1 STATE OF THE ART OF DATA CENTRE NETWORK TECHNOLOGIES CASE: COMPARISON BETWEEN ETHERNET FABRIC SOLUTIONS Supervisor: Prof. Jukka Manner Instructor: Lic.Sc. (Tech) Markus Peuhkuri Francesco Maestrelli 17 November 2011

2 Agenda Data Centre to Finland (DC2F) Master s thesis goals A three-tier network Typical technologies Issues in modern data centres Modern two-tier networks Ethernet fabrics Conclusions Future work

3 Data Centre to Finland (DC2F) In the recent years Finland has emerged as a strategic location where to build new data centres. Energy efficiency is the common driver of all these new installations. DC2F is a research project at Aalto University involving three different departments: Department of Communications and Networking Department of Electrical Engineering Department of Civil and Structural Engineering Comnet s goal is to study the impact of the network and the networking equipment on the energy consumption of a data centre.

4 Master s thesis goals Explore the available solutions in the data centre field to set the basis for the realization of a test data centre which is currently being realized. Understand what is the state of the art of data centre networks Standard solutions Typical problems faced by network designers Trends and future technologies Compare different Ethernet fabric solutions to better understand whether they differ and how.

5 A three-tier network Typically data centres present a subdivision in three tiers Core Aggregation Access Services are typically provided at the Aggregation layer using service modules or dedicated servers. Looped topologies are used to allow redundancy of the connections. Bandwidth oversubscription between layers.

6 Typical technologies Link Aggregation Group: allows bundling together up to 16 links of the same capacity into a logical one. M-LAG: Allows the terminating a LAG on two separate physical devices. Virtual Chassis: Allows linking together two or more switches so that they can act as a single one. Virtual Chassis by Juniper Cisco Fabric extenders Nexus 2000 series Virtualization of data centre resources. Servers are typically virtualized so to maximize the utilization of the physical servers rearranging as needed the VMs. Virtual Ethernet Port Aggregator (VEPA) Port Profile Migration

7 Issues in modern data centres STP is inadequate Halves the network capacity, Too high re-convergence time upon failure, MSTP is too cumbersome to maintain, Non-optimal forwarding. Virtual Machine mobility imposes L2 adjacency IP and MAC addresses should remain the same to ease VM mobility. Need to increase east-west bandwidth. Flattening the data centre networks With the advent of virtualization, 10GbE and faster connections, unified fabrics based on lossless Ethernet, and other technological innovations, data centre networking vendors started proposing two-tier topologies. Ethernet fabrics

8 Modern two-tier networks Access layer and Aggregation layer are merged together. Virtual chassis provides flexibility as needed. Spanning Tree Protocol is no longer needed. Troubleshooting the network might be more challenging

9 Ethernet fabrics Motivations Increase east-west bandwidth Reduce latency by eliminating layers Necessity of utilizing multiple paths simultaneously Abandoning STP providing resiliency and redundancy Scale to large L2 domains

10 Ethernet fabrics Characteristics Utilize high-capacity and non-blocking switching nodes Adapt L3 routing protocols to be utilized at L2 and forward frames on the shortest path between nodes Provide the network administrators better tools to troubleshoot L2 networks Maintain L2 plug-and-play capabilities VMs can move preserving the IP and MAC address Generally based on a so called Clos or fat-tree topology avoid oversubscription guaranteeing a non-blocking switch fabric Topology independent in practice (few exceptions) Eliminate the need to utilize STP maintaining the interoperability with legacy networks

11 Ethernet fabrics Increased network flexibility Simplified VMs mobility thanks to the ability to extend VLANs anywhere is required Resiliency is guaranteed by the many available equal cost paths TRILL and SPB Concurrent standards by IETF and IEEE. Standardization boards do not agree on how to implement the fabric. TRILL has a proprietary header, while SPB uses Provider Backbone Bridge (PBB). SPB-VID and SPB-MAC are the two available alternatives for SPB TRILL has currently no OA&M features, while SPB is compatible with 802.1ag Connectivity Fault Management, 802.3ah and Y.1731 Ethernet OA&M. TRILL and SBP differs in how multicast traffic is handled. TRILL uses only a limited set of distribution trees.

12 Ethernet fabrics Cisco is proposing a TRILL-like technology called FabricPath It is not compatible with TRILL. A set of extensions such as vpc+, multiple topologies, active/active HSRP are available only in FP mode. It uses Conversational MAC learning This reduces the amount of MAC addresses that the access switches have to learn. MAC addresses are not learnt from flooded frames Performances evaluated by NetworkWorkd FP proved to use the available ECMPs in a fair way exhibiting a variation of the load distributed between the equal cost paths of only 0.07% or less. Convergence time while adding new switches to the network is zero. FabricPath converges on average, in case of a link failure, in 162 milliseconds.

13 Ethernet fabrics Brocade is proposing a solution really close to TRILL called VCS It adopts TRILL data plane, so the frames are structured as in TRILL It uses a different control plane since the routing protocol used in this case is Fabric Shortest Path First (FSPF) typically utilized in Fibre Channel fabrics. FSPF only defines and implements point-to-point links. No designated router concept. Brocade is committed to be TRILL compliant once the standard will be approved Only a software update is needed to implement a new routing protocol

14 Ethernet fabrics Juniper is proposing a totally different solution named QFabric It s a proprietary technology that is part of Juniper s Data Centre Network Architecture strategy. QFabric creates the illusion of a singles virtual switch supervised by a management platform that gives one view of the fabric. Evolution of the chassis switch architecture Line cards -> QF/Nodes Backplane -> QF/Interconnects Route engine/ Supervisor card -> QF/Director 40 GbE connections Latency < 5µs

15 Ethernet fabrics From the research field the most interesting idea that emerged in the past years is the realization of a Distributed Hash Table (DHT) to store associations between IP and MAC addresses in switches and eliminate the broadcast traffic generated by ARP requests. Originally introduced in SEATTLE where also the location of the hosts is stored in switches using a DHT Consistent hashing is used to identify where to store tuples of values. Special hashing mechanism where when the number of bins changes only K/n keys need to be remapped. (n = # of bins, K= tuples of values) on average. Can be adapted to be used also with other technology Nuutti Varis and Jukka Manner from the Department of Communications and Networking at Aalto University proposed a way to introduce a DHT in TRILL

16 Conclusions The future of data center network is flat. Two-tier networks, Ethernet fabrics Resource virtualization and consolidation Currently serious risk of vendor lock-in Currently impossible to determine which solution is better. It all depends on how the different vendors will be able to market their solutions. Impossible to test all the solutions proposed by the different vendors, hence the comparison was done interviewing experts form different companies. At the moment TRILL seems to be the most widely adopted solution. Most likely small data centre such as the ones available in Finland will not need, at least in the near future, to adopt Ethernet fabric solutions that seem to be cost effective only for extremely big data centres (tens of thousands of servers/virtual machines)

17 Future Work DC2F is currently entering in its implementation phase. A test data center is being built in Otakaari 5 Test different network configurations such as M-LAG and virtual chassis Test, if possible, FabricPath. At the moment it is available only for Nexus 7000 Series switches, but it should be released also fo the Nexus 5000 Series switches that we have. Investigate if power consumption of the switches is related to the network technology adopted. Currently it seems that newer switches are more energy efficient but because of the hardware they utse and not because of the protocols they support. Much more...

18 THANK YOU! QUESTIONS? Francesco Maestrelli