Solving Scale and Mobility in the Data Center A New Simplified Approach Table of Contents Best Practice Data Center Design... 2 Traffic Flows, multi-tenancy and provisioning... 3 Edge device auto-attachment.4 All the Benefits without the Complexity... 5 Inspired by designs applied in hyper-scale data centers, the broader data center market has changed dramatically. Enterprises as well as data center operators are starting to get access to the same hardware technology that hyper-scale operators have; however, the overall solutions associated with this hardware are burdened with complexity and costs that only the large operators can bear. The Avaya Fabric Connect Data Center solution is squarely aimed at providing enterprises that cannot afford to have their own army of PhD s with the best of both worlds, thus providing customers with the ability to 1 : Ride the ASIC wave (Moore s law) the same way as the large operators, thus benefitting from the ever increasing capacity available in emerging merchant silicon and fixed form factor switches in use in data centers. This leads to reductions in power and space needs and reduced capital expenditure Benefit from designs that allow full scale-out capabilities such as the designs developed in programs like Open Compute and other industry best practice initiatives resulting in optimized performance and infrastructure cost Exploit all of the benefits of hyper scale designs and more using Avaya Fabric Connect. This will provide a fully automated, service provider grade and secure multi-tenant core network infrastructure combined with edge only provisioning and edge automation as well as a single architecture spanning data-center to campus. This can lead to improved time to service, reduced complexity and significantly lower operational cost while increasing business agility The Avaya Fabric Connect Data Center solution offers an industry leading, scalable and cost effective solution with minimal operational cost impact. avaya.com 1
Best Practice Data Center Design The best practice design provided by Avaya is based upon the latest designs and developments that have originated in the hyper-scale datacenters and are finding their way into large data centers across the globe. These designs are based on fixed form factor switch designs, offering reduced space and power requirements and improved scalability (scale-out). This design methodology is reinforced by the fact that the portfolios of the major Original Device Manufacturers (ODM s) who supply the hyper-scale data-center operators, the current and emerging designs of the major merchant silicon suppliers and switch reference designs, are based on efforts from engagements with hyper scale players and industry initiatives such as Open Compute 2 and others. Avaya s best practice design leverages these experiences and developments and merges them with the Avaya Fabric capabilities to a create data center solution with unprecedented scalability, effectiveness and efficiency. Figure 1: Conceptual Datacenter Design The design is based upon the concept of Pod s comprised of 48-port top of rack switches (ToR s), offering up to 10 Gbps per server and interconnected East-West with 40 Gbps. The East-West interconnects at the ToR level, in conjunction with the Avaya Fabric capabilities, are only possible because of the Avaya Fabric s topological flexibility and reduce traffic to the Spine significantly, since data center traffic is > 80% East-West and in most cases between adjacent racks. The spine is connected with up to n x 10 Gbps (or alternatively n x 40 Gbps) to different spine switches to provide route diversity and an oversubscription ratio (ToR: spine capacity) ranging from 1:12 through 1:3 depending on the choice of links 3. Pods can be added as needed and support the scalability of the architecture. The spine-to-spine plane connectivity can be designed as needed to be nonoversubscribed (1:1) to a desired / acceptable overbooking ratio. The spine avaya.com 2
plane switches finally connect out to the core network switches to provide connectivity into and out of the data center. Table 1 shows a few different configuration examples exemplifying the inherent scalability of the configuration and fully loading available switch configurations spine switches usually having port counts of 32 or 64 ports. Example # Pod # ToR s per Pod ToR Capacity Output Ratio #Spine s per Pod Spine Capacity Spine Output ports Ratio Plane Ports Plane Output ports #1 8 24 480 4 x 10 1:12 4 24 x 10 8 x 40 1:4 8 8 1:1 #2 16 16 480 4 x 40 1:3 4 16 x 40 16 x 40 1:1 16 16 1:1 #3 32 32 480 4 x 40 1:3 4 32 x 40 32 x 40 1:1 32 32 1:1 Ratio Table 1: Example possible configurations exemplifying scale-out capabilities As table 1 shows, by varying parameters such as the number of ToR s, the number of Pod s per data center and the number of spine switches as well as oversubscription ratios, a data center designer can make effective use of the physical capacity available and design around particular needs and traffic flows. The table also exemplifies the scale-out capabilities inherent to the design. Traffic Flows, multi-tenancy and provisioning Having covered the physical connectivity and dimensioning, traffic flows and provisioning are areas of significance in modern data centers. With the Avaya Fabric offering route diversity between equal cost paths 4 as well as shortest path capabilities, complex BGP designs are no longer required the Avaya Fabric offers that capability out of the box with unmatched simplicity. The Fabric offers most effective L2 traffic flows and with L3 shortcuts or distributed virtual routing (DvR) 5 L3 traffic will equally follow the shortest path. And given that we have also connected the ToR s East-West with 40 Gbps the majority of the L2/3 traffic does not even need to traverse the spine. The Avaya Fabric enables secure 6 and highly scalable multi-tenancy with 16 million service ID s (I-SID s) and an equal number of virtual service networks (VSN s). This offers data center designers the freedom to re-use VLAN s since they are only (port) local and are mapped onto VSN s and support overlapping MAC and IP address ranges on a per VSN basis. These capabilities offer true, scalable and secure multi-tenancy where worries of overlapping addressing and VLAN exhaustion are a thing of the past. Finally, provisioning is a key element of concern in any networking environment, adding a large portion to the operational expenditure (OpEx) and usually being the largest factor in the total cost of ownership (TCO)! Avaya s Fabric addresses this issue by removing the need for any core network provisioning since the core is fully automated and only requires operators to provision the edge ports. Avaya Fabric Attach technology further lowers TCO by providing full edge avaya.com 3
The best practice design provided by Avaya is based upon the latest designs and developments that have originated in the hyper-scale datacenters and are finding their way into large data centers across the globe. These designs are based on fixed form factor switch designs, offering reduced space and power requirements and automation. This allows a Fabric Attach (FA) capable device to attach to a desired virtual service network (VSN) automatically and the fabric automatically provisioning that VSN to the required edge port, as well as removing the configuration automatically once the device in question is moved or removed. The capabilities offered by the Avaya Fabric ensures optimum simplicity and stability, sub-second automated switchovers in case of failures, shortest and most effective path selection for both L2 and L3 in combination with equal cost multipath as well as exceptional service provider grade multi-tenancy and network virtualization capabilities, core network automation and edge provisioning only with the optional ability to automate edge attachment as well. Edge device auto-attachment Avaya has long realized that simplifying the core network and addressing core network automation, secure and scalable multi-tenancy and network virtualization are only part of the answer to broader networking and data center challenges. While these capabilities enable data center designers to build large and scalable multi-tenant data centers and reduce the effect of outages, nearly 50% which are a result of human error 7, provisioning the edge network manually is still unnecessary and costly in many environments. Figure 2 shows a simple example of a set of Fabric Attach (FA) capable virtual machines (VM). In the second diagram the configuration is mirrored and as a new FA cable VM 8 requests access to the same VSN, the VSN is extended automatically to the new VM. Figure 2: Moving or adding a VM automatically extends the VSN Similarly the VSN is revoked automatically if a VM is removed, removing operational complexity and operational cost and simplifying configuration management. Unlike traditional models, such as Avaya Virtual Provisioning Service (VPS) for VMware ESX that requires synchronization between VMware vcenter, VPS and the Avaya Fabric, Fabric Attach requires nothing more than the edge device (VM in this example) to signal the required mapping to a FA-Server running in the Fabric via an edge switch and attaching to the desired virtual service network (VSN). This capability provides a very cost effective and less complex means for devices, such as virtual machines, to attach to service networks as well as move. A VM move requires no interaction with any other system, the virtual service network (VSN) will simply follow the VM automatically! avaya.com 4
About Avaya Avaya is a leading, global provider of customer and team engagement solutions and services available in a variety of flexible on-premise and cloud deployment options. Avaya s fabric-based networking solutions help simplify and accelerate the deployment of business critical applications and services. For more information, please visit www.avaya.com. 1 Read also Rightsizing the Data Center Network The Move Toward Fixed Form Factor Core, Gartner, January 2015 2 http://www.opencompute.org 3 Note that compared to traditional leaf-spine designs the ToR East- West connectivity drastically reduces the amount of traffic having to be passed via the spine. Assuming 20% North-South traffic a typical design would need to support a maximum of 96 Gbps (with a 480 Gbps ToR), thus reducing the oversubscription to 1:2.4 by using a 4 x 10 Gbps spine only. Using 40 Gbps links would lead to overcapacity in the spine links with a ratio of 1:0.5. 4 2 BVID s in VOS 4.x, 4+ BVID s scheduled for VOS 5.x in 2H/CY 15 5 Scheduled for VOS 5.x in 2H/CY 15 6 For details regarding the secure traffic isolation and PCI compliance please refer to the Avaya Stealth Network Whitepaper 7 Yankee studies show that 48% of network outages are caused by human error 8 Any VM using OVS version 2.4 or higher with auto attach support (Xen, KVM. Hyper-V ports are available) 9 See published independent research proving that Avaya Fabric connect lowers operational cost and improves efficiency and availability All the Benefits without the Complexity Avaya Fabric based data center designs offer data center designers all the benefits of hyper scale without the complexity. The benefits include: The ability to deploy best practise scale-out designs, riding the ASIC wave by using fixed-form-factor devices and the latest merchant silicon for highest capacity and lowest cost while offering industry best practise scale out Optimized traffic flows though full topological flexibility and shortest path equal cost multipath flows, enabling the offload of a vast majority of East-West traffic directly at the Top of Rack, effective Layer 2 traffic flows and Layer 3 traffic flows using distributed virtual routing Very low levels of complexity through the avoidance of BGP, high-levels of resiliency and sub second switchovers in case of link or device failures A single (virtual) network architecture spanning multiple data centers, campus and edge offering a single set of capabilities end-to-end. Consequently reducing staff and training requirements as well as the need for any complex and costly overlay virtualization solutions Service provider grade network virtualization, scalability and monitoring capabilities, providing real-time insight into network performance Shortest possible time to service, significantly lower operational cost and highest availability 9 though full core automation, network virtualization, secure tenant separation and edge attach automation using Fabric Attach SDN ready though Avaya SDN Fx architecture The Avaya Fabric Connect based data center enables businesses to focus on what s important from a business perspective and reduce attention to mundane and costly day to day tasks while driving innovation and increasing business agility! 2015 Avaya Inc. All Rights Reserved. Avaya and the Avaya logo are trademarks of Avaya Inc. and are registered in the United States and other countries. All other trademarks identified by, TM, or SM are registered marks, trademarks, and service marks, respectively, of Avaya Inc. 05/15 DN7708-02 avaya.com 5