WHITE PAPER www.brocade.com DATA CENTER Scalable Approaches for Multitenant Cloud Data Centers
Brocade VCS Fabric technology is the ideal Ethernet infrastructure for cloud computing. It is manageable, flexible, scalable, and secure. VCS Fabric technology gives you a choice of network models Layer 2, Layer 3, single-fabric, or multifabric and Brocade helps you choose the right model for your organization. VCS fabrics integrate easily with legacy networks, while letting you take full advantage of innovative technology, such as Software-Defined Networking (SDN) and Virtual Extensive LAN (VXLAN). INTRODUCTION A cloud environment requires a high-performance networking foundation that can support tens of thousands of servers and hundreds of thousands of Virtual Machines (VMs). Flexibility is critical, because large-scale server virtualization, particularly VM mobility, creates unpredictable bandwidth requirements. Virtualization and Big Data analytics are generating massive levels of East-West traffic between servers that existing network architectures are not designed to support. Brocade VCS Fabric technology is designed for the present and future of data center networking. Ethernet fabrics based on VCS Fabric technology improve network utilization, maximize application availability, increase scalability, and dramatically simplify the network architecture for next-generation, private, and public cloud data centers. The Brocade VCS Fabric architecture is designed for the highest level of functionality and investment protection, making it the perfect building block for a virtualized data center. This paper presents deployment options for VCS fabrics that support scalable cloud network architectures, including innovative Layer 2 and Layer 3 topologies that use singlefabric and multifabric architectures. 2
VCS FABRIC TECHNOLOGY DEPLOYMENT MODEL Brocade VCS Fabric technology provides single-fabric and multifabric designs with native multitenancy, which address the requirements for managing and orchestrating a rapidly increasing number of network domains. The innovative Virtual Fabric (VF) feature provides built-in network multitenancy that overcomes the scalability limitations of traditional VLANs while ensuring secure isolation between tenants. VCS Fabric technology deployment models support a range of requirements, from hundreds of server-facing 10 Gigabit Ethernet (GbE) ports with thousands of VMs to 100,000 ports and one million VMs. A single Ethernet fabric based on Brocade VCS Fabric technology supports up to 48 Brocade VDX switches with up to 10,000 server-facing ports and 100,000 VMs. Management is simplified, because all the switches in a VCS fabric can be managed as a single Logical Chassis. Further, VCS fabrics support native, highly-efficient multipathing at Layers 1, 2, and 3 for higher network utilization, improved resiliency, and fabric-wide VM mobility. Scale-Out Fabric Multifabric Fabric with Overlay VCS Layer 3 BGP or OSPF VCS VCS VCS Layer 3 BGP or OSPF VCS VCS VCS 1K+ Server-Facing Ports (10K+ VMs) 10K+ Server-Facing Ports (100K+ VMs) 100K+ Server-Facing Ports (1M+ VMs) Figure 1. VCS fabrics offer seamless scalability to over 100,000 server ports and 1 million VMs. Brocade VCS Fabric technology also offers multifabric deployment options, using native Layer 2 or Layer 3 technology, for enhancing scalability to more than 10,000 server-facing ports and over 100,000 VMs. Multifabric deployments offer an enlarged VM mobility domain across data centers, while the failure domain is restricted to individual fabrics for resiliency. Ethernet fabrics based on Brocade VCS Fabric technology also support software-based overlay networks for more than one million VMs. VCS Fabric technology provides a multifabric model with overlay that allows seamless integration between physical and virtual compute resources across multiple VCS fabrics in different geographic regions. In addition, you can deploy Brocade VDX switches using a massively scalable Layer 3 Clos architecture without VCS Fabric technology. In this deployment model, the Brocade VDX VXLAN Tunnel Endpoint (VTEP) feature provides multitenancy and Layer 2 service across server racks. LAYER 2 AND LAYER 3 SCALE-OUT OPTIONS Brocade VCS Fabric technology provides Layer 2 and Layer 3 scale-out options, allowing a single VCS fabric to be mapped to a pod that serves as a module within the data center cloud. Each pod can be managed as a single switch and serves as a blueprint for incremental, modular network deployment. 3
Brocade VCS Fabric technology supports Layer 2 two-tier leaf/spine network designs, for enhanced scalability. A two-tier Layer 2 network design uses spine switches at the top tier and leaf switches at the bottom tier. Servers, compute nodes, and storage nodes are attached to leaf switches at the top of every rack, and leaf switches are each connected to two or more spine switches. The leaf/spine network of VCS Fabric technology designs enable horizontal scale-out, with the number of spine switches growing linearly as the number of leaf switches increases. This design can support up to eight active spine switches, for maximum scalability. Brocade VCS Fabric technology supports Equal-Cost Multipath (ECMP) at Layer 2, which distributes traffic load within a Layer 2 scale-out VCS fabric among the multiple equal-cost paths in a resilient Layer 2 leaf/spine network design. External Figure 2. Scale-out Ethernet fabrics based on Brocade VCS Fabric technology. Spine Leaf Scale-Out Ethernet Fabrics with a Layer 3 Spine Brocade VCS Fabric technology also provides standards-based options for leaf/spine fabrics with a Layer 3 spine. This design localizes the Layer 2/Layer 3 gateway for subnets within the Ethernet fabric, providing consistent latency, high cross-sectional bandwidth, link resiliency, and VM mobility. Brocade VCS Fabric technology supports the Virtual Routing Redundancy Protocol- Extended (VRRP-E) protocol, allowing up to eight active/active Layer 3 gateways in a single Layer 3 spine network that is based on VCS Fabric technology. This option helps improve scalability and increase bandwidth, and It optimizes the network for East-West traffic and VM migration. Layer 2/Layer 3 leaf-spine fabrics based on VCS Fabric technology also support scalable VRF-Lite, which provides a standard way of isolating tenants at Layer 3. At the same time, VRF-Lite allows centralized management for multitenancy, which greatly reduces operational overhead. 4
External Spine Leaf Tenant-1 VRF Tenant-2 Tenant-3 Tenant-4 Figure 3. VCS fabrics with Layer 3 spine. The VF feature works with VRF-Lite to extend multitenancy using Layer 3 mechanisms based on VCS Fabric technology. For example, managed hosting providers who are transitioning to full-scale cloud services can manage the migration without disrupting existing client traffic. VF technology in combination with VRF-Lite provides a straightforward means to achieving this transition, one rack at a time, if desired. SCALABILITY AND FLEXIBILITY FOR MULTIFABRIC DEPLOYMENTS This section provides an overview of Layer 2 as well as Layer 3 spine-centric architectures for multifabric deployments that can scale to over 10,000 server ports and over 100,000 VMs. Multifabric deployments of VCS fabrics allow massive scalability while improving flexibility. Multifabric Architecture: Multiple Layer 2 Access Fabrics with Layer 3 Aggregation Fabric A Layer 3 aggregation fabric based on VCS Fabric technology can interconnect multiple Layer 2 leaf/spine access fabrics across geographic locations, to improve scalability and flexibility. In this multifabric architecture, a single Layer 3 aggregation fabric based on VCS Fabric technology can span Layer 2 access fabrics in data centers in different geographic locations. This approach greatly improves scalability for a hierarchical network, while protecting core switches from overloading. Redundant Virtual Link Aggregation Groups (vlags) connect the spine switches in the Layer 2 access fabrics based on VCS Fabric technology to the Layer 3 aggregation fabric. The aggregation fabric can include up to eight active/active switches for link resiliency and high availability (see Figure 4). A multifabric design based on VCS Fabric technology improves elasticity and scalability, while enabling phased fabric deployment across data centers. Further, this design expands the VM mobility domain across all VCS fabrics at the access layer. 5
External VF 6000 OSPF/BGP VF 5000 Aggregation Fabric Figure 4. Using Layer 2 access fabrics with a Layer 3 aggregation fabric based on VCS Fabric technology. vlag Spine Multifabric Architecture with Layer 2/Layer 3 Access Fabrics and Layer 3 Aggregation Fabric The Virtual Fabric Extension (VF Extension) feature in Layer 3 spine switches provides enhanced scalability for multifabric networks based on VCS Fabric technology. In this design, each Layer 2/Layer 3 access fabric uses Layer 3 spine switches connected to an aggregation fabric based on VCS Fabric technology (see Figure 5). VF Extension enables specific Layer 2 domains to be selectively extended across data center boundaries. External VF 6000 OSPF/BGP VF 5000 Aggregation Fabric Figure 5. Multifabric architecture based on Brocade VCS Fabric technology with Layer 3 access fabrics and VF extension. Spine With this solution, the VM mobility domain can span different Layer 2/Layer 3 access fabrics. This allows greater scalability for multitenancy and optimizes flexibility, because tenant resources can span Layer 2 domains across data centers. VF Extension uses industry-standard VXLAN encapsulation to support a flexible, largescale multitenant environment over a shared common physical infrastructure. VXLAN is a Layer 2 overlay on a Layer 3 network. It uses MAC Address-in-User Datagram Protocol (MAC-in-UDP) encapsulation to extend Layer 2 segments across the data center network. 6
NETWORK VIRTUALIZATION CAPABILITIES FOR HYPERSCALE DATA CENTERS For hyperscale data centers with over 100,000 server-facing ports and over one million VMs, Brocade offers a network architecture that leverages multifabric VCS fabrics as a foundation and integrates with the VMware NSX controller system to provide network virtualization capabilities. By leveraging Brocade VCS Fabric technology, VXLAN supports the creation of large numbers of virtual domains above existing networks. This enables organizations to efficiently use their current infrastructure while leveraging the benefits of VXLAN to support multitenancy and large-scale deployment of applications and VMs. External VMware NSX Controller VF 6000 OSPF/BGP VF 5000 Aggregation Fabric Spine Figure 6. Brocade VCS Fabric technology with overlay for hyperscale data centers. By unifying physical and virtual networking, VMware NSX and Brocade VCS Gateway enable seamless connectivity between virtual and nonvirtual workloads, providing data center operators a unified network operations model for all application types. SUMMARY As public and private cloud deployments become more common, efficiently and securely supporting virtualized workloads becomes an increasingly critical requirement. Brocade provides scalable approaches for multitenant cloud data centers that are optimized for different deployments, including multitier and Layer 3 architectures, and data centers spanning geographic locations. In many cases, these techniques can be complementary, with handoffs between Layer 2 and Layer 3 at strategic points in the network, to optimize the overall Infrastructure. Brocade VCS Fabric technology lets you combine a variety of effective techniques, including Layer 3 and Layer 2 separation within a single VCS fabric, multiple fabrics that use the innovative VF Extension feature, and a network virtualization architecture with VMware NSX controller integration. For more information about Brocade solutions, visit www.brocade.com. ABOUT BROCADE Brocade networking solutions help organizations achieve their critical business initiatives as they transition to a world where applications and information reside anywhere. Today, Brocade is extending its proven data center expertise across the entire network with open, virtual, and efficient solutions built for consolidation, virtualization, and cloud computing. Learn more at www.brocade.com. 7
WHITE PAPER www.brocade.com Corporate Headquarters San Jose, CA USA T: +1-408-333-8000 info@brocade.com European Headquarters Geneva, Switzerland T: +41-22-799-56-40 emea-info@brocade.com Asia Pacific Headquarters Singapore T: +65-6538-4700 apac-info@brocade.com 2014 Brocade Communications Systems, Inc. All Rights Reserved. 12/14 GA-WP-1891-00 ADX, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript, VCS, VDX, and Vyatta are registered trademarks, and The Effortless Network and The On-Demand Data Center are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may be trademarks of others. Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government.