Switching Fabric Designs for Data Centers David Klebanov

Switching Fabric Designs for Data Centers David Klebanov Technical Solutions Architect, Cisco Systems klebanov@cisco.com @DavidKlebanov 1

Agenda Data Center Fabric Design Principles and Industry Trends Data Center Fabric Design Evolution and Main Characteristics Extending Data Center Fabric 2

Data Center Fabric Design Principles Industry Trends High scalability and high throughput - Flexible deployment models using 10Gb, 40Gb and 100Gb speeds High availability, resilience and low fate sharing - No domino effect Low and predictable latency/response time - Ultra low latency for HFT and Financials Support for concurrent diverse workloads - Virtualized and non-virtualized Workload mobility - Resource based workload placement 3

Data Center Fabric Design Principles Industry Trends Segmentation, isolation and multi-tenancy - Cloud and non-cloud Visibility and Manageability - Management simplicity and infrastructure transparency Automation and Provisioning - Infrastructure provisioning workflows, APIs and SDN Disaster Recovery - Geo resiliency Evolve physical facilities with network architecture - Green architecture 4

Life Used to Be Easy Layer 3 Layer 2 Routed Domain Aggregation Aggregation Spanning Tree Access Servers POD 1 Core Spanning Tree Access Servers POD n Layer 3 Layer 2 Multilayer Design POD - Access, Aggregation and Core Layer2/Layer3 boundary at the Aggregation Layer - VLAN scope Spanning Tree driven - Special features to keep topology stable Routed interconnect between the PODs - OSPF, EIGRP 5

Or was it Wasted North- South Bandwidth 50% with active/standby links Aggregation Core Aggregation Rigid management framework Power and cooling hoggers Lots of cross-rack server cabling Difficult to troubleshoot Many protocol extensions Prone to outages Layer 2 loops Access Servers Access Servers Limited crosssectional East-West Bandwidth Limited application mobility scope (server virtualization) 6

Multi-Chassis Solution EtherChannel Layer 3 Core Layer 3 Core Aggregation Control Plane Aggregation Control Plane Aggregation Control Plane Data Plane Data Plane Chassis #1 Chassis #2 Data Plane Data Plane Chassis #1 Chassis #2 Access EtherChannels to simplify STP topology Access Servers Servers 8

Fabric Solution Clos Fabric Middle Ingress Egress Ingress Middle Egress http://en.wikipedia.org/wiki/clos_network Formalized by Charles Clos in 1953 Multistage fabric Leaf Nodes - Ingress/Egress stage - End host (server) connectivity Spine Nodes - Middle stage - Interconnect between ingress and egress stage fabric nodes 9

Clos Fabric Main Characteristics Spine Path Diversity Fabric No Spanning Tree Routing protocol driven topology L1 L2 L3 Ln Leaf Equal Cost MultiPathing L1 L2 L3 Ln Cut-through switching Predictable latency Efficient hardware table programming Layer 2 or Layer 3 10

Understanding Port Extension Extending the Fabric Edge Eth 1 Eth 2 Eth 3 Eth 4 Eth 48 Control Bridge Eth Extended Port Edge of Port Extension the Fabric (802.1BR) Port Extender Port Port Port Port Port 1 2 3 4 48 Physical Port Servers Multiple port extenders per control bridge All configurations are carried out on the Control Bridge 12

Benefits of Port Extension Spine Logical Modular Chassis Supervisor Supervisor Fabric Leaf Leaf Switch Fabric Switch Fabric Line Card Line Card Line Card Fabric Links Servers Extend Fabric capacity without extending the management domain Top of rack scalable server cabling Leaf Nodes Port Extenders Port Extenders Port Extenders 13

Cisco Unified Data Center Fabric Leading with Innovation Portfolio Nexus 7K MDS 9000 Nexus 6K Nexus 2K Nexus 3K Nexus 5K Nexus 1K UCS Unified Fabric Financial MSDC/Cloud Service Provider Enterprise Low Latency Scalability Agile Service Delivery Private Cloud Automation 14

Thank you.