Architecting Data Center Networks in the era of Big Data and Cloud Spring Interop May 2012
Two approaches to DC Networking THE SAME OLD Centralized, Scale-up Layer 2 networks Monstrous chassis es TRILL OpenFlow VEPA SPB Or a Different Approach Distributed, Scale-out Layer 3 fabrics Efficient fixed es Open, industry standard protocols 2
Networks that suck for Cloud & Big Data Dist Core Network Topology Access PARTITIONED CAPACITY Capacity Topology Data center networks are in my way -James Hamilton, AWS 3
Networks that Don t suck for Cloud & Big Data Spine Network Topology Leaf UNIFORM CAPACITY Capacity Topology All points equidistant 4
Big Data TCP TCP Name Rack 1 Job Tracker Rack 2 Secondary NN Rack 3 Client Rack 4 TCP World Inverse Virtualization Workloads orchestrated like cattle or network. Does it matter? TCP Client Rack N 5
Basic requirements of Cloud (IaaS) Physical Network World FW LB Virtual Network Secure, Scalable Multi Tenancy Location independence On Demand virtual networks 6
Blend the Virtual and Physical Networks VLAN 20 VLAN 10 vswitch vswitch Host Host Tenant subnet = Network VLAN 7
Abstract the Virtual Network from Physical Segment ID 20 Segment ID 10 vswitch vswitch Host Host Network Virtualization Overlay Tenant subnet = Software VLAN 8
Scale-up centralized Layer 2 2-post Rooted Architecture Centralized / //ARP table scale? Scale w/ Bigger Boxes Precious Pets VLAN Provisioning? Broadcasts vswitch vswitch 9
Scale-out Layer 3 Leaf/Spine Fabric (16) (8) (2) (128) (64) (16) 1980 3072 6144 768 Server ports Mesh from Leaf to Spine OSPF, ISIS, BGP, TRILL ToR w/ 16 uplinks (ECMP) Non-blocking Spine 3:1 @ ToR 128 port 2RU Spine 10
Uniform fabric for Cloud & Big Data (16) (8) Hadoop (2) Storage Access (128) (64) Name Client Rack 1 Client Rack 2 (16) Job Tracker Secondary NN Database Rack 3 6144 Server ports Block I/O NAS Object Rack N vswitch vswitch 11 Global Brad Marketing Hedlund
Attaching Services & North/South (16) (8) (2) (128) (64) Name (16) Job Tracker Secondary NN LB LB Firewall Firewall x86 Gateways Client Rack 1 Client Rack N v v v v v v World 12 Global Brad Marketing Hedlund
Generic Logical Architecture 1 Overlay based Physical/Static FW Fabric World DC router FW NAT FW NAT LB LB Green Co. Big Data Orange Co. 13
Generic Logical Architecture 2 Overlay based Virtual/Mobile FW Overlay Gateway Fabric Pub DMZ World DC router FW NAT FW NAT LB LB Green Co. Big Data Orange Co. 14
Density: Fixed vs. Chassis 140 10G per RU @ Line Rate () 120 100 80 60 Chassis Fixed 40 20 0 2008 2010 2012 2014 15
Power: Fixed vs. Chassis 18 Max Watts / Line Rate 10G () 16 14 12 10 8 Chassis Fixed 6 4 2 0 2010 2012 2014 16
What are the Challenges? (128) (16) (8) (16) (2) BOOTH DEMO Dell Fabric Manager Validate deployment Automate fabric configuration Monitoring & Operations Deployment & Cabling Configuration & Policy Monitor & Troubleshoot Layer 2 (TRILL?) Design Best Practices Dell Fabric Manager 17 Global Brad Marketing Hedlund
Webinar: CLOS Fabrics Explained DISCOUNT CODE: BradInterop http://closfabric.eventbrite.com/ Wednesday, June 20, 2012 from 10:00 AM to 1:00 PM (ET) HOST CO-HOST Ivan Peplenjak Yours Truly 18 Global Brad Marketing Hedlund
The power to do more Visit us at booth #1727 19
Generic Logical Architecture 3 No Overlays TRILL based Virtual/Mobile FW Fabric TRILL Pub DMZ World DC router FW NAT FW NAT LB LB Green Co. Big Data Orange Co. 20
Three Stage Layer 3 Leaf/Spine Fabric (64) (8) (2) /26 (128) 0/0 (512) /26 /26 0/0 24,576 Server ports Non-blocking @ top tiers Default route @ ToR & Leaf Leaf+ToR mesh groups ~8usec worst case 21
8192 non-blocking Fabric (64) (8) (2) (128) 8192 non-blocking 22
4096 non-blocking Fabric (32) (2) (64) 4096 non-blocking 23
2048 non-blocking Fabric (16) (2) (32) 2048 non-blocking 24
256 non-blocking Fabric (2) (4) 256 non-blocking 25
The case for 40G QSFP ports 10G 10G 10G 10G 10G 10G 10G 10G QSFP $1,800 VS SFP+ SFP+ SFP+ SFP+ $1K $1K $1K $1K 32 ToR 26 $230K $512K