Using Network Virtualization to Scale Data Centers Synopsys Santa Clara, CA USA November 2014 1
About Synopsys FY 2014 (Target) $2.055-2.065B* 9,225 Employees ~4,911 Masters / PhD Degrees ~2,248 Patents Granted Second largest semiconductor IP vendor** #1 supplier of interface, analog, memories & physical IP** More than 2100 IP engineers Best-in-class tools and IP * These targets were provided by Synopsys as of August 20, 2014, and are not being updated at this time **Source: Gartner, April 2014
Data Center Virtualization Cloud computing drives Data Center virtualization Any Application on any server using VMs Server Virtualization increases demand on Data Center network infrastructure Leaf-spine architecture- Increasing volume of higher speed links Classic Tiered DC Distributed fabric spine switches ToR/Leaf switches Increased East-West VM-VM traffic
Scaling the Data Center Multi-tenancy infrastructure: scalable pools of HW & SW improve DC utilization Speed Migration on Servers Total Market* Provisioning is Virtual Challenges: scaling, provisioning, VM migration, network management & SDN IP subnets limit VM mobility *Source: Speeds in Data Centers, Alan Weckel, Dell Oro Group,October 2014
VXLAN Network Overlay Framework to use VXLAN layer 2 tunneling to move VMs to any server in the data center VTEPs implemented in Hypervisor or VXLAN capable switch HW Eases tunneling scheme using 24b VXLAN Network Identifier (VNI) compared to MPLS
What is VXLAN: Virtual extensible LAN? IETF Informational RFC Overlay network carries MAC traffic from individual VMs in an encapsulated format over a logical "tunnel Extends layer 2 subnets across layer 3 networks Extends 4094 VLAN limit to 16M VXLAN segments 24b VXAN Network Identifier Overcomes Spanning tree limits Target Application Intra data center / Cloud computing where # of VMs can be > 4K Open vswitch (OVS): Running in Hypervisor Many vendors adding HW support to ensure stateless offloads work for VXLAN tunneled packets SW products support existing specs Source: IETF http://www.rfc-editor.org/rfc/rfc7348.txt
Why VXLAN? Problems with VLAN Two obvious ones Can t go beyond 4094 segments STP based VLANs further reduces the number Compared to VLAN Provides same broadcast segmentation as VLANs to end systems Other Limitations In a cloud, some times need to allocate multiple VLANs per tenant No cross pod expansion/scalability (due to expansion a user may need require VMs from other pods) More VMs => Larger MAC address tables per switch Other options NVGRE, Generic Network Virtualization Encapsulation (Geneve)
ASSPs Utilizing VXLAN Over Typically implemented in 28nm or 16/14nm SDN-enabled switch ASSPs Multiple 10G to 40G SDN-enabled comms processor Running OpenFlow SW stack Micro server host processor 64b ARM CPU 10G/40G RX Ports 10G/40G PCIe Control I/F SDN / OpenFlow Optimized Switch Fabric 10G/40G TX Ports 10G/40G DDR3/4 PHY DDR3/4 Cntrl CPU0 10G/40G PHYs Cntrl L-3 Cache OpenFlow & Autonomous Accelerators Interconnect AMBA 4 AXI / AMBA 3 CPU1 CPUn PCIe USB DDR 3/4 4x, 8x ARM v8 CPU w/ Caches SATA 6G Mgmt Engine 1G 10G 40G Port 1 AMBA 4 AXI Application Accelerator #1 1G 10G 40G Port n 1G/10G/40G Switch Application Accelerator #n Santa Clara, CA USA November 2014 8
Data Center SoC Requirements for IP 1 Low Latency Low Power Advanced RAS 16/14nm FF Protocols VXLAN increased bandwidth used to support VXLAN traffic Multiple networks with an increased packet size will consume more bandwidth VXLAN encapsulation header adds 50 bytes to the overall size of an frame. Requires jumbo frame support VXLAN packet parsing Not interfere with other offload functions: DMA mapping, TCP Segment Offload (TSO), ARP Offload, Checksum Offload etc
Synopsys 10G Controller IP Multi-speed 40GE/10GE/1GE MAC & multi-protocol PHY Flexible PCS 1GE-40GE RX filtering MAC DA/SA based L3/L4 Header based VLAN-tagged frame detection, stripping, and filtering High performance DMA module TCP Segment Offload ARP Offload Data Center Bridging support Priority based Flow Control Energy Efficient (EEE) VXLAN (& NVGRE) processing of tunneled packets Linux 3.0 Reference Drivers Multi-channel, VLAN, EEE, TSO, Jumbo Santa Clara, CA USA November 2014 10 AXI Mstr I/F AXI Slave I/F XG-MAC AXI DMA Arbiter TSO Memory Tx DMA Channel TSO Engine Rx DMA Channel DMA Channel CSR Tx Memory MTL Tx FIFO Controller MTL Rx FIFO Controller MTL CSR XG-MAC MTL Rx Memory MAC Tx MAC Rx MAC CSR GMII XGMII Select XG-MAC Core
10GE Controller VXLAN Implementation 10G Controller example HW switching in VTEP for traffic between the VMs running on the same Hypervisor (without encap/decap) VXLAN Parsing: Programmable UDP Destination Port ID Encapsule/decapsule VXLAN header for traffic Mark or Tag VXLAN packet Calculate & insert outer & inner IPv4 header checksum & payload checksum Identify tunnelled packets Header payload splitting AXI Mstr AXI Slave DMA Arbiter XG-MAC AXI TSO Memory Tx DMA TSO Rx DMA Channel DMA Channel CSR Modify TSO to support VXLAN packets Tx Memory Tx FIFO Contr Rx FIFO Contr MTL CSR XG-MAC MTL Rx Memory MAC Tx MAC Rx MAC CSR XG-MAC Core Jumbo frame support to improve efficiency of using additional VXLAN headers DMA mapping for VXLAN traffic L4-L3 based filters Modify Checksum Offload to support VXLAN packets
DWC Enterprise 40G DesignWare Enterprise 40G MAC DesignWare Enterprise 40G PCS DesignWare Enterprise 12G PHY Target Applications Data Center switches and routers Server inic Server blade chassis Enterprise computing and storage networks Supports 40GBASE-KR4, XLAUI, XLPPI, 40GBASE-CR4, 10GBASE- KR, XFI/SFI (SFF-8431), SGMII, QSGMII, Santa Clara, CA USA November 2014 12
VXLAN for Data Center Applications Data Center Virtualization enables new workloads for Cloud Computing Virtual extensible LAN extends Layer 2 for scalable VMs 10GE with VXLAN support becoming popular 10GE implementations optimized to support VXLAN Synopsys IP responds to Data Center SoC needs Low latency, Low power, Advanced protocols/features, RAS and 28nm-16/14nm FF process
DesignWare IP for Data Centers Micro Servers Leading Technologies 28HP/HPM to 16/14FinFET IP for Data Center SoCs DDR3/4 CTL + PHY 12G Enterprise PHY SDN Switches PCIe 3.1/4.0 CTL + PHY Memories Compilers w/ SEU Mitigation 1G/10G CTL + PHY AMBA Fabrics 40G CTL + PHY SATA 6G Host/Device USB 2.0/3.0 CTL + PHY ARC 32-bit CPUs Verification IP