SDN for the Cloud. Albert Greenberg Distinguished Engineer Director of Microsoft Azure [email protected]

Transcription

1 SDN for the Cloud Albert Greenberg Distinguished Engineer Director of Microsoft Azure [email protected]

2 Road to SDN WAN Motivating scenario: network engineering at scale Innovation: infer traffic, control routing, centralize control to meet network-wide goals Cloud Motivating scenario: software defined data center, requiring per customer virtual networks Innovation: VL2 scale-out L3 fabric network virtualization at scale, enabling SDN and NFV TomoGravity ACM Sigmetrics 2013 Test of Time Award RCP Usenix NSDI 2015 Test of Time Award 4D ACM Sigcomm 2015 Test of Time Award VL2 ACM Sigcomm 2009

3 Cloud provided the killer scenario for SDN Cloud has the right scenarios Economic and scale pressure huge leverage Control huge degree of control to make changes in the right places Virtualized Data Center for each customer prior art fell short Cloud had the right developers and the right systems High scale fault tolerant distributed systems and data management At Azure we changed everything because we had to, from optics to host to NIC to physical fabric to WAN to Edge/CDN to ExpressRoute (last mile)

4 Hyperscale Cloud

5 Microsoft Cloud Network 15 billion dollar bet

6 Microsoft Cloud Network

7 Microsoft Cloud Network

8 Compute Instances 100K Millions Azure Storage Datacenter Network 10 s of PB 10 s of Tbps Exabytes Pbps

9 >85% Fortune 500 using Microsoft Cloud >93,000 New Azure customers a month 425 MILLION Azure Active Directory users >18 BILLION Azure Active Directory authentications/week Scale 1 TRILLION Azure Event Hubs events/month >60 TRILLION Azure storage objects >5 MILLION requests/sec 1 out of 4 Azure VMs are Linux VMs 1,400,000 SQL databases in Azure

10 Agenda Consistent cloud design principles for SDN Physical and Virtual networks, NFV Integration of enterprise & cloud, physical & virtual Future: reconfigurable network hardware Demo Career Advice Acknowledgements

11 Azure SmartNIC

12 Cloud Design Principles Scale-out N-Active Data Plane Embrace and Isolate failures Centralized control plane: drive network to target state Resource managers service requests, while meeting system wide objectives Controllers drive each component relentlessly to the target state Stateless agents plumb the policies dictated by the controllers These principles are built into every component

13 Hyperscale Physical Networks

14 VL2 Azure Clos Fabrics with 40G NICs Regional Spine T3-1 T3-2 T3-3 T3-4 Scale-out, active-active Data Center Spine T2-1-1 T2-1-2 T2-1-8 T2-2-1 T2-2-2 T2-2-4 Row Spine T1-1 T1-2 T1-7 T1-8 T1-1 T1-2 T1-7 T1-8 T1-1 T1-2 T1-7 T1-8 Rack T0-1 T0-2 T0-20 Servers T0-1 T0-2 T0-20 Servers T0-1 T0-2 T0-20 Servers Outcome of >10 years of history, with major revisions every six months Scale-up, active-passive L3 LB/FW LB/FW LB/FW LB/FW L2 16

15 Challenges of Scale Clos network management problem Huge number of paths, ASICS, switches to examine, with a dynamic set of gray failure modes, when chasing app latency issues at % levels Solution Infrastructure for graph and state tracking to provide an app platform Monitoring to drive out gray failures Azure Cloud Switch OS to manage the switches as we do servers

16 Azure State Management System Architecture Infrastructure Provisioning Fault Mitigation Traffic Engineering Checker Centralized control plane: drive network to target state Observed State Monitor Proposed State Updater Target State Abstract network graph and state model: the basic programming paradigm XXx10G BGP XXx10G XXx10G XXx10G BGP XXx10G BGP BGP XXx10G XXx10G XXx10G 16 Spine Switchs BGP 16 Spine Switchs 16 Spine Switchs 16 Spine Switchs Border Routers Each border router connects to each Interconnect switch n7k GNS/WAN Network n7k BGP BGP High scale infrastructure is complex: multiple vendors, designs, software versions, failures 64x10G 64x10G 64x10G 64x10G 64x10G 64x10G 64x10G 64x10G ebgp ebgp ebgp ebgp 17 pods 17 pods 64x10G 48 servers pod podset 64x10G 48 servers podset

17 App I: Automatic Failure Mitigation Fault Mitigation Checker Observed State Proposed State Target State Device monitor Link monitor Link updater Device updater Repair bad links Repair bad devices

18 Gbps App II: Traffic Engineering Towards High Utilization Traffic Engineering Checker 250 SWAN 200 Topology, link load Observed State Device &Link monitor Routes LSP monitor Proposed State Service demand Service monitor Target State Routing updater Add or remove LSPs, change weights Service LB updater Sun Mo n = Total traffic = Elastic 100 Gbps Tue We d Thu Fri Sat = Background 100 Gbps = Interactive Gbps

19 Azure Scale Monitoring Pingmesh Problem: Is it the app or the net explaining app latency issues? Solution: Measure the network latency between any two servers Full coverage, always-on, brute force Running in Microsoft DCs for near 5 years, generating 200+B probes every day Normal Podset down Podset failure Spine

20 EverFlow: Packet-level Telemetry + Cloud Analytics Cloud Network Challenge Guided prober Packet drop Latency spike Loop Load imbalance Cloud Scale Data Match & mirror Reshuffler Collector Collector Filter & aggregation EverFlow controller Query Distributed storage Azure Analytics Azure Storage Collector 22

21 Hard ware Operating System Azure Cloud Switch Open Way to Build Switch OS Switch Control: drive to target state SAI collaboration is industry wide SAI simplifies bringing up Azure Cloud Switch (Azure s switch OS) on new ASICS Provision Deployment Automation Hw Load Balancer BGP SNMP Other SDN Apps LLDP ACL Ethernet Interfaces Switch ASIC Driver Switch State Service Sync Sync Sync Agent1 Agent2 Agent3 Switch Abstraction Interface (SAI) Switch ASIC SDK User space Switch ASIC SAI is on github Front Panel Ports

22 Hyperscale Virtual Networks

23 L3 Tunnel Network Virtualization (VNet) Microsoft Azure Virtual Networks Azure is the hub of your enterprise, reach to branch offices via VPN Microsoft Azure 10.1/ /16 VNet is the right abstraction, the counterpart of the VM for compute Efficient and scalable communication within and across VNets Internet ISP/MPLS QoS

24 Hyperscale SDN: All Policy is in the Host Proprietary Appliance Management Management Plane SDN Azure Resource Azure Resource Azure Manager Resource Manager Manager Control Data Control Plane Controller Controller Controller Data Plane Switch (Host) Switch (Host) Switch (Host)

25 Key Challenges for Hyperscale SDN Controllers Must scale up to 500k+ Hosts in a region Needs to scale down to small deployments too Must handle millions of updates per day Must support frequent updates without downtime

26 Microsoft Azure Service Fabric: A platform for reliable, hyperscale, microservice-based applications App1 App2

27 Regional Network Manager Microservices Management REST APIs Gateway VIP Manager Service Manager (VMs, Cloud Services) VNET Manager (VNETs, Gateways) Software Load Balancers MAC Manager Fabric Task Driver Partitioned Federated Controller Network Resource Manager (ACLs, Routes, QoS) Other Appliances Cluster Network Managers

28 Regional Network Controller Stats 10s of millions of API calls per day Write API Transactions API execution time Read : <50 milliseconds Write : <150 milliseconds Varying deployment footprint Smallest : <10 Hosts Largest : >100 Hosts AddOrUpdateVnet AllocateMacs AllocateVips AssociateToAclGroup CreateOrUpdateNetworkServi ce DeleteNetworkService FreeMac

29 Hyperscale Network Function Virtualization

30 Azure SLB: Scaling Virtual Network Functions Key Idea: Decompose Load Balancing into Tiers to achieve scale-out data plane and centralized control plane Router Router Tier 1: Distribute packets (Layer 3) Routers ECMP Tier 2: Distribute connections (Layer 3-4) Multiplexer or Mux Enable high availability and scale-out Tier 3: Virtualized Network Functions (Layer 3-7) Example: Azure VPN, Azure Application Gateway, third-party firewall Loadbalanced IP Packets Mux Mux Mux Load-balanced Connections VNF VNF VNF

31 Express Route: Direct Connectivity to the Cloud Connectivity Provider Infrastructure (physical WAN) Azure Edge (network virtual function) Virtual Networks

32 Data Center-Scale Distributed Router Centralized control plane Distributed data plane Customer s network Microsoft WAN Customer s Virtual Network in Azure

33 Building a Hyperscale Host SDN

34 Virtual Filtering Platform (VFP) Acts as a virtual switch inside Hyper-V VMSwitch NIC vnic vnic VM Switch VM VM VFP ACLs, Metering, Security VNET SLB (NAT) Provides core SDN functionality for Azure networking services, including: Address Virtualization for VNET VIP -> DIP Translation for SLB ACLs, Metering, and Security Guards Uses programmable rule/flow tables to perform per-packet actions Supports all Azure data plane policy at 40GbE+ with offloads Coming to private cloud in Windows Server 2016

35 Flow Tables: the Right Abstraction for the Host VMSwitch exposes a typed Match- Action-Table API to the controller Controllers define policy One table per policy Key insight: Let controller tell switch exactly what to do with which packets e.g. encap/decap, rather than trying to use existing abstractions (tunnels, ) VNet Routing Policy Controller NAT Endpoints VNet Description Tenant Description ACLs Host: Flow Action Flow Action VFP Flow Action NIC TO: 10.2/16 Encap to GW TO: Encap to TO:!10/8 NAT out of VNET TO: TO:!10/8 DNAT to SNAT to TO: /24 Allow 10.4/16 Block TO:!10/8 Allow VM VNET LB NAT ACLS

36 Table Typing/Flow Caching are Critical to Performance COGS in the cloud is driven by VM density: 40GbE is here First-packet actions can be complex Established-flow matches must be typed, predictable, and simple hash lookups First Packet NIC Flow TO: 10.2/16 TO: TO:!10/8 Action Encap to GW Encap to NAT out of VNET Flow TO: TO:!10/8 Action DNAT to SNAT to Flow TO: /24 Action Allow 10.4/16 Block TO:!10/8 VNET LB NAT ACLS Allow Blue VM Subsequent Packets Connection Action , ,80,9876 DNAT + Encap to GW , ,80,9876 Encap to , ,6754,80 SNAT to VFP

37 RDMA/RoCEv2 at Scale in Azure Memory NIC NIC Memory Buffer A Write local buffer at Address A to remote buffer at Address B Buffer B is filled Buffer B Application Application RDMA addresses high CPU cost and long latency tail of TCP Zero CPU Utilization at 40Gbps μs level E2E latency Running RDMA at scale RoCEv2 for RDMA over commodity IP/Ethernet switches Cluster-level RDMA DCQCN* for end-to-end congestion control *DCQCN is running on Azure NICs

38 RDMA Latency Reduction at 99.9 th %ile in Bing TCP 99 th percentile RDMA 99.9 th percentile RDMA 99 th percentile

39 Host SDN Scale Challenges Host network is Scaling Up: 1G 10G 40G 50G 100G The driver is VM density (more VMs per host), reducing COGs Need the performance of hardware to implement policy without CPU Need to support new scenarios: BYO IP, BYO Topology, BYO Appliance We are always pushing richer semantics to virtual networks Need the programmability of software to be agile and future-proof How do we get the performance of hardware with programmability of software?

40 Azure SmartNIC Use an FPGA for reconfigurable functions FPGAs are already used in Bing (Catapult) Roll out Hardware as we do software Programmed using Generic Flow Tables (GFT) Language for programming SDN to hardware Uses connections and structured actions as primitives SmartNIC can also do Crypto, QoS, storage acceleration, and more NIC ASIC Host CPU SmartNIC FPGA ToR

41 SmartNIC Controller Controller Northbound API Southbound API Controller SLB Decap SLB NAT VNET ACL Metering Rule Action Rule Action Rule Action Rule Action Rule Action * Decap * DNAT * Rewrite * Allow * Meter VFP Rewrite Flow > , >80 Encap Transposition Engine Action Decap, DNAT, Rewrite, Meter GFT Offload Engine VM First Packet VMSwitch GFT Offload API (NDIS) SmartNIC 50G GFT Flow Action > , >80 Decap, DNAT, Rewrite, Meter Crypto RDMA GFT Table QoS 43

42 Azure SmartNIC

43 Demo: SmartNIC Encryption

44 Closing Thoughts Cloud scale, financial pressure unblocked SDN Control and systems developed earlier for compute, storage, power helped Moore s Law helped: order 7B transistors per ASIC We did not wait for a moment for standards, for vendor persuasion SDN realized through consistent application of principles of Cloud design Embrace and isolate failure Centralize (partition, federate) control and relentlessly drive to target state Microsoft Azure re-imagined networking, created SDN and it paid off

45 Career Advice Cloud Software leverage and agility Even for hardware people Quantity time With team, with project Hard infrastructure problems take >3 years, but it s worth it Usage and its measurement oxygen for ideas Quick wins (3 years is a long time) Foundation and proof that the innovation matters

46 Shout-Out to Colleagues & Mentors AT&T & Bell Labs Han Nguyen, Brian Freeman, Jennifer Yates, and entire AT&T Labs team Alumni: Dave Belanger, Rob Calderbank, Debasis Mitra, Andrew Odlyzko, Eric Sumner Jr. Microsoft Azure Reza Baghai, Victor Bahl, Deepak Bansal, Yiqun Cai, Luis Irun-Briz, Yiqun Cai, Alireza Dabagh, Nasser Elaawar, Gopal Kakivaya, Yousef Khalidi, Chuck Lenzmeier, Dave Maltz, Aaron Ogus, Parveen Patel, Mark Russinovich, Murari Sridharan, Marne Staples, Junhua Wang, Jason Zander, and entire Azure team Alumni: Arne Josefsberg, James Hamilton, Randy Kern, Joe Chau, Changhoon Kim, Parantap Lahiri, Clyde Rodriguez, Amitabh Srivastava, Sumeet Singh, Haiyong Wang Academia Nick Mckeown, Jennifer Rexford, Hui Zhang

47 SIGCOMM 15 Everflow Packet-level telemetry for large DC networks 10 6 x reduction in trace overhead, pinpoint accuracy Corral Joint data & compute placement for big data jobs 56% reduction in completion time over Yarn R2C2 Network stack for rack-scale computing Rack is the new building block! Iridium Low-latency geo-distributed analytics 19x query speedup, 64% reduction in WAN traffic DCQCN Congestion control for large RDMA deployments 2000x reduction in Pause Frames, 16x better performance PingMesh DC network latency measurement and analysis 200 billion probes per day! Silo Virtual networks with guaranteed bw and latency No changes to host stack! Hopper Speculation-aware scheduling of big-data jobs 66% speedup of production queries Eden Enable network functions at end host Dynamic, stateful policies, F#-based API