Innovating in the Kushagra Vaid GM, Azure Cloud Hardware Engineering Microsoft

Transcription

1 Innovating in the Kushagra Vaid GM, Azure Cloud Hardware Engineering Microsoft 1

2 Operating at scale Infrastructure capex $B s annual spend on Cloud infrastructure Servers account for ~60% of overall infrastructure capex Components (CPU, DRAM, SSD etc) account for 85% of server capex Network 7% DC build cost 33% rack infra supplychain cpu nic+fpga Server capex 60% storage (ssd+hdd) memory Infra capex breakdown Server capex breakdown 2

3 Operating at scale Design consistency and standardization Supply chain bandwidth and latency Global datacenters (28 regions worldwide for Azure) Infrastructure Opex Server utilization and resource efficiency Extra capacity buffers for turn space and unexpected issues Server maintenance and repair costs (RMA) DC facilities maintenance costs 3

4 Driving Open Compute Blade (2014/15) SERVER STORAGE NETWORKING Chassis Manager (2014) JBOD (2014) Switch Abstraction Interface (SAI) (2015) Server UPS (2015) Chassis (2014) Storage Blade (2015) M.2 Cloud SSD (2014/15) Software for Open Networking in the Cloud (SONiC) (2016) 4

5 Cloud scale networking challenges On-demand provisioning of customer network & policies Millions of VMs spread across 100 s of thousands of servers Tolerance to hardware and software failures Division of labor across network Software focus on policy and implementation Hardware focus on capacity, reliability and performance 5

6 Networking Hardware implications Bandwidth: 1G 10G 40G 50G in just 6 years! Flat networks Routable RDMA (ROCE v2) SDN Control plane on host vswitch offload acceleration NFV Disaggregated appliances Host based security, load balancing, DNS, DPI In-transit wire encryption New customer scenarios: BYO IP, BYO Topology, BYO Appliance 6

7 Azure SmartNIC (using FPGAs) HW is needed for scale, perf, and COGS at 40G month ASIC cycle + time to roll new HW is too slow Implement custom features for differentiated offerings Combines agility of SDN with speed+cogs of HW Roll out Hardware just like Software! NIC ASIC Server CPU SmartNIC FPGA Bump in the Wire: Reconfigurable FPGA + NIC ASIC ToR 7

8 Example scenario: Virtual Network Encryption SmartNIC can dial encrypted virtual network tunnels for each tenant Provides E2E security and privacy against actors inside the NW fabric Line Rate Encryption at 40Gbps Fabric SmartNIC SmartNIC Host Host VM VM VM VM 8

9 What is the larger trend at play here? 9

10 Its all about the workloads Emergence of new workloads Machine learning: model training and model evaluation NW processing for SDN and NFV Storage processing Datacenter tax Compression, Encryption etc Characteristics of these workloads Highly parallel, streaming, don t map well to CPU ISA Evolving faster than hardware can keep up Can run entirely in the IO complex with minimal CPU interaction 10

11 Implications to Systems Design Distributed compute between CPU and IO complex IO taking increasingly larger role for computational tasks mapped to specialization needs Disaggregation an emerging theme for Storage and Network Is the classic Compute model obsolete? No, still needed for legacy workloads Examples of new architectures with IO based-compute Nvidia DGX-1 with Tesla NVLink FPGAs with DSPs Several stealth mode startups building specialized silicon 11

12 Levels of Efficiency (SW vs HW) More efficient (ops/$ or ops/w) 1x 3x 5x-30x 20x 100x > 100x Software on CPUs Software on many-core chips Programmable logic (FPGAs) Structured ASICs Custom ASICs Services running at small scale or evolving rapidly More flexible (rapid development) Algorithms running at large scale and are stable 12

13 Summary Infrastructure scale driving architectural re-think Cloud driving new dimension for HW requirements System architectures need to evolve to accommodate specialized compute scenarios Significant opportunities for breakthrough HW innovation how will the industry respond? 13

14 2015 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. 14