Server Use Cases for 40GE and 100GE July 11, Robert Winter, Dell

Transcription

1 Server Use Cases for 40GE and 100GE July 11, 2007 Robert Winter, Dell

2 Fabric Comparison EN, 10Gbps IB, 4xDDR FC, 4 10 Gb/s Ethernet is good enough (for all applications) 100+ PCIe gen1 x (actual data rates) Public Data, Various Sources BANDWIDTH (Gbps) LATENCY (µsec) REACH (m) IEEE Higher Speed Study Group 2

3 Fabric Comparison EN, 10Gbps IB, 4xQDR FC, Gb/s Ethernet may become a 3 rd place fabric (for some applications) 100+ PCIe gen2 x (actual data rates) Public Data, Various Sources BANDWIDTH (Gbps) LATENCY (µsec) REACH (m) IEEE Higher Speed Study Group 3

4 Fabric Comparison Ethernet is disadvantaged at Layer 2 (compared to other fabrics) I Layer Model Ethernet/IP PCIe FC IB 7 APPLICATION User Endpoint TELNET PnP User Model 6 Data PRESENTATION Data Representation and Encryption ASCII Driver Model Apps Apps 5 SESSION Connection Establishment Sockets 4 Segments TRANSPORT Reliable Communications TCP/UDP PCIe DLLP FC-4 IBA Ops 3 2 Packets Frames NETWORK Routing and Addressing DATA LINK Physical Addressing IP STP, VLAN PCIe TLP PCIe DLLP FC-3 FC-2 FC-1 Network Encode MAC 1 Bits PHYSICAL Bit Format and Conversion Layer Responsible for Guaranteed Packet Delivery Ethernet PCIe LLP IEEE Higher Speed Study Group 4 FC-0 Physical

5 Server I/O Drivers Gb/s Server I/O Capacity (Virtualization, Bus Speeds, Multi-Cores, Offloads, etc.) PCIe Gen 3, x8 40 4xQDR IB PCIe Gen 2, x xDDR IB 10 Gb/s Ethernet PCIe Gen 1, x Public Data, Various Sources IEEE Higher Speed Study Group 5

6 General Blade Server Chassis Architecture Front Compute Blade IO Module IO Module Compute Blade Mid-plane ( Multi-Tbps fabric) Back ( various networks ) IEEE Higher Speed Study Group 6

7 Virtualization and Ethernet FC EN IB Fibre Channel Ethernet Infiniband Single on a single physical machine Each type of network access requires a different type of I/O interface IEEE Higher Speed Study Group 7

8 Virtualization and Ethernet FC EN IB Fibre Channel Ethernet Infiniband Multiple es on a single physical machine Each has a virtual copy of each I/O interface Each type of network access requires a different type of I/O interface IEEE Higher Speed Study Group 8

9 Virtualization and Ethernet FC EN IB FC EN IB FC EN IB Fibre Channel Ethernet Infiniband Multiple es on multiple physical machines Each has a virtual copy of each I/O interface Each type of network access requires a different type of I/O interface IEEE Higher Speed Study Group 9

10 Virtualization and Ethernet EN EN EN EN EN EN Single adapter, multiple uses (storage, cluster, chassis, remote computing fabrics) Ethernet Multiple es on multiple physical machines Each has a virtual copy of each I/O interface Ethernet becomes a Unifying Fabric Bandwidth > 10Gbps L2 Guaranteed packet delivery L2 OOO packet detection Deterministic rate limiting Bandwidth metering Better congestion management IEEE Higher Speed Study Group 10

11 40GE Drivers PCIe Gen 1 x8 consumes 16 Gb/s PCIe Gen 2 x8 will not fit in 2 x 10 Gb/s pipe (32 Gb/s) PCIe Gen 3 x8 will not fit in 4 x 10 Gb/s pipes (64 Gb/s) Chassis will be developed that may have multiple KRs A single mezzanine card can then push > 10 Gb/s Eliminate a major advantage IB has over EN IEEE Higher Speed Study Group 11

12 100GE Drivers Needed for x10 aggregation of 10GE Else 10GE is limited to stackable/pizza-box switching Needed to push Ethernet into the Telecom space Achievable Ethernet Everywhere Internet IEEE Higher Speed Study Group 12

13 40GE/100GE Co-Existence Dell sees a need for both 40 Gb/s and 100 Gb/s If achieving one doesn t seriously impact delivery of the other As long as we prevent creation of two different Ethernets 40 Gb/s Will happen either inside or outside IEEE (prefer inside) Mostly applies to servers and chassis mid-plane Will guaranteed L2 delivery be a requirement? 100 Gb/s Mostly applies to 10 Gb/s aggregation and telecommunications Credit-based congestion management is primary here Some switching issues 40 Gb/s does not aggregate well into 100 G/bs Switch silicon may be needed that supports 10/40/100 IEEE Higher Speed Study Group 13