PCI Express Supersedes SAS and SATA in Storage

Transcription

1 PCI Express Supersedes SAS and SATA in Storage Akber Kazmi PLX Technology Santa Clara, CA USA October

2 Agenda PCIe Roadmap/History Quick Overview of PCIe Enhancements in PCIe for New Applications Vendor Defined Enhancements New Usage Models and Applications Summary Santa Clara, CA USA October

3 SAS / SATA / PCIe Roadmaps SAS4? 24Gb/s Or SASe 16 Gb/s SATAe 16Gb/s PCIe Gen4 16Gb/s SAS 12Gb/s SATAe* 8Gb/s PCIe Gen3 8Gb/s Speed SAS 6Gb/s SATA 6Gb/s SAS 3 Gb/s PCIe Gen2 5Gb/s SATA 3 Gb/s PCIe Gen1 3Gb/s * No SATA 3 Adopted PCI Express (SATAe) Santa Clara, CA USA October

4 Quick Overview of PCIe Technology Replaces PCI bus with full software compatibility Serial point-to-point Packet based protocol Three generations of evolution Speeds - 2.5, 5.0 and 8.0 G/lane Support mix of Gen 1, 2 & 3 ports Ports can scale to x1 to x32 Low Latency x1 x2

5 PCIe Protocol Enhancements SR-IOV Sharing of I/O with multiple VMs DPC/eDPC Handling of surprise down SRIS Clock-less cabling PCIe Cable - New Spec for low cost cable PCIe Retimer Standardize re-timer Santa Clara, CA USA October

6 Single Root I/O Virtualization (SR-IOV) SR-IOV defines shared devices in PCIe Devices offer multiple virtual functions (VFs) The VMM assigns one or more VFs to a VM Higher performance & lower cost, foot-print & power Server Server VMM VM 1 VM N PCIe SR- IOV EP SR- IOV EP Santa Clara, CA USA October PF VF 1 VF M

7 SRIS Clocks SRIS (Separate Refclk Independent SSC) Enables new PCIe applications like Cables Allow each side of the link to have its own clock Two kinds of clocking supported 600 ppm difference with no SSC (aka SRNS) 5600 ppm difference with independent SSC (aka SRIS)

8 PCIe Cable PCIe cable spec (aka OcuLink) Targeted for low cost Optical and Copper cables Spec will address Active, half active and passive cables Scope of definition Electrical, Mechanical, FF, signaling, power, etc. Expected completion 2014

9 PCIe Re-timers / Repeaters A new ECN for re-timer in review Current re-timers don t work well in PCIe3 systems Defining repeater and re-timer function for PCIe3 Useful in active cable apps Will also be used in boards and backplanes

10 DPC/eDPC Downstream Port Containment (DPC) Prevents host time-out or blue screen When triggered by error from an endpoint Take down link to endpoint Send error message/interrupt to host Reply to read request in time-out window Completion synthesis for root ports Host CPU Host s/wto handle responses PCIe I/O Device I/O Device I/O SSD Device I/O Device

11 Vendor Defined PCIe Enhancements Sharing SR-IOV device with multiple hosts Host isolation/failover Embedded DMA controller Host-to-Host communication Santa Clara, CA USA October

12 SSD Adapter Card ASIC1 ASIC2 Two SSD Controllers NVMe Form-Factor 2.5 Only ASIC1 (controller) exposed to Host ASIC1 servicing interrupts from ASIC2 ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC Two or more SSD controllers aggregated with a switch Adapter Card Form-Factor Host isolation capability One ASIC exposed to Host Services interrupts from other ASICs

13 Server Motherboards Capacity expansion in server enclosures Aggregation through PCIe NIC Server Motherboard RAM CPU RAM Santa Clara, CA August

14 FLASH Appliances IO IO Expansion Box PCIe over Cu/Opt Servers Santa Clara, CA August

15 Future PCIe Applications Expanded Use in Data Center Santa Clara, CA USA October

16 Hide Some Endpoints From The Host One endpoint visible to Host, rest are hidden Exposed endpoint communicates with the host and manages hidden endpoint Host CPU Devices on left not visible to Host PCIe I/O or SSD Device I/O or SSD Device I/O or SSD Device I/O or SSD Device Green endpoint visible to host

17 Flash Appliances with I/O Sharing Share SR-IOV SSD modules in expansion chassis Shared by multiple servers/hosts or blade servers IO IO Expansion Box PCIe over Cu/Opt Servers Santa Clara, CA August

18 Today s Data Center Rack Storage Multiple Fabrics 1-2 es Networking LOM/NIC Ethernet Bridging Devices LOM/NIC HBA LOM/NIC HBA LOM/NIC HBA PCI Express LOM/NIC HBA HBA Fibre Channel PCI Express Already available in Every Box

19 ExpressFabric Rack Scale Integration Shared I/O to Drop Costs & Power Pushes the Network to Top of Rack..Removing Costly NICs/HBAs HBA NIC ExpressFabric Converged Fabric PCI Express * PCI Express Retimer High Performance 32Gbs Links (x4) Scalable to x8, x16 PCI Express Retimer Retimer Retimer Simple ~ $5 and ~ 1 Watt PCIe Retimers PCI Express Summary ~½ the Cost ~½ the Power

20 ExpressFabric Rack Scale Integration HBA NIC Flash Flash Flash Flash Flash Flash Flash Flash GPU GPU GPU GPU GPU GPU us us us us us us SSD Controller SSD Controller GPU GPU GPU us us us PCI Express ExpressFabric us Retimer PCIe Box Serving the Rack Can be in any location Enables Shared SSD Storage PCIe offers the Highest Performance GPGPU Computing PCIe is Standard for GPUs Microservers PCIe as Future Fabric

21 Summary PCIe offers lowest latency and scalability PCIe has been adopted in many new applications & expected to expand in others PCI-SIG has added new features to help support current & future use of PCIe PCIe vendors adding new features to enhance performance & ease of use in Data Center Santa Clara, CA USA October