Data Center Ethernet Facilitation of Enterprise Clustering. David Flynn, Linux Networx Orlando, Florida March 16, 2004

Transcription

1 Data Ceter Etheret Facilitatio of Eterprise Clusterig David Fly, Liux Networx Orlado, Florida March 16,

2 2 Liux Networx builds COTS based clusters

3 3 Clusters Offer Improved Performace Scalability Reliability Flexibility Price Maiframe Workstatio Mii-computer PC Vector Supercomputer

4 4 Phase-1: Move HPC ito Eterprise # 6 # 7 N/A Los Alamos - Lightig TFLOPS Liux Networx Evolocity 2816 Optero Processors Lawrece Livermore - MCR 11.2 TFLOPS Liux Networx E2 2,304 Itel Processors Los Alamos - Pik 10 TFLOPS Liux Networx E2 2,048 Itel Processors LANL BlueSteel 2.5 TFLOPS, PCR 1.7 TLOPS, Argoe 1.68 TFLOPS, Seitel 1.4 TFLOPS, Boeig 1.2 TFLOPS

5 5 Cluster Solutios Mea Better Simulatios & Aalysis 100 TFLOPS Protei Foldig which meas Weather & Climate Modelig 50 TFLOPS 3D Explosio 10 TFLOPS 1 TFLOPS Crash Testig Bioiformatics More Efficiecy More Productivity Greater ROI More Profit

6 6 The HPC Cluster Market High performace cluster solutios for the scietific, fiacial & eterprise markets Egieerig Desig Life Scieces Special FX/Etertaimet Fiacial Services / Capital Markets Eterprise Ifrastructures Research & Defese

7 7 HPC Cluster applicatios Customers rely o clusters to solve their most complex ad techical problems

8 HPC Clusterig i the Eterprise It s ot just a competitive edge i Faster desig ad aalysis leads to shorter desig cycles i Helps products arrive to market faster i Esures safety ad effectiveess of products i Saves time ad moey It s required to eve compete. 8

9 9 The Expadig Uiverse of HPC Cluster Idepedet Software Vedors (ISVs)

10 10 HPC Cluster Market Growth $ Billios HPC Clusters Traditioal HPC All HPC IDC 2003 est.

11 11 Phase-2: Eterprise Sweet Spot Productivity Applicatios Geeral Busiess Applicatios Research Labs Databases Sweet Spot

12 12 HPC Cluster Techologies Applied to Traditioal Eterprise Problems Distributed databases Distributed trasactio processig Distributed parallel data miig Parallel high performace file systems High availability N modular redudacy Server virtualizatio Utility computig

13 Pure HPC Market Opportuity Scietific Research & Defese Egieerig Oil ad Gas Life Scieces Etertaimet $5 Billio. IDC 2003 est. Future growth is expected to come from all segmets of the HPC market 13

14 14 Combied Market Opportuity Fiacial Services Eterprise $64 Billio Fiacial Services aloe IDC 2003 est. $5 Billio HPC

15 What Goes Ito a Cluster Mother boards CPUs Memory Storage Itercoect / etwork These must be Based o stadards Reliable ad versatile Sold i volume Available from multiple vedors Richly supported by software ad experiece I a word commodity All ca be foud COTS - except for the itercoect - 15

16 Compay 1 Pros Cos Most advaced techology Desiged specifically for HPC O-switch barrier & atomic ops O-NIC reductios 1GB/s badwidth ~3us latecy Most expesive Sigle supplier Proprietary Rigid topology Demadig software setup Fragile Copper oly No Etheret iteroperability Oly useful as itercoect Needs secodary etwork Limited multicast 16

17 Compay 2 Pros Cos Lowest cost Fiber O-NIC reductios Flexible topologies <260MB/s badwidth ~5us latecy Sigle supplier Proprietary Lack of ECC throughout Lack of copper No broadcast / multicast Limited Etheret iteroperability Needs secodary etwork 17

18 IfiiBad Pros Cos Stadards based Relatively cheap Fiber & Copper Flexible topologies ~5us latecy No o-nic reductios Sigle supplier for key tech Host cliet adapters Switches DDR IB to Etheret bridgig Lik aggregatio Too may suppliers of Some Etheret iteroperability Multicast Driver software Maagemet software Lacks wide acceptace Revolutio ot evolutio MAC address icompatibility 18

19 What It Takes to Be a Cluster Itercoect Reliable data delivery Got to get the data! Low latecy ( <= 3 micro secods) Got to get it ow! High badwidth ( >= 1GB/s) Got to get lots of it! Perform well at saturatio All eed it at oce! Simulatios that ru o HPC clusters commuicate i bursts. The machies operate i lock step. All compute the all exchage data. The simulatio caot proceed to the ext phase util the boudaries are exchaged ad commuicatio is complete. Time spet waitig o commuicatio effectively idles the horsepower of the etire cluster. HPC is the the worst case sceario for etwork commuicatio i that; All commuicate at oce ad oe proceed util all are doe. 19

20 20 Scalig Efficiecies As a fuctio of itercoect & cluster size Compay 1 Ifiibad Compay 2 1GigE A cluster s scalig efficiecy is determied by what percetage of total time the cluster speds calculatig. Time spet commuicatig is time lost calculatig.

21 21 Avoidig Priority Iversio Problem: The itercoect must carry traffic of differig priority. High priority sychroizatio ad latecy sesitive traffic. Low priority file system access, moitorig ad maagemet Low priority traffic ca be large ad slow to be serviced. High priority packets may therefore block o large slow packets. The cluster odes stall waitig for importat data Solutio: The itercoect must implemet lik level quality of service or QOS. Protocols, services, ad applicatios must be coded to operate i a QOS aware fashio. Itercoects ad HPC software today correctly maage QOS. Itercoect today use virtual chaels / laes / etworks.

22 Avoidig Ijectio Rate Rampig Problem: Solutio: Rampig up ijectio rates wastes precious latecy/badwidth. Burstig applicatios cause each other to back off. Dropped packets are iterpreted as cogestio - causig back off. Higher badwidth etworks take loger to ramp up. Use lik-level flow cotrol. Itercoects today use lik credit / debit systems for flow cotrol. Multiple buffer spaces or virtual laes are used to avoid depedecy cycles. 22

23 Problem: Solutio: Avoidig Packet Loss Packets are ijected o the etwork at a high rate from may sources. These packet storms ca overflow switch bufferig. Packets the get dropped o the floor. Data corruptio ca also cause packet loss. Dropped packets must be retrasmitted by the sedig edpoit. Iterruptig the sedig OS adds sigificat latecy. A TOE ca help but there is still 2 more etwork trips of latecy. The itercoect must guaratee delivery of packets. Each device esures ucorrupted deliver to the ext. Itercoects today guaratee data delivery or are lossless. Packet drop rates are less tha 1 i 1e10th packets. Edpoit protocols are optimistic optimized for successful delivery 23

24 TCP is Not Suited for Clusters TCP is avoided by applicatios from MPI to Oracle. TCP caot ru the etwork at ear saturatio. TCP caot eve kow where saturatio is. TOEs do ot help. Pessimistic protocols will ot work. HPC Itercoects show what will work. Optimistic protocols require Data Ceter Etheret. The MAC must provide key features 24

25 MAC / Etheret Needs Lik level QOS Lik level flow cotrol Lik level guarateed delivery Protocols ad services to match With these features Clusterig will migrate smoothly ito eterprise. Etheret will become ubiquitous i clusterig. Clusterig will become ubiquitous. 25

26 26 Liux Networx Needs Data Ceter Etheret Eterprise customers wat a Etheret solutio No oe likes havig two etworks. Doubles the complexity Doubles the maiteace Icreased costs Iteroperability cocers Commodity always wis. Everyoe kows how to admiister Etheret. All software supports Etheret. Proprietary itercoects are the most problematic compoet. Liux Networx must provide a Etheret solutio to satisfy eterprise customer requiremets. BONUS: It will save us time ad moey.