Performance of an Infiniband cluster running MPI applications

Transcription

1 Jsé Miguel-Alns1, Txema Mercer2, Edu Ogand2 In 1Department f Cmputer Architecture and Technlgy, UPV/EHU the 2Scientific Cmputing Service, UPV/EHU f this paper we shw the Technical results f Reprt running EHU-KAT-IK a cllectin f well-knwn benchmarks n Perfrmance f an Infiniband cluster running MPI cmpare the Arina/Maiz Basque Cuntry. cluster managed Fr the purpse by the Scientific f putting Cmputing the numbers Service in perspective, f the University we applicatins managed The benchmarks perfrmance the by the results Barcelna f chice with thse are Supercmputing btained frm Center. the MareNstrum supercmputer cllective f the different We fcus MPI n primitives, the perfrmance including f uni-directinal pint-t-pint as well as Intrductin Experiments MareNstrum difference transferring class f cdes peratins. thugh, perfrm Arina/Maiz executed have in been supercmputing run an empty centers. Results that means ur they intentin that have results been was are executed t smewhat prvide when sme biased the figures machine against useful system. the was fr MareNstrum. in Hwever, cmparisn, prductin. in the platfrm an exhaustive evaluatin f bth systems. Nte, This Details f shw t the run that characteristics parallel the Arina/Maiz applicatins. f the cluster, Arina/Maiz althugh cluster limited are given in size, in Appendix a pwerful A. nt Details t f The terms n interesting the Intel MareNstrum MPI Bechmarks, are prvided versin in 2.3, Appendix are designed B. In invlving a given f latency platfrm. and/r They thrughput) are nt really f the useful mst t representative t measure actual MPI the applicatins, primitives perfrmance running but (in can rder this test just we indicatrs tw will MPI nly abut prcesses. use the perfrmance benchmark As we always that f the measures allcate intercnnectin ne uni-directinal prcess mechanism. t ne data CPU, transfers, give netwrks: use t bth put numbers terms withut in cntext, distinctin. these are We the have main run characteristics the tests different f the intercnnectin netwrks. we In IMB Micr-benchmarks - Intel NAS MPI Parallel Benchmarks benchmarks [1],[2], a cllectin a cllectin f small f applicatins prgrams representative that test the f the

2 VAPI. TCP-GE. GM. used Myrinet-2000 Infiniband t Gigabit access TCP netwrk Ethernet this netwrk cnnectins netwrk. [3], netwrk [4], at The 102 n (1 Gb/s. Gb/s. MPI tp Gb/s). f implementatin VAPI GM this The is is netwrk. the MPI the name name implementatin (MPICH-GM) is f cnfigured f the the libraries libraries is t is and use and We the Arina/Maiz cnfigured SHMEM. Internal t use GM. have messages measured cluster: are thrughput interchanged cmmunicatin and using delay shared fr between these memry cmbinatins prcessrs structures. in f the nde same / netwrk nde. MPI VAPI-I. SHMEM-I. VAPI-O. Itanium Optern Itanium prcessrs in in different the same ndes, nde, via via via the the Infiniband memry. f Cmpilers netwrk. SHMEM-O. memry. TCP-GE-O. Ethernet. Optern prcessrs different the same ndes, nde, cnnected via shared Fr the Arina/Maiz cmparisn and MPI purpses, implementatin we als include prvided these cnfiguratins by HP. that d nt Gigabit GM-MN. Myrinet-2000 SHMEM-MN. shared memry. cluster, Tw netwrk. Tw IBM but t IBM PwerPC the PwerPC MareNstrum: prcessrs prcessrs in different in the same ndes, nde, cnnected cnnected crrespnd via via t The GCC Results can see MPI family. that: f implementatin experiments are is summarized MPICH-GM, in prvided Fig. 1 (delay) by Myricm. and in Fig. Cmpilers 2 (thrughput). are f the in small applicatins perfrmance cases. and medium Delays a single f messages. are the very nde. MPI small, implementatins S it is and a gd thrughput idea ver t is shared run very small-scale high especially memry is MPI very gd fr We - The implementatin the Althugh quite peak VAPI lw, thrughput Gigabit cmpared intercnnectin fr Ethernet is t slightly Optern Infiniband. prvides can lwer. be prvides used very fr gd less MPI delay levels applicatins, than f perfrmance. that the the perfrmance Itanium, The but is

3 - The Myrinet-2000 netwrk used in the MareNstrum prvides much better results Infiniband. than Gigabit Ethernet; hwever, it des nt reach the gd results f VAPI-I Fig Summary f experiments SHMEM-I scales with in bth the axes. IMB. The Delays lwer, fr the different better. message sizes. Lgarithmic VAPI-O 100 SHMEM-O TCP-GE-O Message size (Bytes) Delay (us) GM-MN SHMEM-MN VAPI-I Fig. 2 Summary 100 f experiments SHMEM-I scales with in the bth IMB. axes. Thrughput The higher, fr the different better. message sizes. Lgarithmic VAPI-O 10 SHMEM-O TCP-GE-O GM-MN 1 SHMEM-MN Message size (Bytes) Thrughput (Mbytes/s)

4 evaluate derived and NPB-2 They typical NAS frm the Parallel perfrmance cmputatinal Benchmarks f fluid parallel (NPB) dynamics supercmputers. are a (CFD) small set applicatins, f The prgrams benchmarks, cnsist designed which f five t are help three pseud-applicatins. The NPB cme several flavrs. In particular, kernels size. are are intended MPI-based t be surce-cde run with little implementatins n tuning, and written apprximate and distributed the perfrmance by NAS. the applicatins user can expect are cmpiled t btain fr fr a a given prtable number parallel f prcesses, prgram. and a given prblem a NAS applicatin class We A have quite chsen small, 64 but prcesses mre adequate (and always t test allcate the impact ne CPU f the per netwrk MPI prcess), n speed. Benchmarks have (Infiniband) They PwerPC Arina, run are the summarized applicatins prcessrs) netwrk. experiments f Results in have the Fig. n been Mare The (16x4) cmpiled given Nstrum. same Itanium2 as experiments generated using prcessrs, versins by have the 7 benchmarks: and been using 9 run f the Intel in MPI 16 in cmpilers. ndes ver Mp/s. VAPI (64 We NPB Class A.64 running n 16x4 prcessrs It the Icc v7 than is nly interesting thse exceptin cmpiled t see f with FT, the Fig. imprtance versin prgrams 3 Summary 7. cmpiled f f the experiments cmpiler with versin with in the 9 achieved NPB are faster perfrmance. (r much faster) With Icc v MN bt cg ep ft is lu mg sp Mp/s

5 The PwerPC In Arina/Maiz MareNstrum has prcessrs. 88 pwerful and Still, Itanium2 Arina/Maiz mst prcessrs, f the represent technlgy while different the used MareNstrum classes bth f clusters machines: incrprates is similar, current used terms can perfrm f raw cmmunicatin sme cmparisn perfrmance, tests. the Infiniband netwrk used the using by MPI-based clusters parallel ffers very applicatins. gd levels The f MPI perfrmance, implementatin which wrks can be very efficiently well and allcated shared t memry; same nde. small (with up t 4 prcesses) the best when Cnclusins Gigabit gd f implementatin Regarding shws interest as hw Ethernet that a the cmparisn f experiments a higher f specific-purpse Myrinet an ptin perfrmance f the with available when new the netwrk, building f Myrinet-10G NPB, runs Arina s at such cheap 2 cmparisn CPU Gb/s. as with clusters, Myrinet and Infiniband, intercnnectin f but r Arina Infiniband. its but perfrmance vs. the MareNstrum netwrk nly It wuld is results nt be as be We evaluated much explited gratefully better by acknwledge results large applicatins. Arina. the supprt Obviusly, f these ptential institutins, f the that MareNstrum prvided the can machines nly Wrk TIN C dne (Spain) with this the wrk: supprt f the Spanish Ministeri de Educación y Ciencia, grant Acknwledgements [1] [2] [3] [4] Intel NAS Infiniband Myricm, Parallel MPI Inc. Trade Benchmarks. Assciatin. Available (February 16, 16, 2007) at at References - Barcelna Scientific Cmputing Supercmputing Service, Center The / University Centr Nacinal f the Basque de Supercmputación Cuntry

6 Arina (update Infrmatin is 18 4) a service Cluster cmpute perating taken HP frm Integrity Server with 88 CPUs February Itanium2 16, prcessrs and Red Hat Linux AS 4 system. These are sme detailed characteristics f Arina: nde ndes fr 1.3 lgin running and Rx4640 at cmpilatin 1.3 Ghz. 4 GB Rx2600 HP, HP, f each RAM Chipset with per 4 nde. HP. 4 Itanium 22 prcessrs Appendix A: characteristics f Arina and Maiz Maiz 4 cmpute ndes, Rx7640 Chipset, each with 4 Itanium 2 prcessrs (update is Infiniband Gigabit Peak running intercnnectin at 1.6 Ghz. netwrk 16 GB f (Vltaire). RAM per nde. 1 Ethernet intercnnectin netwrk. 4 perfrmance (Rpeak): 496 Gflps. GB a Cluster DL585 Server with 40 CPUs Optern and Red Hat Linux AS 4 Infiniband Gigabit 4) service perating system. lgin These and are cmpilatin, sme detailed DL385. characteristics f Maiz: Peak cmpute f perfrmance RAM Ethernet intercnnectin ndes, nde. intercnnectin DL585, (Rpeak): each netwrk 192 with Gflps. netwrk (Vltaire) 4 dual cre Opterns running at 2.4 Ghz. 16

7 the MareNstrum Infrmatin taken frm February 16, BladeCenter, March 2004 the Spanish gvernment and IBM signed an agreement t build ne f See fastest large belw demand cmputers a summary a is Linux a f supercmputer scientific in system f Eurpe. the system: and prjects. In a based July Myrinet 2006 n prcessrs intercnnectin. its capacity PwerPC, has been the increased architecture due t be Appendix B: characteristics f the Mare Nstrum MareNstrum has Myrinet racks and and takes Gigabit up a Ethernet space f 120m2. - Peak Intercnnectin Linux: TB + Perfrmance 90 IBM f SuSe TB main Pwer Distributin f memry disk netwrks: f PC strage 94,21 970MP Teraflps prcessrs at 2.3 GHz (2560 JS21 blades)