Jan Wagner, Jouko Ritakari, Metsähovi Radio Observatory

Size: px

Start display at page:

Download "Jan Wagner, Jouko Ritakari, Metsähovi Radio Observatory"

Edward Lambert
7 years ago
Views:

1 A Playstation3 based software correlator for evlbi Software correlation on the Cell processor Jan Wagner, Jouko Ritakari, Metsähovi Radio Observatory

2 Software correlation (1/2) Software correlators have been used in production around the world, and are now slowly appearing in the EVN, too. Hardware correlators: - power efficient high-perf number cruncher, fixed capabilities Software correlators: - easily extendable, reconfigurable, scalable - data via IP protocols (perhaps evlbi) - no custom computing hardware; commodity PC cluster - free existing s/w correlators can be modified, adapted to own requirements DiFX from Swinburne is a popular, well written software correlator. DiFX was used as a basis in Metsähovi development.

3 Software correlation (2/2) DiFX was written by A. Deller, Swinburne: - production quality - used for VLBI, geo-vlbi, pulsar binning - uses Intel's optimized math library (IPP) - cluster architecture (normal MPI) leads to easy scaling to additional stations Original DiFX runs only on Intel. End of 2006, the first version of the IBM Cell Broadband Engine (processor) was released. IBM Cell beats current Intel in: - significantly more GFLOPs - better FLOPS per Watt, lower cost - better memory bus architecture Porting DiFX to Cell seemed attractive!

4 The Cell Broadband Engine IBM Cell is a heterogenous multi-core processor: 1 scaled-down PowerPC core with AltiVec vector unit (PPU) 8 special Synergetic Proessing Unit vector processors (SPU) Total computing power Ghz 35W Cores are on an interconnect bus (~0.3 TB/s) Constrained by memory port (~25 GB/s) Cell is available in Cell Blade QS20, Playstation 3, and various computing boards.

5 Cell processor figures Typically instructions take ~1ns, ~5 cycles 1024 point FFT: 6us with IBM SDK c2c FFT, 9us with Metsähovi patched Cell FFTW Complex conjugate, multiply and accumulate for 4 complex pairs only four instructions Calculating four sin/cos pairs takes five instructions (for predetermined quadrant)

6 Sony Playstation 3 Metsähovi bought a PS3 in January 2007 for evaluation purposes. It's a low-cost IBM Cell platform install Linux and IBM Cell SDK. Expecting to need 1/6 th of PS3s vs Pcs. Began porting DiFX in January, with pauses. First stage of porting DiFX to Cell: - Replaced closed-source Intel IPP in DiFX with platform-independent math functions - Runs on AMD, Cell, others - Completed in February 2007

7 Results of first DiFX port stage With the Intel IPP maths replaced, DiFX of course got slower. Original Intel IPP DiFX 15s 1 Intel Dual Core, 3.2 GHz 40s Intel Pentium GHz Platform-independent DiFX 110s Intel Pentium GHz 220s PS3 Cell PPU unit 3.2 Ghz The next port stage was to begin using Cell SPE cores to get to the real Cell computing power. 1 Times for test data set from the DiFX homepage (4 stations x 40MB).

8 Results of ongoing DiFX port Total wallclock time 48s, or 36s without local disk I/O. Intel was 40s on P4, 15s on Dual Core. Core::processdata() 21s 100% PPU baseline MAC Mode::process() 7s 70% SPU fringe rot, FFT,... Mode::unpack() 7s either PPU or SPU raw to float Disk reading 13s PPU Others 1s PPU

9 Own core test results Written 2 weeks ago. Raw data first streamed to one SPE for unpacking, it streams floats onwards to processing SPEs Current throughput of core routines per single SPE: raw data to float 84 Gbps ~2.5 Gsps for 2 bit in, float out fringe rotation w/ 60 Gbps 960 Mcsps quadrature oscillator 1024 point c2c FFT 10.5 Gbps 170k FFT/s or ~135 Mcsps complex MAC 95 Gbps ~3x500 Mcsps rotation, FFT, MAC 7.1 Gbps ~110 Mcsps(x5..6)

10 Conclusions and outlook Own correlator core: one PS3 could correlate > 1Gbps real time, but LAN limited a 16 PS3 cluster should handle 10 station 1 Gbps real time still some more programming to be done... DiFX TODO: move baseline processing from PPU onto all SPEs refactor DiFX correlation code, so it works in streamed processing fashion (much faster!), just like our own core The final core(s) should be available in 1 month

SPEEDUP - optimization and porting of path integral MC Code to new computing architectures

SPEEDUP - optimization and porting of path integral MC Code to new computing architectures V. Slavnić, A. Balaž, D. Stojiljković, A. Belić, A. Bogojević Scientific Computing Laboratory, Institute of Physics