SA2 REPORT. First report from Activity SA2 (M8): Provide a suitable Software environment for users

Transcription

1 Subject: First report from Activity SA2 (M8): Provide a suitable Software environment for users Author(s): Alejandro Soba Distribution: Public Introduction The main target of Service Activities 2 (SA2, High Performance Computing (HPC) Infrastructure deployment) is to secure and support access for the code developers and code owners within JRA2 to the facilities, massively parallel supercomputers (MPSCs), and customer services at a set of such supercomputer centres (that is, in Helsinki, in Barcelona, and (UEDIN) in Edinburgh). The code developers within JRA2 are focussed on optimizing codes from within the plasma fusion community that need MPSCs to run efficiently. In order for the optimization work to function properly the code developers must have access to suitable computers for testing, debugging, and running. All the three centres listed above (, and ) can provide suitable computers, programming environment and expertise for JRA2 code developers. SA2 is focused on taking advantage of the existing service activity at the computer centres. The code developers of JRA2 are divided in to three groups each associated with one of the computer centres. This arrangement allows the code developers to use all the hardware, software and expertise services provided by the computer centres [7]. SA2 uses the expertise within the JRA2 partners, as well as the associated hardware, to provide access to HPC or MPSC resources for fusion code owners and developers. Fundamentally, SA2 should enable code owners to perform large scale simulation runs to investigate and produce world leading fusion science. The final list of codes available to install and optimize in the EUFORIA project is summarized in table I. In this table a first view characterization of the codes has been made in order to allow planning and coordination between the three centres in JRA2/SA2. Besides that, there is information about the languages, parallelization mode and a first projection of the necessary effort required for each code. Finally the centres responsible for the codes are listed in the last column. FP REPORT 1/6

2 Code BIT1 CENTORI EIRENE ELMFIRE ERO ESEL Table I: Main Characterization of the codes available for EUFORIA project Optim ization Needs parallelizatio n (plan for parallelizatio n exists) No, runs on >1024 No, code is already fully parallelized No, runs on 256 No? Runs on 32*10 (Vector?) ( version exist, MPI version is under development ). Shares code base with TYR. Gridparallel version needed No of lines/ subroutin es / / / / Language Estimated effort needed Assigned to C Low 95 Low 90 / C C / C / 77/90 Medium Low Medium Medium GEM No 4000 / 20 Low GENE No, scales well to / 105 Low ISDEP No No 1000 / 4 C Low SOLPS TYR Abo, parallelizatio n plans were looked at and rejected No, scales up to 32. Also. Shares code base with ESEL / / 150 High C Medium FP REPORT 2/6

3 HPC Resources: Programming environment HPC Resources: Programming Environment This activity will provide access to the programming environment for the code developers of JRA2 and the fusion code owners at the computer centres. The programming environment includes e.g. licence to use compilers, debuggers and, if available, software profiling tools. The aim will be to ensure that EUFORIA users have the appropriate tools to utilise these HPC resources. Access to run time on the three centres is provided, free of charge, to such an extent that can be motivated for testing of the optimized codes [7]. In table II an extensive list of compilers, the parallel environments, the libraries installed and the batch systems of each HPC centre are summarized. (The respective web pages of each centre and users guide of each supercomputer are referred in order to see a more extensive explanation of each topic). All the centres have an extensive expertise, user s guide, examples of use of batch system, compilers, parallel environments, etc. to provide the necessary support to any users. Table II: Description of the available HPC resources Centres Resources Hardware Resources Software Cray Supercomputer: Cray XT4 (2024 cores, performance 10,5 Tflops). This system is known as Louhi. [1,5] HP XC-Super cluster, 2048 cores, peak performance 10.6 Tflops. [1] Compilers Portland Group (PGI) GNU PathScale Parallel environments MPI Cray SHMEM library Cray Supercomputer: 60 Cray XT4 cabinets (11,323 cores, 33.2Tb of memory, expected peak performance of 59Tflops). This system is known as HECToR [2,6] Libraries ACML LibSci FFTW PETSc Batch System Portable Batch System Professional (PBS Pro). HECToR compilers PGI, PathScale and GNU HECToR Parallel enviroments MPI HECToR Libreries Cray provides libraries XT-LibSci Linear algebra: BLAS and LAPACK BLACS FP REPORT 3/6

4 IBM BladeCenter JS21 Cluster, PPC 2.3 GHz. Peak performance 94 Tflops, 20 TB of memory. [3,4] ScaLAPACK SuperLU Iterative Refinement Toolkit AMD Core Math Library - ACML Cray FFT interface Random Number Generators PGI issue FFTW PETSc NAG Library, Mark 21 NAG SMP Library, Mark 21 NAG Parallel Library 3 HECToR Batch PBS pro to schedule batch jobs Compilers IBM XL GNU Compilers Parallel environments MPI Libreries ATLAS with ALTIVEC BLACS BLAS BOOST ESSL/PESSL CFITSIO FFTW GLPK GNU-GSL GOTO Optimized for PPC970 HDF4 HDF5 IOAPI LAPACK LP_SOLVE MASS NCARG NETCDF SBML SCALAPACK Batch Slurm+MOAB The first step into the JRA2 work package was the installation/compilation of the codes into the supercomputers, and the realization of an extensive profiling of them. This first step was completely finishing in June of 2008 and the requirements of each code in reference to compilers, libraries, debuggers and profiling were captured at each centre. Table III summarized the minimal list of Libraries needed for each code to run with its best performance. All the required libraries are available in the respective HPC centres, with the exception of some of the machine specific libraries (such as ESSL and PESSL which are only available on IBM machines) where alternative libraries are available. FP REPORT 4/6

5 Table III: Libraries needed for all the codes of EUFORIA project Codes HPC centre Libraries needed CENTORI --- GEM FFTW ERO IBM ESSL and PMAPI EIRENE --- ELMFIRE GSL, BLAS, PETSC, PESSL GENE ESSL, MKL or FFTW BLAS/LAPACK PETSC/SLEPC BIT1 X11 Esel HDF4(version 4.2r2) Tyr HDF4(version 4.2r2) FFTW 3.1(version 3.1.2) Isdep --- SOLPS NCARG (version 5.0.0) NetCDF (version 3.6.2) ACML (version 4-0-1) (or some other BLAS/LAPACK library) X11 Conclusion At the moment, the final eleven codes in EUFORIA were installed / compiled and performed and all the requirements of Libraries, compilers, and debugging - profiling tools are available in the respective HPC centres. Each centre has all the necessary resources to provide services to users or developers in order to realize testing, debugging, and running of those codes. The requirement within JRA2 of HPC resources are completely satisfied in each centre and the collaboration and communication between the project partners involved in the project facilitates the interchange of information, supports and knowledge in order to provide adequate answer to any needs. Apart of that and based on the software requirements needs for fusion as detailed within the ITM-TF software inventory report, the EUFORIA HPC infrastructure should cover a broader range of modelling needs than the current scope of edge and core turbulence and transport. The state of each centre, in relation to software resources and behaviour of the codes, ensures the infrastructure needed to start on time the next step of optimization within JRA2 and ensure the run time access to supercomputers within SA2. FP REPORT 5/6

6 References [1] [2] [3] [4] MareNostrum User's Guide Barcelona Supercomputing Center CNS [5] Louhi User's Guide, the 2nd Edition Release 2.0, Feb 2008 [6] User Guide to the HECToR Service (Version 0.1) [7] Grant agreement for: Combination of Collaborative projects & Coordination and support actions. Annex I - Description of Work (2007) FP REPORT 6/6