Future Exascale Supercomputers

Transcription

1 Future Exascale Supercomputers Prof. Mateo Valero Director Top10 Rank Site Computer Procs Rmax Rpeak 1 RIKEN Advanced Institute for Computational Science (AICS) Fujitsu, K computer, SPARC64 VIIIfx 2.0GHz, Tofu interconnect 2 Tianjin, China XeonX5670+NVIDIA Oak Ridge Nat. Lab. Crat XT5,6 cores Shenzhen, China XeonX5670+NVIDIA GSIC Center, Tokyo XeonX5670+NVIDIA DOE/NNSA/LANL/SNL Cray XE6 8-core 2.4 GHz NASA/Ames Research Center/NAS SGI Altix ICE 8200EX/8400EX, Xeon HT QC 3.0/Xeon 5570/ Ghz, Infiniband DOE/SC/LBNL/NERSC Cray XE6 12 cores Commissariat a l'energie Bull bullx super-node Atomique (CEA) S6010/S DOE/NNSA/LANL QS22/LS21 Cluster, PowerXCell 8i / Opteron Infiniband

2 Parallel Systems Interconnect (Myrinet, IB, Ge, 3D torus, tree, ) Node Node Node * Node * Node * Node ** Node ** Node ** Node Node Node Memory SMP multicore IN multicore multicore multicore homogeneous multicore (BlueGene-Q chip) heterogenous multicore general-purpose accelerator (e.g. Cell) GPU FPGA ASIC (e.g. Anton for MD) Network-on-chip (bus, ring, direct, ) 3 Riken s Fujitsu K with SPARC64 VIIIfx Homogeneous architecture: Compute node: One SPARC64 VIIIfx processor 2 GHz, 8 cores per chip 128 Gigaflops per chip 16 GB memory per node Number of nodes and cores: 864 cabinets * 102 compute nodes/cabinet * (1 socket * 8 CPU cores) = cores. 50 by 60 meters Peak performance (DP): cores * 16 GFLOPS per core = PFLOPS Linpack: PF 93% efficiency. Matrix: more than rows!!! 29 hours and 28 minutes Power consumption 12.6 MWatt, 0.8 Gigaflops/W 4 2

3 Looking at the Gordon Bell Prize 1 GFlop/s; 1988; Cray Y-MP; 8 Processors Static finite element analysis 1 TFlop/s; 1998; Cray T3E; 1024 Processors Modeling of metallic magnet atoms, using a variation of the locally self-consistent multiple scattering method. 1 PFlop/s; 2008; Cray XT5; 1.5x10 5 Processors Superconductive materials 1 EFlop/s; ~2018;?; 1x10 8 Processors?? (10 9 threads) Jack Dongarra 5 6 3

4 Nvidia GPU instruction execution MP 1 MP 2 MP 3 MP 4 instruction 1 instruction 2 Long latency 3 Instruction 4 Mexico SBAC-PAD, DF, November, Vitoria October th, Potential System Architecture for Exascale Supercomputers System attributes Difference System peak 2 Pflop/s 200 Pflop/s 1 Eflop/sec O(1000) Power 6MW 15 MW ~20 MW System memory 0.3 PB 5 PB PB O(100) Node performance Node memory BW Node concurrency Total Concurrency Total Node Interconnect BW 125 GF 0.5 TF 7 TF 1 TF 10 TF O(10) O(100) 25 GB/s TB/sec 0.4 TB/sec 4 TB/sec O(100) TB/sec 12 O(100) O(1,000) O(1,000) O(10,000) O(100) O(1000) 225,000 O(10 8 ) O(10 9 ) O(10,000) 1.5 GB/s 20 GB/sec 200 GB/sec O(100) MTTI days O(1day) O(1 day) - O(10) EESI Final Conference Mexico DF, November, Oct. 2011, 2011 Barcelona 8 4

5 2. To faster air plane design Boeing: Number of wing prototypes prepared for wind-tunnel testing Date Airplane B757/B767 B777 B787 # wing prototypes Plateau due to RANS limitations. Further decrease expected from LES with ExaFlop EESI Final Conference Mexico DF, November, Oct. 2011, 2011 Barcelona 9 Diseño del Airbus

6 2. To faster air plane design Airbus: "More simulation, less tests From A380 to A350-40% less wind-tunnel days - 25% saving in aerodynamics development time - 20% saving on wind-tunnel tests cost thanks to HPC-enabled CFD runs, especially in high-speed h regime, providing even better representation of aerodynamics phenomenon turned into better design choices. Acknowledgements: E. CHAPUT (AIRBUS) EESI Final Conference Mexico DF, November, Oct. 2011, 2011 Barcelona Oil industry EESI Final Conference Mexico DF, November, Oct. 2011, 2011 Barcelona 12 6

7 Diseño del ITER TOKAMAK (JET) 13 Fundamental Sciences EESI Final Conference Oct. 2011, Barcelona 14 7

8 Materials: a new path to competitiveness On-demand materials for effective commercial use Conductivity: energy loss reduction Lifetime: corrosion protection, e.g. chrome Fissures: saftety insurance from molecular design Optimisation of materials / lubricants less friction, longer lifetime, less energy-losses Industrial need to speed up simulation from months to days EESI Final Conference, Mexico Oct. DF, 2011, November, Barcelona 2011 All atom 15 Multi-scale Exascale enables simulation of larger and realistic systems and devices Life Sciences and Health Population Organ Tissue Cell Macromolecule Small Molecule Atom EESI Final Conference, Mexico Oct. DF, 2011, November, Barcelona

9 Supercomputación, teoría y experimentación 17 Cortesia de IBM Supercomputing, theory and experimentation 18 Cortesia de IBM 9

10 Holistic approach Towards exaflop Applications Job Scheduling Programming Model Run time Interconnection Processor/node architecture Load Balancin ng Comput. Complexity Moldability Resource awareness User satisfaction Address space Async. Algs. Dependencies Work generation Locality optimization Concurrency extraction Topology and routing External contention NIC design Run time support Memory subsystem Hw counters Core Structure Pflop/s systems planned Fujitsu Kei 80,000 8-core Sparc64 VIIIfx processors 2 GHz, (16 Gflops/core, 58 watts 3.2 Gflops/watt), 16 GB/node, 1 PB memory, 6D mesh-torus, 10 Pflops Cray's Titan at DOE, Oak Ridge National Laboratory Hybrid system with Nvidia GPUs, 1 Pflop/s in 2011, 20 Pflop/s in 2012, late 2011 prototype $100 million 20 10

11 10+ Pflop/s systems planned IBM Blue Waters at Illinois 40,000 8-core Power7, 1 PB memory, 18 PB disk, 500 PB archival storage, 10 Pflop/s, 2012, $200 million IBM Blue Gene/Q systems: Mira to DOE, Argonne National Lab with 49,000 nodes, 16-core Power A2 processor (1.6-3 GHz), 750 K cores, 750 TB memory, 70 PB disk, 5D torus, 10 Pflop/s Sequoia to Lawrence Livermore National Lab with nodes (96 racks), 16-core A2 processor, 1.6 M cores (1 GB/core), 1.6 Petabytes memory, 6 Mwatt, 3 Gflops/watt, 20 Pflop/s, Japan Plan for Exascale Heterogeneous, Distributed Memory GigaHz KiloCore MegaNode system K Machine 10K Machine 100K Machine 10 PF 100 PF ExaFlops Feasibility Study ( ) Exascale Project ( ) Post-Petascale Projects 22 11

12 Thanks to S. Borkar, Intel 23 Thanks to S. Borkar, Intel 24 12

13 Nvidia: Chip for the Exaflop Computer Thanks Bill Dally 25 Nvidia: Node for the Exaflop Computer Thanks Bill Dally 26 13

14 Exascale Supercomputer Thanks Bill Dally 27 BSC-CNS: International Initiatives (IESP) Improve the world s simulation and modeling capability by improving the coordination and development of the HPC software environment Build an international i plan for developing the next generation open source software for scientific high-performance computing 28 14

15 Back to Babel? Book of Genesis Now the whole earth had one language and the same words The computer age Fortran & MPI Come, let us make bricks, and burn them thoroughly. "Come, let us build ourselves a city, and a tower with its top in the heavens, and let us make a name for ourselves And the LORD said, "Look, they are one people, and they have all one language; and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them. Come, let us go down, and confuse their language there, so that they will not understand one another's speech." ++ Fortress Cilk++ X10 CUDA Sisal HPF StarSs RapidMind Sequoia CAF OpenMP UPC ALF SDK Chapel MPI Thanks to Jesus Labarta 29 You will see. in 400 years from now people will get crazy New generation of programmers Multicore/manycore Architectures Parallel Programming New Usage models Source: Picasso -- Don Quixote Dr. Avi Mendelson (Microsoft). Keynote at ISC

16 Different models of computation. The dream for automatic parallelizing compilers not true so programmer needs to express opportunities for parallel execution in the application SPMD OpenMP 2.5 Nested fork-join OpenMP 3.0 DAG data flow And asynchrony (MPI and OpenMP too synchronous): Collectives/barriers multiply effects of microscopic load imbalance, OS noise, Huge Lookahead &Reuse. Latency/EBW/Scheduling 31 StarSs: generates task graph at run time #pragma css task input(a, B) output(c) void vadd3 (float A[BS], float B[BS], float C[BS]); #pragma css task input(sum, A) output(b) void scale_add (float sum, float A[BS], float B[BS]); #pragma css task input(a) inout(sum) void accum (float A[BS], float *sum); Task Graph Generation for (i=0; i<n; i+=bs) // C=A+B vadd3 ( &A[i], &B[i], &C[i]);... for (i=0; i<n; i+=bs) // sum(c[i]) accum (&C[i], &sum);... for (i=0; i<n; i+=bs) // B=sum*E scale_add (sum, &E[i], &B[i]);... for (i=0; i<n; i+=bs) // A=C+D vadd3 (&C[i], &D[i], &A[i]);... for (i=0; i<n; i+=bs) // E=C+F vadd3 (&C[i], &F[i], &E[i]);

17 StarSs: and executes as efficient as possible #pragma css task input(a, B) output(c) void vadd3 (float A[BS], float B[BS], float C[BS]); #pragma css task input(sum, A) output(b) void scale_add (float sum, float A[BS], float B[BS]); #pragma css task input(a) inout(sum) void accum (float A[BS], float *sum); Task Graph Execution for (i=0; i<n; i+=bs) // C=A+B vadd3 ( &A[i], &B[i], &C[i]);... for (i=0; i<n; i+=bs) // sum(c[i]) accum (&C[i], &sum);... for (i=0; i<n; i+=bs) // B=sum*E scale_add (sum, &E[i], &B[i]);... for (i=0; i<n; i+=bs) // A=C+D vadd3 (&C[i], &D[i], &A[i]);... for (i=0; i<n; i+=bs) // E=C+F vadd3 (&C[i], &F[i], &E[i]); StarSs: benefiting from data access information Flat global address space seen by programmer Flexibility to dynamically traverse dataflow graph optimizing Concurrency. Critical path Memory access Opportunities for Prefetch Reuse Eliminate i antidependences d (rename) Replication management 34 17

18 StarSs: Enabler for exascale Can exploit very unstructured parallelism Not just loop/data parallelism Easy to change structure Supports large amounts of lookahead Not stalling for dependence satisfaction Allow for locality optimizations to tolerate latency Overlap data transfers, prefetch Reuse Nicely hybridizes into MPI/StarSs Propagates to large scale the node level dataflow characteristics Overlap communication and computation A chance against Amdahl s law Support for heterogeneity Any # and combination of CPUs, GPUs Including autotuning Malleability: Decouple program from resources Allowing dynamic resource allocation and load balance Tolerate noise Data-flow; Asynchrony Potential is there; Can blame runtime Compatible with proprietary low level technologies StarSs: history/strategy/versions Basic SMPSs must provide directionality argument Contiguous, non partially overlapped Renaming Several schedulers (priority, it locality, ) lit No nesting C/Fortran MPI/SMPSs optims. SMPSs regions C, No Fortran must provide directionality argument ovelaping &strided Reshaping strided accesses Priority and locality aware scheduling OMPSs C/C++, Fortran under development OpenMP compatibility (~) Dependences based only on args. with directionality Contiguous args. (address used as centinels) Separate dependences/transfers Inlined/outlined pragmas Nesting SMP/GPU/Cluster No renaming, Several schedulers: Simple locality aware sched, 36 18

19 Multidisciplinary top-down approach Performance analysis and prediction tools Application and algorithms Interconnect Programming models Investigate solutions Load to these and other problems Power Processor and node balancing Power (MW) $3M Computer Center Power Projections Cooling Computers $9M $17M 37 $23M $31M Year 38 19

20 Green/Top 500 November 2011 Green500 Top500 _Rank _Rank Mflops/Watt Power Site Computer ,48 85,12 IBM - Rochester BlueGene/Q, Power BQC 16C 1.60 GHz, Custom ,48 85,12IBM Thomas J. Watson Research Center BlueGene/Q, Power BQC 16C 1.60 GHz, Custom ,09 170,25 IBM - Rochester BlueGene/Q, Power BQC 16C 1.60 GHz, Custom ,56 340,5 DOE/NNSA/LLNL BlueGene/Q, Power BQC 16C 1.60 GHz, Custom ,86 38,67IBM Thomas J. Watson Research Center NNSA/SC Blue Gene/Q Prototype ,32 47,05 Nagasaki University DEGIMA Cluster, Intel i5, ATI Radeon GPU, Infiniband QDR ,26 81,5 Barcelona Supercomputing Center Bullx B505, Xeon E5649 6C 2.53GHz, Infiniband QDR, NVIDIA ,11 108,8 TGCC / GENCI Curie Hybrid Nodes - Bullx B505, Xeon E GHz, Infiniband QDR Institute of Process Engineering, Chinese Mole-8.5 Cluster, Xeon X5520 4C 2.27 GHz, Infiniband QDR, ,7 515,2 Academy of Sciences NVIDIA 2050 GSIC Center, Tokyo Institute of , ,8 Technology ,96 126,27Virginia Tech ,54 117,91 Georgia Institute of Technology CINECA / SCS - SuperComputing , Solution NVIDIA ,87 76,25 Forschungszentrum Juelich (FZJ) HP ProLiant SL390s G7 Xeon 6C X5670, Nvidia GPU, Linux/Windows SuperServer 2026GT-TRF, Xeon E5645 6C 2.40GHz, Infiniband QDR, NVIDIA 2050 HP ProLiant SL390s G7 Xeon 6C X Ghz, nvidia Fermi, Infiniband QDR idataplex DX360M3, Xeon E5645 6C 2.40 GHz, Infiniband QDR, NVIDIA 2070 idataplex DX360M3, Xeon X5650 6C 2.66 GHz, Infiniband QDR, NVIDIA 2070 Xtreme-X GreenBlade GB512X, Xeon E5 (Sandy Bridge - EP) 8C 2.60GHz, Infiniband QDR ,19 198,72 Sandia National Laboratories RIKEN Advanced Institute for , ,89 Computational Science (AICS) K computer, SPARC64 VIIIfx 2.0GHz, Tofu interconnect , National Supercomputing Center in Tianjin NUDT YH MPP, Xeon X5670 6C 2.93 GHz, NVIDIA , DOE/SC/Oak Ridge National Laboratory Cray XT5-HE Opteron 6-core 2.6 GHz National Supercomputing Centre in , Shenzhen (NSCS) Dawning TC3600 Blade System, Xeon X5650 6C 2.66GHz, Infiniband QDR, NVIDIA 2050 Mexico DF, November, SBAC-PAD, 2011 Vitoria October 28th, Green/Top 500 November 2011 BSC, Xeon 6C, NVIDIA 2090 GPU Top500 rank Nagasaki U., Intel i5, ATI Radeon GPU IBM and NNSA, Blue Gene/Q Mflops/watt Mwatts/Exaflop 2026, , , , Mflops/watt >1 GF/watt MF/watt MF/watt 40 20