High Performance Linux Cluster and Multicore Nehalem Processors

Size: px

Start display at page:

Download "High Performance Linux Cluster and Multicore Nehalem Processors"

Britney Clarke
7 years ago
Views:

1 High Performance Linux Cluster and Multicore Nehalem Processors Zhang Xinhuai (High Performance Computing, Computer Centre) The fast and constant improvements in microprocessors have continued to enhance High Performance Computer (HPC) systems. The launch of the Intel Nehalem processors last year marked a new era for microprocessors, dramatically advancing the efficiency of HPC servers and providing unmatched computing capability. A new Linux cluster acquired in the Compute Centre last year has been equipped with Nehalem processors. It has a theoretical peak performance of 8.17 TFLOPs and a total of 4,608 GB memory. The benchmark results show that, on average, the cluster (Atlas5) with Nehalem processors X5550 at 2.66GHz is 25% faster than the cluster (Atlas3) with Intel Dualcore processors X5160 at 3.0 GHz and 19% faster than the cluster (Atlas4) with Intel Quadcore processors E5430 at 2.66GHz. 1. Beowulf Linux Cluster Since the first Beowulf cluster was setup in the world in the mid 1990, the commodity-based Linux clusters which are built using identical, commercially available computers and a high speed network have gradually replaced the large shared memory Symmetric Multiprocessing (SMP) computers or Non- Uniform Memory Access (NUMA) architecture computers which had dominated High Performance Parallel computing ten years ago. The world s TOP 500 supercomputer list in November 2009 gives us a rough idea of how far this trend has come: 83% or 417 systems out of 500 are cluster systems, while 10 years ago there were only seven clusters or 1.4% in the TOP 500 supercomputers. The driving forces for this trend, I think, are: 1) Continued and fast increases in microprocessor capacity have rendered other forms of computers almost obsolete 2) The cost of building a cluster is much lower than the traditional SMP or NUMA system 3) The speed of the cluster interconnection has increased and the latency decreased dramatically. 2. Linux cluster at HPC, Computer Centre At HPC, Compute Centre, we acquired our first Atlas series of Beowulf Linux cluster for High Performance Parallel computing in In the years after, the Linux cluster gradually became the main task force for HPC. The table below shows details of the configurations of the clusters. From the table we

2 can see how the compute power increases dramatically with the development of the new microprocessors, especially with the introduction of the multicore processor systems. Table 1: System Configuration of HPC Linux cluster at Computer Centre Cluster Name(launc h time) Atlas1(2003) Atlas2(2005) Model of Systems 6 nodes 2-way Dell PE 2650 and 12 nodes 2-way Dell PE nodes 2-way IBM HS20 Blade Processors 2.8GHz Cluster Interconne ct Total Numb er of Proce ssor/ Cores Total Memory (GB) Theoretical Peak Performance (GFlops) Myrinet Infiniband Atlas3(2007) HP Blade 460c 2- way Quadcore & 2 ways Dualcore Atlas4(2008) HP Blade BL460c, 2-way Quadcore E5430 and 4-way quadcore E7330 Atlas5(2010) HP BL460c, 2- way Quadcore Nehalem X5550 X5355, 2.66GHz & X5160, 3.0GHz E5430, 2.66 GHz & E7330, 2.40GHz X5550, 2.66GHz Infiniband Infiniband Infiniband Multicore Processors About five years ago, microprocessor designers began to acknowledge that they could no longer rely on higher clock rates and increasing Instruction-Level Parallelism (ILP) for performance increases. The increased power consumption with the increased clock rate proved to be a key challenge faced by the micro architecture designers. It was found that with 13% increase of the clock rate, the power consumption increased by about 73%. Other issues like memory bandwidth was also one of factors affecting performance improvement. On the other hand, multicore processors are ideal as they are able to improve performance while addressing the above issues of single high speed processors. A multi-core processor combines two or more processing elements (called cores) in a single package, on a single die or

3 multiple dies. The cores share the interconnects to the rest of the system and often share on-chip cache memory. The currently available dualcore or quadcore processors allow performance gains approaching two times or four times without dramatically changing the programming model or requiring specialpurpose tools and expert knowledge. Hence, multicore became the way forward. Of course, multicore processor systems open opportunities for large scale parallel computing. Just imagine if hundreds of cores are available in one processor, a two-processor personal computer will be a powerful supercomputer which can allow you to run large parallel codes with hundreds of threads. This may happen not too long in the future. However, there are also issues with the multicore system which can impair performance. For example, quad-core processors are composed of two separate dies, which means some cached data have to travel outside the processor to get from core to core. This brings out the new architectural design of the Nehalem processors. 4. Nehalem Processors In March 2009, Intel announced the release of Nehalem processors. The Nehalem processor is a singledie, 64-bit architecture with 8MB of fully shared L3 cache readily available to each of the four processor cores. The result is fast access to cache data and greater application performance. In the Intel processor, system memory is often connected to a processor through a separate I/O controller. But each Intel Nehalem processor features an integrated memory controller. The integrated memory controller, along with fast 1066MHz DDR3 ECC SDRAM, allows for enhanced system performance. Turbo Mode allows the processor to opportunistically improve performance by raising the frequency of the processor when there is thermal and electrical headroom. The magnitude of the frequency improvement is greater as the number of active cores decreases. In addition to new micro architecture performance features, Nehalem also radically changes the way memory and I/O are accessed. The Nehalem architecture eliminates the front side bus for accessing memory and I/O and moves to one where each CPU package includes an integrated memory controller and one or more high speed serial link known as Quick Path Interconnect (QPI) to access other CPU packages and I/O. The result is a massive improvement in memory and I/O bandwidth.

4 The processor cache is also revamped. The new cache hierarchy is known as Smart Cache. The Smart Cache implementation on Nehalem has two small caches included in each processor core to improve performance and scalability. A shared third level cache is included in the un-core. This allows the cache to vary in size with the number of cores, and can easily be increased in size with future implementations. The L3 cache has an inclusive policy everything in each core s L1/L2 cache must be present in the L3 cache. This keeps snoop traffic constant as the core count increases, and minimises the effective cache latency by eliminating cross-core snoops in the common case. 5. Atlas5 Cluster Performance In February 2010, we launched the new Linux cluster with Nehalem processors - the atlas5 cluster. With atlas5 cluster, we have the following improvements over the previous atlas clusters: 1) New Nehalem processors at 2.66GHz, 8 MB smart cache. 2) 36 ports Infiniband switch interconnect allows a 4:1 blocking factor in the intercommunication between nodes, which enables faster message passing between nodes for the MPI parallel applications. 3) 48 GB memory on each node provides sufficient memory for the most memory intensive applications. 4) 40tb High Performance parallel files system used as the working space for faster disk I/O access. The following is the benchmark results showing the performance improvement for some commercial and open source applications.

5 Table 1, Performance comparison between three Linux clusters Code No. of Procs Benchmark Results atlas3 atlas4 atlas5 Atlas5 Speedup over Atlas4 Atlas5 Speedup over Atlas3 Amber 16 CPU time (s) Turn Around Time (s) % 32.55% Gromacs 16 CPU time (s) Turn Around Time(s) % 16.92% serial parallel Linda Linda Linda 1 CPU time(s) Turn Around Time(s) % 11.08% 4 CPU time (s) Turn Around Time(s) % 26.01% 4 CPU time(s) Turn Around Time(s) % 4.15% 8 CPU time(s) Turn Around Time(s) % 37.48% 16 CPU time(s) Turn Around Time(s) % 50.81% Average 19.01% 25.57% It is obvious that the performance of the parallel Amber and Gromacs jobs and Gaussian 09 jobs on the atlas5 cluster is much better than the performance of those codes on the atlas3 and atlas4 clusters. The average speedup for atlas5 over atlas3 is 25% and 19% over atlas4. Let us also look at the scalability of the codes on the atlas5 cluster. Figure 1 and Figure 2 show the benchmark results for Amber, Linda and Gromacs jobs using a different number of CPU/cores. In Figure 3, we have the speed up of the three codes when they are run on the atlas5 cluster. The speedup for running Amber and Gromacs code using 32 CPUs can reach 19.2 and 24.5 respectively. For Gaussian 09 Linda, the speedup is about 8.5 for a job with 16 Linda processors.

6 Amber10 Linda Figure 1, Benchmark results of Amber10 and Linda code using different CPU/cores Gromacs Figure 2, Benchmark results of Gromacs code using different CPU/cores

7 Amber Gromacs Linda Figure 3, Speedup for different applications on Atlas5 cluster 6. Conclusion The development of microprocessors has ushered in a new era in the High Performance scientific computing area. As more and more super powerful computers are being built up, the solving of once difficult and challenging problems has now become quite easy. Multicore processors enable people to run large parallel jobs which helps to reduce computing time and increase the job efficiency.

A Study on the Scalability of Hybrid LS-DYNA on Multicore Architectures

11 th International LS-DYNA Users Conference Computing Technology A Study on the Scalability of Hybrid LS-DYNA on Multicore Architectures Yih-Yih Lin Hewlett-Packard Company Abstract In this paper, the