HPC performance applications on Virtual Clusters

Transcription

1 Panagiotis Kritikakos EPCC, School of Physics & Astronomy, University of Edinburgh, Scotland - UK [email protected] 4 th IC-SCCE, Athens 7 th July 2010

2 This work investigates the performance of (Java) multi-threaded applications that execute on multi-core virtual machines that can perform as virtual clusters. Our investigation and comparison is carried across individual physical and virtual machines nodes with identical hardware configuration. The work presents an evaluation of various virtualisation configuration capabilities that we applied during the experimental process. The project was supported by EPCC at The University of Edinburgh and was carried out as an Honours (final year) Project in the undergraduate degree of BEng Internet Computing of the Edinburgh Napier University.

3 Until recently... Slowly increasing computational power Expensive multi-core systems Performance penalty of virtualisation HPC applications on large scale Grid computing...from today to the future... Emerging computational power Multi-core systems as a standard Virtualisation support by default HPC applications on various scales Cloud computing

4 Technical specifications 16 CPUs 16GB RAM SCSI storage Xen Hypervisor Scientific Linux Java Grande Benchmarking Suite Multi-threading disk I/O and networking benchmarks Virtual Clusters setup 4 x VMs (4 cores, 3.5GB RAM) 2 x VMs (8 cores, 7.5GB RAM) 4 x VMs (16 cores, 3.8GB RAM) Experiments and statistics 10 execution loops starting from 10 threads and finishing with 100 threads Comparing results against physical system with Linux SMP kernel Used of the α > 0.95 significance level with Student's t-test

5 Software specifications Java Grande Benchmarking Suite - Thread synchronisation RayTracer Moldyn Multi-threaded disk I/O Multi-threaded TCP I/O Multi-threaded UDP I/O Operating System VM Benchmark Linux VM Benchmark Linux Virtual Machine Monitor Scientific Linux 5 x86_64 Virtualisation platform Xen Hypervisor Xen Hypervisor Physical hardware (CPU, Memory, Storage, Network)

6 Thread syncronisation (computationally intensive) Syncs performance on range of cores Significant difference in favour of the virtual machines P-value = 1.55 (10 threads)

7 Thread syncronisation (computationally intensive) Syncs performance on shared cores No statistical significant difference

8 Series (computationally intensive) No statistical significant difference

9 Sparse (memory intensive) No statistical significant difference up to 40 threads. From 50 -> 100, P-values = 1.71, 9.24, 6.82, 8.29, 4.44

10 RayTracer (memory and computationally intensive) Statistical significant difference in favour of the physical machine P-value = 1.46 (10 threads)

11 Moldyn (computationally intensive) Statistical significant difference only at 100 threads P-value = 2.61

12 Disk I/O Statistical significant difference in favour of the virtual machine P-value = 8.46 (100 threads)

13 Disk I/O Statistical significant difference in favour of the physical machine P-value = 9.87 (30 threads)

14 TCP I/O Statistical significant difference in favour of the physical machine P-value = 7.90 (10 threads)

15 UDP I/O No statistical significant difference

16 Some conclusions... Range of CPUs performance = Dedicated CPUs performance Rage of/dedicated CPUs performance > Shared CPUs performance Improved memory algorithms Memory chunks size play major role on performance Well performing disk I/O scheduling algorithm = Fast writes Poor I/O buffer rings = Slow reads Network handshaking and integrity checking overhead (TCP) Improved performance on connection-less network protocols (UDP)

17 Why using virtualisation? Reliability / Availability / Fault tolerance Portability Productivity / Development Management and some more Security Economical Greener The wide virtualisation support in modern commodity hardware shows great promise for virtualisation to become one of the default ICT infrastructure technologies of the future.

18 ?Thank you Presentation