A Holistic Model of the Energy-Efficiency of Hypervisors

Transcription

1 A Holistic Model of the -Efficiency of Hypervisors in an HPC Environment Mateusz Guzek,Sebastien Varrette, Valentin Plugaru, Johnatan E. Pecero and Pascal Bouvry SnT & CSC, University of Luxembourg, Luxembourg 1 / 29

2 Summary 1 Introduction, Context & Motivations 2 Modeling 3 Experimental Setup & Experiments Performed 4 Results 5 Conclusion 2 / 29

3 Introduction, Context & Motivations Summary 1 Introduction, Context & Motivations 2 Modeling 3 Experimental Setup & Experiments Performed 4 Results 5 Conclusion 3 / 29

4 Introduction, Context & Motivations HPC at the Heart of our Daily Life Today... Research, Industry, Local Collectivities... Tomorrow: applied research, digital health, nano/bio techno N 4 / 29

5 Introduction, Context & Motivations Cloud Computing in an HPC context Horizontal scalability: perfect for replication/ HA (High Availability) best suited for runs with minimal communication and I/O nearly useless for true parallel/distributed HPC runs Cloud Data storage Data locality enforced for performance Data outsourcing vs. legal obligation to keep data local Accessibility, security challenges Cost effectiveness chaos+gaia usage: 11,154,125 CPUhour (1273 years) since ,06M$ on EC2 cc2.8xlarge vs. 4 Me cumul. HW investment Virtualization layer impact on performance? most probably decreased performance huge overhead induced on I/O + no support of IB [Q E F]DR 5 / 29

6 Introduction, Context & Motivations Objectives of this study Better than assumptions/a-priori: concrete models and experiments Evaluate impact of the underlying hypervisor at the heart of any cloud middleware so far analysis of the most widespread virtualization frameworks propose a lightweight, high-level model of a virtualized machine. Evaluate a real HPC platform (or anything as close as possible) concrete deployment on top of the Grid5000 platform select benchmarking tools to reflect an HPC usage Abstract from the specifics of a single processor architecture Evaluate Intel vs AMD Thanks Georges ;) 6 / 29

7 Introduction, Context & Motivations Virtualization Frameworks Enables finer grain resource provisioning Provides new functionalities (e.g. migration, suspension) The study includes the most commonly used hypervisors Hypervisor: Xen 4.0 KVM 0.12 ESXi 5.1 Host architecture x86, x86-64, ARM x86, x86-64 x86-64 VT-x/AMD-v Yes Yes Yes Max Guest CPU Max. Host memory 1TB - 2TB Max. Guest memory 1TB - 1TB 3D-acceleration Yes (HVM Guests) No Yes License GPL GPL/LGPL Proprietary deployment on the same Debian instance on Grid / 29

8 Introduction, Context & Motivations HPC benchmarks Selected to represent various use cases of HPC systems: HPCC : new reference benchmark suit for HPC includes HPL 7 tests to stress CPU/disk/RAM/network usage Bonnie++ : a file system benchmarking suite IOZone : cross-platform benchmark of file operations read, write, re-read, re-write, read backwards/strided, mmap... 8 / 29

9 Modeling Summary 1 Introduction, Context & Motivations 2 Modeling 3 Experimental Setup & Experiments Performed 4 Results 5 Conclusion 9 / 29

10 Modeling Resource model The model divides machine into distinct types. Further defined by resource supplies: their capacity and architecture. Node PowerEdge R310 Processor Memory Storage Network Intel Xeon X3430, 2.4 GHz, 8M Cache, Turbo 4GB Memory (2x2GB), 1333MHz Single Ranked UDIMM 500GB 7.2K RPM SATA 3.5in No Raid Broadcom 5709 Dual Port 1GbE NIC w/toe iscsi, PCIe-4 C o r e C o r e C o r e C o r e 4 GB 500 GB 1 Gbps 1 Gbps / 29

11 Modeling Resource allocation model The resource allocation is three-tier: Task, VM, Machine. T1 D: 1 A: 1 T2 D: 2 A: 1 T3 D: 1 A: 1 T4 D: 2 A: 1 T5 D: 1 A: 1 T6 D: 1 A: 1 T7 D: 1 A: 2 T8 D: 2 A: 2 T9 D: 2 A: 2 VM1 Type 1 D: 5 P: 4 A: 1 VM2 Type 2 D: 6 P: 5 A: 1 VM3 Type 3 D: 6 P:4 A: 1 VM4 Type 3 D: 6 P:4 A: 1 VM5 Type 2 D: 6 P: 5 A: 1 Node 1 P: 20 U: 17 A: 1 Node 2 P: 32 U: 12 A:2 Each level is described by the previously presented resource model, either as a resource provision or as a resource demand. Validation if such model can estimate the avg. power of a node. 11 / 29

12 Experimental Setup & Experiments Performed Summary 1 Introduction, Context & Motivations 2 Modeling 3 Experimental Setup & Experiments Performed 4 Results 5 Conclusion 12 / 29

13 Experimental Setup & Experiments Performed Setup Name Site Cluster #cpus/n #RAM Processor R peak Intel Lyon taurus 2 32GB Intel Xeon E5-2630@2.3GHz 6C 110,4 GFlops AMD Reims stremi 2 48GB AMD Opteron 6164 HE@1.7GHz 12C GFlops Deploy of recent platforms 2011, Intel or AMD Experimental setup not straightforward see the paper for details. 13 / 29

14 Experimental Setup & Experiments Performed Runs & Monitoring config: baseline KVM Xen VMWare ESXi Observation No. stremi stremi stremi stremi taurus taurus taurus taurus Avg power values based on: OmegaWatt taurus average every 1s Raritan stremi instant. every 3s dstat used to capture utilization (every 1s): CPU user, system, idle, wio (%) Memory used, buffered, cached, free (B) Disk read, write (B) Network received, send (B). 14 / 29

15 Results Summary 1 Introduction, Context & Motivations 2 Modeling 3 Experimental Setup & Experiments Performed 4 Results 5 Conclusion 15 / 29

16 Results 1000 Performance HPCC results Baseline Intel Baseline AMD Xen Intel Xen AMD KVM Intel KVM AMD VMWare ESXi Intel VMWare ESXi AMD 100 Raw Result 100 Raw Result HPL (TFlops) 1 DGEMM (GFlops) 0.1 STREAM (MB/s) 100 Copy Scale Add Trial Raw Result Raw Result RandomAccess (GUPs) / 29

17 Results Performance IOzone results 6000 rewrite random_write IOzone 64MB file, 1MB record test stremi baseline taurus baseline stremi KVM taurus KVM stremi Xen taurus Xen stremi ESXi taurus ESXi read random_read reread Intel-based platform outperforms the AMD-based one ESXi outperforms baseline in some cases on Intel platform (caching strategy?) 17 / 29

18 Results consumption profiles I Baseline Intel (Lyon) Baseline AMD (Reims) Power usage evolution Baseline Intel (Lyon) Power usage evolution Baseline AMD (Reims) 350 HPCC HPCC Bonnie J Bonnie J IOZone 4427s IOZone 2805s J J 7436s 1475s J 51407J s 458s Power usage [W] Power usage [W] Xen Intel (Lyon) Xen AMD (Reims) Power usage evolution Xen Intel (Lyon) Power usage evolution Xen AMD (Reims) J 2693s HPCC Bonnie++ IOZone J 3486s HPCC Bonnie++ IOZone J 1700s 62161J 532s J 3302s J 1428s Power usage [W] Power usage [W] / 29

19 Results consumption profiles II KVM Intel (Lyon) KVM AMD (Reims) Power usage evolution KVM Intel (Lyon) Power usage evolution KVM AMD (Reims) 350 HPCC HPCC J Bonnie++ Bonnie++ IOZone J IOZone 3561s s J 57950J J J 1471s 538s s 2321s Power usage [W] 150 Power usage [W] ESXi Intel (Lyon) ESXi AMD (Reims) Power usage evolution VMWare ESXi Intel (Lyon) Power usage evolution VMWare ESXi AMD (Reims) J 2693s HPCC Bonnie++ IOZone J 48143J J 2063s HPCC Bonnie++ IOZone s 388s J J Power usage [W] 150 Power usage [W] s 1252s / 29

20 Results Models and their accuracy The proposed models are based on multiple linear regression principle and include different subsets of available inputs. Model R 2 Residuals Error St.er. Min 1Q Median 3Q Max W % Basic % Refined % No Phases % CPU Hom % CPU Het % Only phases % No group % Clusterwise % Group only % 20 / 29

21 Results Numerical values for sample model Categorical predictors: node stremi-3 stremi-30 stremi-31 stremi-6 taurus-10 taurus-7 taurus-8 taurus-9 value phase Bonnie DGEMM FFT HPL IOZONE PTRANS RandAcc STREAM idle value hypervisor ESXi KVM Xen baseline value Numerical Predictors: metric Intercept cpu user cpu system cpu idle cpu wio mem used value E-9 metric mem buffers mem cached mem free disk write bytes rec. bytes send value -3.1E E E E E E / 29

22 Results Predictions I Baseline Intel (Lyon) Baseline AMD (Reims) Power [W] Observed Refined Power [W] Observed Refined / 29

23 Results Predictions II Xen Intel (Lyon) Xen AMD (Reims) Power [W] Observed Refined Power [W] Observed Refined / 29

24 Results Predictions III KVM Intel (Lyon) KVM AMD (Reims) Power [W] Observed Refined Power [W] Observed Refined / 29

25 Results Predictions IV ESXi Intel (Lyon) ESXi AMD (Reims) Power [W] Observed Refined Power [W] Observed Refined / 29

26 Conclusion Summary 1 Introduction, Context & Motivations 2 Modeling 3 Experimental Setup & Experiments Performed 4 Results 5 Conclusion 26 / 29

27 Conclusion Conclusion CC in an HPC context requires a better understanding of the performance of virtualization middlewares. In this talk: evaluation of the 3 widespread virtualization frameworks Xen, KVM and VMware ESXi vs. baseline practical and lightweight power model of virtualized nodes holistic model is the most accurate deployment/experiments on a real HPC environment Grid 5000, Intel vs. AMD evaluation Middleware affects both performance and energy results observed overhead (20-30%) is acceptable. Hardware heterogeneity is noticeable Better results on Intel than on AMD 27 / 29

28 Conclusion Future work 1 Performance modelling and comparing it with the power model. 2 Analysis of effects of multiple VMs on a single node. 3 Including temperature as an environmental factor. 4 Analysis of overhead of cloud management system OpenNebula, OpenStack, Eucalyptus, Nimbus etc. Snooze ;) 5 Experimenting with network-intensive workloads 6 Derivation of model from component, rather than full platform level 28 / 29

29 Conclusion Thank you for your attention... 1 Introduction, Context & Motivations 2 Modeling 3 Experimental Setup & Experiments Performed 4 Results 5 Conclusion Contacts: {firstname.name}@uni.lu 29 / 29