Virtual Machine Dynamic Provision and Image Management for UniCloud

Transcription

1 Virtual Machine Dynamic Provision and Image Management for UniCloud Pangfeng Liu April 10, Related work 1.1 Virtual Machine Application performance isolation in virtualization Performance isolation is that performance of one VM should not affect performance of another VM in the same computer, which application providers concerned about because there are several kinds of applications in different virtual machine on a same physical machine. Therefore, Somani and Chaudhary presented the good isolation with credit scheduler of Xen, and the other scheduler called SEDF has shown the better performance than credit scheduler in [14] Live Migration of Virtual Machines Clark et al. compared original migration (stop-and-copy) with live migration (pre-copy), and observed performance under VM migration with bandwidth and the number of pre-copy iterations. The result in [9]is that pre-copy is always better than stop-and-copy. In addition, more pre-copy iterations and more bandwidth can effective reduce downtime. In these year, xen and vmware has supported the live migration Post-Copy Based Live Virtual Machine Migration Using Adaptive Pre-Paging and Dynamic Self-Ballooning Post-Copy is a new method of migration compared with Pre-Copy. Post-Copy migration transfer VM s processor state to target host first, and then send memory contents [10]. As the result, all the memory content will be transferred at most once, which Pre-Copy migration scheme will iterative transfer the dirty pages. As soon as processor state arrives, then dealing the page faults through network with Pre-Paging strategy. Furthermore, Hines and Gopalan eliminate the transfer of unallocated memory pages through Dynamic Self-Ballooning. 1

2 1.2 Cloud Infrastructure and Management Memory Buddies: Exploiting Page Sharing for Smart Colocation in Virtualized Data Centers Memory Buddies [15] is a memory sharing-aware placement system for virtual machines. The goal is to maximize the number of VMs in a data center. It uses the VMware ESX [5] technique content-based page sharing to share pages from different VMs in the same host. The algorithm uses fingerprint to predict what kind of combination saving the most memory pages Virtual Organization Cluster Murphy et al. proposed the idea of using one cluster to simulate several virtual clusters to support virtual organizations [11]. The virtual cluster for individual virtual organization is called virtual organization cluster. Every virtual organization provides VM images they needed. Cluster administrators run a virtual cluster supporting each virtual organization Dynamic Provisioning of Virtual Organization Clusters Based on virtual organization cluster [11], Murphy et al. further refine the virtue machine framework so that it can dynamically adjust the number of VMs to support virtual organization [12]. They use condor system and maintain a job queue for system to detect the load of each virtual organization. Using this information, they can decide VM numbers for different virtual organization in different situation An Efficient Resource Management System For On-line Virtual Cluster Provision Chen et al. studied how to reduce image preparation overhead by VM image caching and multicasting VM image to physical machines [8]. Also the Load-Aware Mapping,a resource mapping strategy, can further reduce deploying overhead by awaring VM image cache information and make resource used efficiently Dynamic Provisioning of Resources in Data Centers Simmons et al. proposed the idea of how to allocate the resources in the data center [13]. In this paper, it describes some policies that we should concern to make th balance between the datacenter and the customers. Finally, decisions are based on the model which combines all the policies. 2

3 1.3 Tools Xen Xen [7] is an open source virtual machine monitor. It can execute sereval guest OS on the same machine. Xen systems include Xen hypervisor as the lowest and most privileged layer to schedule different guest OS across the physical CPUs. It needs modified Unix-like OS to be domain 0 and domain U Hadoop Hadoop [2] is a open source software framework which support data-intensive distributed applications. Its capable for thousands of nodes and petabytes of data for applications. Hadoop also include a distribute file system inspired by Googles GFS and the framework is by Googles MapReduce MPI MPI [3], which stand for message passing interface, allows many computers communicate with one another. MPI specify a communication protocol for parallel computing. It is a message-passing application interface for programmers. MPI is common used in high-performance computing GlusterFS GlusterFS is a application level distributed share file system [1]. it has client and server components. Server side store the data, And client side using TCP/IP to access data on servers. GlusterFS has two modes to store data. Those modes are file-based striping and file-based mirroring,respectively OpenNebula OpenNebula can provide an interface to manage vitualinfrastructure including storage, network [4]. And it also provide virtualization technologies to enable migration and deploying VM on distributed system XCP- Xen Cloud Platfrom XCP [6]is a platfrom for the cloud enviroment. It collects several tools in the previous products, and combine the xen machines to a resource pool. Therefore, it can monitor VMs, do load balancing and migrate automatically the pool. 2 progress report 2.1 Xen We install xen 4.0/3.2 [7] on our machines. We have 8 machines running Debian system. First, we install xen 3.2 but there exist a problem. The problem is that 3

4 there are some machines can not run VM. These machines is grid01,grid02,grid04 and grid05. Now we test xen 4.0 on grid04 and grid05. It works! Although xen 4.0 is unstable, it fixes many problems in xen 3.2. Now, we describe how we install xen 4.0/3.2 and how to create a VM image Install Xen 3.2 We use debian packages to install xen 3.2. We just type apt-get install to install packages. It needs packages xen-linux-system xen-686,libc6-xen and bridge-utils. Make sure that menu.lst is correct. It must boot by xen hypervisor. And modify /etc/xen/xend-config.sxp file if need. For example, if you want to do live migration, the line #(xend-relocation-hosts-allow ) must deletes the # sign Install Xen 4.0/4.1 unstable version Type hg clone to download the source code. Enter the directory. Type make xen; make tools; make install-xen; make install-tools. At this step, you may need to install some packages like bin86,dev86,etc. Go to to download modified kernel supporting xen. Compile new kernel for booting. Modify menu.lst and xendconfig.sxp. Boot in new kernel Create VM images First, we use dd to create two images, one for disk and one for swap. And we use mkfs.ext3 and mkswap commands to format this two images. In order to install a basic Debian system in our image, we mount the image and run debootstrap. For example, we type mount -o loop disk /mnt; debootstrap arch i386 lenny /mnt In this step, debootstrap will ask us to set up root password. Next, we set up our VM environment like network, fstab, hostname, etc. Modify VM configure file. There is an example: 4

5 2.2 OpenNEbula description kernel = /boot/vmlinuz xen-686 ramdisk = /boot/initrd.img xen-686 memory = 512 root = /dev/hda1 ro disk = [ file:/testvm/swap.img,hda2,w, file:/testvm/disk.img,hda1,w, ] name = testvm1 vif = [ ] on poweroff = destroy on reboot = restart on crash = restart extra = console=hvc0 xencons=tty we use OpenNebula(ONE) [4] v1.4 to build private cloud on ubuntu before-install We need install ruby, sqlite3, xmlrpc-c, openssl, ssh, scons, g++, flex, blson on ONE server and Install ruby on all cluster nodes. All cluster nodes should have ssh server running and hypervisor working properly configured and ssh-key exchange to ONE server. Build a share file system which can be accessed by all nodes in order to live migration VM. in our case we use NFS or GlusterFS [1] to build share file system install Download ONE and install it on share file system configure put all VM images on share filesystem. configure etc/one.conf as follow(let ONE use share image mode) TM MAD = [ name = tm nfs, executable = one tm, arguments = tm ssh/tm nfs.conf, default = tm ssh/tm nfs.conf ] add cluster nodes to ONE server. build a VM configure file for example. 5

6 NAME = xen hdp CPU = 1 MEMORY = 2048 OS = [kernel=/boot/vmlinuz xen,initrd=/boot/initrd.img xen,root=hda2 ] DISK = [source= /srv/cloud/xen hdp/disk.img,target=hda2,readonly= no ] DISK = [source= /srv/cloud/xen hdp/swap.img,type =swap,size =512,target=hda1,readonly= no ] NIC = [ip= ] GRAPHICS = [type= vnc,listen= ,port= 5900 ] we can use this VM configure file to create VM. And then we can depoly VM on spicific cluster node. Finally we can live-migrate VM between cluster nodes by ONE. 2.3 GlusterFS description we use GlusterFS [1] to store our VM images before-install install sshfs, build-essential, flex, bison, byacc, fuse on all node. make directory /export/sdb1 on all server nodes install install glusterfs on all nodes configure Execute glusterfs-volgen to generate glusterfs configure file. For example make and node as server. glusterfs-volgen name test :/export/sdb :/export/sdb1; copy this configure file to all server node /etc/glusterfs/glusterfsd.vol. start glusterfs server on all server nodes. /etc/init.d/glusterfsd start; copy the configure file to client nodes and make directory for mounting. mounting glusterfs on specific directory without direct IO (direct IO is fast but xen can not boot sucessfuly if image is on direct IO mode filesystem). glusterfs disable-direct-io-mode -f /usr/src/glusterfs-2.0.9/test2- tcp.vol /mnt/glusterfs; 6

7 2.3.5 potential problem GlusterFS dose not synchronize clients cache data. VM images may not sync when VM migration. 2.4 Hadoop Performance We first consider how virtualization affects on hadoop performance, and then find opportunities of improve performance running hadoop under virtual machines. We setup a hadoop cluster with three nodes each on heterogeneous machine, say native cluster. Then build virtual machine on each computer to run virtual cluster, which using the same settings with native cluster, and we can obtain their performance. The following table describes our hardware. Intel Core 2 CPU 2.00GHz Processor Architecture Intel Core 2 CPU 2.00GHz AMD Athlon 64 Processor GHz 4GB Memory 4GB 4GB 58GB Disk 63GB 500GB Network Connection 1G Host Os Ubuntu 8.04 Guest OS Ubuntu 8.04 Xen version Xen 3.2 Memory to Guest 512MB Disk (Guest) 2G No. of VCPUs to all OSs 2 We first run a RandomWrite generate 30maps, 10MB per maps and measure the job complete time. The virtual cluster has 71.52% performance compare with native cluster in average. Then we run Sort job to sort the random data generated before, which is 66.73% of native. The following are result data. RandomWrite sec Native VM Avg Performance 71.52% 7

8 Sort sec Native VM Avg Performance 66.73% 2.5 MPI Benchmark Intel MPI Benchmarks 3.2 The IMB 3.2 package consists of 3 parts: IMB-MPI1 2 MPI-2 functionality parts IMB-EXT (One-sided Communications benchmarks), and IMB-IO (I/O benchmarks) Machine The three real machines and VM configuration are as follows: 8

9 Intel Core 2 CPU 2.00GHz Processor Architecture Intel Core 2 CPU 2.13GHz AMD Athlon 64 Processor GHz 4GB Memory 4GB 4GB 63GB Disk 20GB 500GB Network Connection 1G Host OS Ubuntu 8.04 Guest OS Ubuntu 8.04 Xen version Xen 3.2 Memory to Guest 512MB Disk (Guest) 2G No. of VCPUs to all OSs Performance We set the number of MPI processes to run IMB be three and one virtual machine per real computer and put the image files on the local, nfs, and the gluster file systems [1]. IMB-MPI1 test1 test2 average local 127m s 135m s 131m s nfs 162m s 157m s 159m s glusterfs 159m s 162m s 160m s The time taken to complete IMB-MPI1 for nfs and glusterfs is nearly same and both of them are greater than the time for local. IMB-IO test1 test2 average local 21m s 30m s 26m s nfs 75m s 80m s 77m s glusterfs 28m s 27m s 27m s Unlike IMB-MPI1, the taken time of IMB-IO for glusterfs is close to the time for local, but the time for nfs is still longer than others. References [1] Gluster file system. [2] Hadoop. 9

10 [3] Mpi. [4] Opennebula. [5] Vmware esx. [6] Xcp. [7] Xen. [8] Y. Chen, T. Wo, and J. Li. An efficient resource management system for on-line virtual cluster provision. Cloud Computing, IEEE International Conference on, 0:72 79, [9] C. Clark, K. Fraser, S. Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live migration of virtual machines. In NSDI 05: Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation, pages , Berkeley, CA, USA, USENIX Association. [10] M.R. Hines and K. Gopalan. Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In VEE 09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, pages 51 60, New York, NY, USA, ACM. [11] M. A. Murphy, M. Fenn, and S. Goasguen. Virtual organization clusters. In PDP 09: Proceedings of the th Euromicro International Conference on Parallel, Distributed and Network-based Processing, pages , Washington, DC, USA, IEEE Computer Society. [12] M. A. Murphy, B. Kagey, M. Fenn, and S. Goasguen. Dynamic provisioning of virtual organization clusters. Cluster Computing and the Grid, IEEE International Symposium on, 0: , [13] B. Simmons, A. McCloskey, and H. Lutfiyya. Dynamic provisioning of resources in data centers. In ICAS 07: Proceedings of the Third International Conference on Autonomic and Autonomous Systems, page 40, Washington, DC, USA, IEEE Computer Society. [14] G. Somani and S. Chaudhary. Application performance isolation in virtualization. Cloud Computing, IEEE International Conference on, 0:41 48, [15] T. Wood, G. Tarasuk-Levin, P. Shenoy, P. Desnoyers, and M. D. Corner E. Cecchet. Memory buddies: exploiting page sharing for smart colocation in virtualized data centers. SIGOPS Oper. Syst. Rev., 43(3):27 36,