Migration of virtual machine to cloud using Openstack Python API Clients

Migration of virtual machine to cloud using Openstack Python API Clients Jyoti Joshi 1, Manasi Thakur 2, Saurabh Mhatre 3, Pradnya Usatkar 4, Afrin Parmar 5 1 Assistant Professor Computer, R.A.I.T., University of Mumbai, India 2,3,4,5 B.E. Computer, R.A.I.T., University of Mumbai, India Abstract A Cloud is a virtual space available to deploy the applications, whereas Cloud Computing is a general term for anything that involves delivering hosted services over the Internet. It is delivering the resources and capabilities of information technology dynamically as a service. Cloud Computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet. Cloud migration refers to migrating the services or applications from existing system to the cloud. Our project Cloud Migration Tool is aimed at designing a tool for migrating a machine from physical environment to cloud to provide cloud services to clients. This private cloud is capable of providing the infrastructure and platform as a service. Infrastructure includes the storage, servers, virtualization, compute and network services, middleware and runtime environment. The private cloud implementation is done using OpenStack and Ubuntu. The project allows clients to launch their own private cloud instances for various purposes. The project also allows clients to deploy existing virtual machines to cloud to directly migrate existing applications to cloud. The private cloud implementation is done using Openstack API for building private cloud. Openstack python clients for deployment of existing virtual machines to cloud and MAAS for controlling cluster of servers for private cloud provisioning using Ubuntu. Keywords Cloud computing, Cloud migration, private cloud, Openstack, MAAS. I. INTRODUCTION A. Virtualization Virtualization is a core technology for cloud computing. Virtualization is essentially a technology that allows creation of different computing environments. Virtualization technologies are also used to replicate runtime environments for programs. Virtualization is a technology that allows running two or more operating systems side-by-side on just one PC or embedded controller. B. Cloud Computing A cloud is a combination of hardware, networks, storage, services and interfaces that helps in delivering computing as a service. Cloud computing is a technology that uses the internet and central remote servers to maintain data and applications. Cloud computing allows consumers and business to use applications without installation and access their personal files at any computer with internet access. This technology allows for much more efficient computing by centralizing storage, memory, processing, bandwidth.. C. Deployment Models used in Cloud 1) Public Cloud or external cloud: It is freely available for access. You can use a public cloud to collect data pertaining to the purchase of items from a web site on the internet [1]. 2) Private Cloud: Allows the usage of services by a single client on a private network [1]. 3) Hybrid Cloud: Consists of multiple service providers. This model integrates various cloud services for hybrid web hosting. It is basically a combination of private and public cloud feature [1]. D. Models of Cloud Services 1) Infrastructure as a Service (IaaS): IaaS provides storage and computer resources that developers and IT organizations can use to offer customized business solutions [1]. 2) Platform as a Service (PaaS): PaaS enables the provider to deliver the additional amount of infrastructure. It offers a solution stack, which is a mutually inclusive set of software to help developers at the time of development of an application and its execution [1]. 3) Software as a Service (SaaS): SaaS delivers the software as a service over the internet. SaaS also provides security and reliability by incorporating virtual private networks (VPN s). This enables the user to transfer the data securely over the network, such as internet [1]. E.CLOUD MIGRATION Cloud migration is the process of moving data, applications or other business elements from an organization's onsite computers to the cloud, or moving them from one cloud environment to another. 217

II. PROPOSED SYSTEM Proposed system is an on-demand, web-based, userfriendly, off-premise solution that does not require any software to be loaded on users computers. It is a pay-perlicense/month solution, operational with existing Internet access, rapidly configured and deployed to best fit a company s needs. The solution is easy-to-implement without any customizations, thereby requiring less time from discovery to implementation. Fig:1 Overview of proposed system Cloud computing is also known as "Internet-based computing".cloud computing lets you keep information on a remote server (the cloud), instead of trapped in a computer. You can access your data from a smartphone, a tablet, a laptop, or a desktop wherever you have an Internet connection. Cloud migration is the process of moving data, applications or other business elements from an organization's onsite computers to the cloud, or moving them from one cloud environment to another. Cloud Computing basically provides three types of servcies Iaas (Infrastructure as a Service), Paas (Platform as a Service) and SaaS (Software as a Service). In our project "Cloud Migration Tool" with the help of migration technology we provide Iaas service. Cloud Migration tool is used to transfer a machine from physical environment to the virtual environment i.e to the cloud. In our project we transfer a Vmdk file to the Openstack cloud. The migration is being performed between the two machines. One is Ubuntu desktop where we have our Vmdk file stored and the other machine is the Ubuntu Server where our openstack cloud is being deployed. For implementing migration we have used Psyftp. Major challenge is Cloud Migration lies in transferring the file from physical environment to virtual environment without the loss of data and machine should provide same functionalities in the virtual environment. When migrating an ISO image from one virtual machine to another, you may run into the same limitations as when migrating one physical sever to another. For example, you cannot move a 64-bit system image to a 32-bit platform. You are also most likely to encounter Hardware issues when restoring a Windows image onto the virtual machine. Proposed methodology In our project of Cloud Migration, we have used Ubuntu operating system to reach our goal of migrating an Operating system from one system to another system without any problems regarding kernel of the destination machine or any changes of environmental conditions of that particular system. On Ubuntu Operating system, firstly we install KVM for making virtual machines. In that KVM, we created three VMs (Virtual Machines). First VM is of MAAS for creating a nodes and maintaining the status of all the other VMs and configuring its uses by providing service. Second VM consists of Ubuntu Server which has Lubuntu environment for user interface and on that particular VM, devstack is installed to access the Openstack Dashboard and all the components of Openstack like Nova, Glance, Keystone, Horizon, Swift, and many more. And lastly on the third VM, Ubuntu desktop is being installed. On that particular VM, VMWare player is installed. Our aim is to transfer the ISO image of ubuntu operating system from the VMware player of Ubuntu desktop to openstack dashboard and boot that particular image on the openstack environment and run it. So to transfer the ISO image, we used PYSFTP operation of the python language. By using it, we transferred the ISO image automatically. And by using a CGI scripting language with HTML code, we created a GUI and onclick boot of image on openstack. All the components of openstack are modified in python script for boot of an image. Secure Shell (SSH) is a cryptographic network protocol for secure data communication, remote command-line login, remote command execution, and other secure network services between two networked computers. It connects, via a secure channel over an insecure network, a server and a client running SSH server and SSH client programs, respectively. 218

A. Stages of Cloud Migration Tool 1)Import image To import image from Ubuntu desktop to Ubuntu server, PYSFTP is used.to connect to FTP server, we first have to import the pysftp module and specify server, username and password credentials. We also import the sys module, since we want the user to specify the file to download / upload. The Connection object is the base of pysftp. It supports connections via username and password. We can also use RSA and DSA key pair. If you key is password protected, just add private_key_pass to the argument list. The connection object also allows you to use an IP Address for the host and you can set the port which defaults to 22, as well. Pysftp s get method allows you to copy all the files in a remote directory to a local path. Put method allows you to copy the contents of a local directory to a remote one via SFTP. 2)Create Image To create Image, we have to enter following values Name: Enter a name for the image. Description: Optionally, enter a brief description of the image. Image Source: Choose the image source from the list. Your choices are Image Location and Image File. Image File or Image Location: Based on your selection for Image Source, you either enter the location URL of the image in the Image Location field. or browse to the image file on your system and add it. Format: Select the vmdk format. Architecture: Specify the architecture. For example, i386 for a 32-bit architecture or x86_64 for a 64-bit architecture. Minimum Disk (GB) and Minimum RAM (MB) : Leave these optional fields empty. Public: Select this check box to make the image public to all users with access to the current project. Protected: Select this check box to ensure that only users with permissions can delete the image. 3) Creating and Running Instance When you launch an instance from an image, OpenStack creates a local copy of the image on the node where the instance starts. Virtual computing environments, known as instances. Preconfigured templates for your instance that package the bits you need for your server. Secure login information for your instances using key pairs. Storage volumes for temporary data that's deleted when you stop or terminate your instance, known as instance store volumes. Multiple physical locations for your resources, such as instances known as regions and Availability Zones. A firewall that enables you to specify the protocols, ports, and source IP ranges that can reach your instances using security groups. To create instance we have to mention following values Availability Zone: By default, this value is set to the availability zone given by the cloud provider. For some cases, it could be nova. Instance Name: Assign a name to the virtual machine. The name you assign here becomes the initial host name of the server. After the server is built, if you change the server name in the API or change the host name directly, the names are not updated in the dashboard. Flavor: Specify the size of the instance to launch.the flavor is selected based on the size of the image selected for launching an instance. For example, while creating an image, if you have entered the value in the Minimum RAM (MB) field as 2048, then on selecting the image, the default flavor is m1.small. Instance Count: To launch multiple instances, enter a value greater than 1. The default is 1. Instance Boot Source: If you choose Boot from image option, a new field for Image Name displays. You can select the image from the list. Image Name: This field changes based on your previous selection. Since you have chosen to launch an instance using an image, the Image Name field displays. Select the image name from the dropdown list. 4) Verify By default floating IP address is allocated from public key to instance.instance runs on client machine. To provide a remote console or remote desktop access to guest virtual machines, VNC(Virtual Network Computing) is used. The VNC proxy is an OpenStack component that enables compute service users to access their instances through VNC clients. 219

The VNC console connection works as follows: A user connects to the API and gets an access_url such as, http://ip:port/?token=xyz. The user pastes the URL in a browser or uses it as a client parameter. The browser or client connects to the proxy. The proxy talks to nova-consoleauth to authorize the token for the user, and maps the token to the private host and port of the VNC server for an instance. The proxy initiates the connection to VNC server and continues to proxy until the session ends. This stage verifies whether the virtual machines run properly on cloud after the migration of the application and system. III. IMPLEMENTATION A. Installation and Configuration of Openstack The proposed system in implemented using opensource software called Openstack, server provisioning tools called MAAS and Ubuntu operating system. First we download and install kvm and supporting packages for creating virtual machines by the following command. sudo apt-get install qemu-kvm libvirt-bin bridge-utils virt-manager We create a virtual machine on kvm having ubuntu as operating system and then install MAAS on that virtual machine using the following command. sudo apt-get install maas maas-dhcp mass-dns Once MAAS server has been setup and configured the next step is to create virtual nodes on kvm and set them up for network booting. The nodes are automatically provisioned and controlled by MAAS server and should be configured to boot ubuntu server edition. We select one such virtual node and access that node remotely using ssh connection. The next step is to install openstack on this node. Openstack can be installed using devstack in 3 steps which are: Git clone https://git.openstack.org/openstackdev/devstack cd devstack./install.sh Once openstack has been installed and configured the components of openstack can be accessed using openstack python API clients and also by logging into openstack dashboard. B. Python Scripts 1)Credentials import os import cgi, cgitb cgitb.enable() def get_keystone_creds(): d = {} d['username'] = "admin" d['password'] = "nova" d['auth_url'] = "http://10.16.16.114:5000/v2.0" d['tenant_name'] = "admin" return d def get_nova_creds(): d = {} d['username'] = "admin" d['api_key'] = "nova" d['auth_url'] = "http://10.16.16.114:5000/v2.0" d['project_id'] = "admin" return d 2) Import image: import pysftp sftp=pysftp.connection(hostname,username=user,password =password) sftp.get(' ubuntu.vmdk', preserve_mtime=true) 3 ) Create image: import os import keystoneclient.v2_0.client as ksclient import glanceclient from credentials import get_keystone_creds import cgi, cgitb cgitb.enable() creds = get_keystone_creds() keystone = ksclient.client(**creds) glance_endpoint=keystone.service_catalog.url_for(service_ type='image',endpoint_type='publicurl') glance=glanceclient.client('1',glance_endpoint, token=keystone.auth_token) with open('ubuntu.vmdk') as fimage: glance.images.create(name="ubuntu",is_public=true, disk_format="vmdk",container_format="bare", data=fimage) 220

4) Create instance: import cgi, cgitb cgitb.enable() import os import time import novaclient.v2.client as nvclient from credentials import get_nova_creds creds = get_nova_creds() nova = nvclient.client(**creds) image = nova.images.find(name="ubuntu") flavor = nova.flavors.find(name="m1.rb") instance=nova.servers.create(name="ubuntu", image=image, flavor=flavor) IV. SYSTEM REQUIREMENTS A. Hardware Requirements Ubuntu 14.04.1 LTS 64-bit operating system Processor: Intel(R) Core(TM ) i3-3217u CPU @ 1.80Ghz RAM: 8GB Ubuntu 14.04.1 LTS servers (64-bit) System should have minimum following configuration RAM: 8GB Free hard disk: 100 GB System should have recommended configuration RAM: 12 GB Free hard disk: 500 GB Quad-core dual processor systems or dual-core quad processor sytems. B. Software requirements There are packages required for the installation of cloud using Ubuntu. 1) Openstack OpenStack is open source software, which means that anyone who chooses to can access the source code, make any changes or modifications they need, and freely share these changes back out to the community at large. Nova is the primary computing engine behind OpenStack. It is a "fabric controller,"which is used for deploying and managing large numbers of virtual machines and other instances to handle computing tasks. Horizon is the dashboard behind OpenStack. Keystone provides identity services for OpenStack. Glance provides image services to OpenStack. 2) MaaS (Metal as a Service) Its major function is to bring language of the cloud to physical servers. It makes it easy to set up the hardware on which services are deployed that needs to be scaled up and down dynamically. This feature is important in cloud since cloud requirement can fluctuate dynamically. 3) Apache2 Server Apache is generally recognized as the world's most popular Web server (HTTP server).. The Apache Web server provides a full range of Web server features, including CGI, SSL, and virtual domains. Apache also supports plug-in modules for extensibility. Apache is free software, distributed by the Apache Software Foundation that promotes various free and open source advanced Web technologies. 4) CGI (Common Gateway Interface) The common gateway interface (CGI) is a standard way for a Web server to pass a Web user's request to an application program and to receive data back to forward to the user. When the user requests a Web page (for example, by clicking on a highlighted word or entering a Web site address), the server sends back the requested page. However, when a user fills out a form on a Web page and sends it in, it usually needs to be processed by an application program. The Web server typically passes the form information to a small application program that processes the data and may send back a confirmation message. This method or convention for passing data back and forth between the server and the application is called the common gateway interface (CGI). It is part of the Web's Hypertext Transfer Protocol (HTTP). 5) Python Python is a high-level, interpreted, interactive and object-oriented scripting language. Python is copyrighted. Like Perl, Python has relatively few keywords, simple structure, and a clearly defined syntax. Python provides a better structure and support for large programs than shell scripting. 221

6) Pysftp (Python Secure File Transfer Protocol) Pysftp is an easy to use sftp module that utilizes paramiko and pycrypto. It provides a simple interface to sftp. It is used to securely exchange files over the Internet. To connect to FTP server, we first have to import the pysftp module and specify server, username and password credentials. We also import the sys module, since we want the user to specify the file to download / upload. The Connection object is the base of pysftp. It supports connections via username and password. We can also use RSA and DSA key pair. If you key is password protected, just add private_key_pass to the argument list. The connection object also allows you to use an IP Address for the host and you can set the port which defaults to 22, as well. Pysftp s get method allows you to copy all the files in a remote directory to a local path. Put method allows you to copy the contents of a local directory to a remote one via SFTP. 7) Virtual Machine Manager The virt-manager application is a desktop user interface for managing virtual machines through libvirt.it presents a summary view of running domains, their live performance & resource utilization statistics Using KVM, one can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, sound card etc. An embedded vnc client viewer presents a full graphical console to the guest domain. It also allows users to clone a virtual machine for testing and other purposes. V. RESULT Fig. 3 Creating image Fig. 4 Creating instance VI. CONCLUSION In this project, a new and powerful infrastructure for building private was introduced which is capable of serving the user request for Infrastructure and Platform services. Services include launching different flavours of images, instances and services etc. The cloud based instances could be used for variety of different purposes in the future. The system was able to simulate a private cloud infrastructure which could easily be deployed on cluster of servers for dynamic scaling and provisioning of cloud services. The aspect of migration of existing system environments into cloud was also covered by which existing system environments could be simulated on private cloud server. The resulting infrastructure was not only elastic and flexible but also easier to maintain. The future work can be implementation of Software as a Service with the help of Openstack and Ubuntu. Fig. 2 Importing image 222

Acknowledgment We remain immensely obliged to Vijay Jadhav and Vaibhav Kale, Persistent System Ltd. and, for providing us with the idea of this topic, and for their invaluable support in garnering resources for us either by way of information or computers also their guidance and supervision of Prof.Jyoti Joshi which made this project happen. We would like to say that it has indeed been a fulfilling experience for working out this Project topic. REFERENCES [1] J Priya Shanti,Parsi Kalpana Migration of Existing Applications to Cloud and Among Clouds Volume 3, Issue 4, April 2013 [2] Saju Mathew Implementation of Cloud Computing in Education A Revolution Vol. 4, No. 3, June 2012 [3] Rohit Kambhoj, Anoop Arya Openstack: Open Source Cloud Computing Iaas Platform volume 4, Issue 5, May 2014 [4] Omar SEFRAOUI, Mohammed AISSAOUI Openstack: Towards an Open-Source Solution for Cloud Computing [5] Jun-Feng Zhao Jian-Tao Zhou Strategies and Methods for Cloud Migration April 2014, 143-152 [6] Girish L S, Dr. H S Guruprasad Building Private Cloud using OpenStack Volume 3, Issue 3, May June 2014 223