Cloud@Ceid Seminars Intro to Virtualization Christos Alexakos Computer Engineer, MSc, PhD C. Sysadmin at Pattern Recognition Lab 1 st Seminar 19/3/2014
Contents What is virtualization How it works Hypervisor s role Some types of virtualization Virtualize resources What needs for installing a virtual environment Examples of benefits of virtualizations PR Lab Story
What is virtualization? Virtualization is a technology that combines or divides (computing) resources to present one or many operating environments. Hardware/software partitioning (or aggregation) Partial or complete machine simulation Emulation (again, can be partial or complete) Time-sharing (in fact, sharing in general) In general, can be M-to-N mapping (M real resources, N virtual resources)
Traditional Data Center
Virtualized Data Center
Traditional Concept Positives Easy to conceptualize Fairly easy to deploy Easy to backup Virtually any application/service can be run from this type of setup Negatives Expensive to acquire and maintain hardware Not very scalable Difficult to replicate Redundancy is difficult to implement In many cases, processor is under-utilized
Virtualized Concept Positives Resource pooling Highly redundant Highly available Rapidly deploy new servers Easy to deploy Reconfigurable while services are running Optimizes physical resources by doing more with less Negatives Slightly harder to conceptualize? Slightly more costly (must buy hardware, OS, Apps, and now the abstraction layer)
How it works A hypervisor or virtual machine monitor (VMM) is a piece of computer software, firmware or hardware that creates and runs virtual machines. A virtual machine (VM) is a software-based emulation of a computer. Virtual machines operate based on the computer architecture and functions of a real or hypothetical computer.
Virtualization Approaches Paravirtualization (PV) PV does not require virtualization extensions from the host CPU The guest operating system is not only aware that it is running on a hypervisor but includes code to make guestto-hypervisor transitions more efficient Full Virtualization (HVM) Full Virtualization or Hardware-assisted virtualizion uses virtualization extensions from the host CPU to virtualize guests Intel VT or AMD-V The guest is unaware that it is being virtualized and requires no changes to work in this configuration
Resources Virtualization
Inside Hypervisor The hypervisor requires a small number of items to boot a guest operating system: a kernel image to boot, a configuration (such as IP addresses and quantity of memory to use), a disk, and a network device. The disk and network device commonly map into the machine's physical disk and network device A set of guest tools is necessary to launch a guest and subsequently manage it.
Type of Hypervisors http://wiki.xenproject.org/wiki/xen_overview#guest_types
Known Hypervisors KVM a Linux based open source hypervisor. Xen An open source hypervisor of type 1 VMware - its hypervisor is very mature and extremely stable Hyper-V Hyper-V is a commercial hypervisor provided by Microsoft
Device Virtualization HVM PV
CPU Virtualization
Memory Virtualization
What you need to set up a virtual environment
What you really need!! Network Compute Storage
Why Virtualization? Some examples Best resources management Resources Availability Application Uptime Backup & Disaster Recovery
Typical small IT department Production (Web Server, Application Server, Database) Server Linux 16 GB RAM, 200GB HDD, 4 Cores Production (Application Server, Mail Server, ERP) Server Windows 16 GB RAM, 200GB HDD, 4 Cores Test Development (Web Server, Application Server, Database) Server Linux 8GB RAM, 200GB HDD, 6 Cores Usage Administration developer Requests sysadmin
Virtualization Environment Virtual Machines Physical Servers Production (Linux 8GB) (Web Server, Application Server, Database) Production ( Windows 4GB) (Mail Server) Production ( Windows 6GB) (ERP) Test Development (Linux 4 GB) (Web Server, Application Server, Database) Usage Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 8GB RAM, 200GB HDD, 6 Cores - OFF Administration developer Requests sysadmin
Resources Availability Production (Web Server, Application Server, Database) Production (Application Server, Mail Server, ERP) Test Development (Web Server, Application Server, Database) Server Linux 16 GB RAM, 200GB HDD, 4 Cores Server Windows 16 GB RAM, 200GB HDD, 4 Cores Server Linux 8GB RAM, 200GB HDD, 6 Cores I need a Windows Server for development with at least 8GB RAM We don t have any now, we have to buy a new one or wait 2 weeks until the develop server gets free developer Purchase Time: 2 days 1 week Setup Time: 1 day Cost: > 2000 sysadmin
Resources Availability Production (Linux 8GB) (Web Server, Application Server, Database) Production ( Windows 4GB) (Mail Server) New Server Windows 8GB Production ( Windows 6GB) (ERP) Test Development (Linux 4 GB) (Web Server, Application Server, Database) Virtual Machines Physical Servers Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 8GB RAM, 200GB HDD, 6 Cores - OFF I need a Windows Server for development with at least 8GB RAM I will setup it for you developer Purchase Time: 0 Setup Time: 15 minutes 3 hours Cost: 0 sysadmin
Application Uptime Production (Web Server, Application Server, Database) Production (Application Server, Mail Server, ERP) Test Development (Web Server, Application Server, Database) Server Linux 16 GB RAM, 200GB HDD, 4 Cores Server Windows 16 GB RAM, 200GB HDD, 4 Cores Server Linux 8GB RAM, 200GB HDD, 6 Cores Our Mail Server Crashed. We need to install it from the beginning and apply last backup OK, I will do it developer Setup Time: 1-3 hours sysadmin
Application Uptime Production (Linux 8GB) (Web Server, Application Server, Database) Production ( Windows 4GB) (Mail Server) New Server Windows 8GB Production ( Windows 6GB) (ERP) Test Development (Linux 4 GB) (Web Server, Application Server, Database) Our Mail Server Crashed. We need to install it from the beginning and apply last backup Virtual Machines I will apply the last working snapshot Physical Servers Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 8GB RAM, 200GB HDD, 6 Cores - OFF Go to an older snapshot developer Setup Time: 5-15 minutes Clean Installation sysadmin
Disaster Recovery Production (Web Server, Application Server, Database) Production (Application Server, Mail Server, ERP) Test Development (Web Server, Application Server, Database) Server Linux 16 GB RAM, 200GB HDD, 4 Cores Server Windows 16 GB RAM, 200GB HDD, 4 Cores Server Linux 8GB RAM, 200GB HDD, 6 Cores SOS Our server has been melted down!!! Ohh!!! My God! I have to find what happened Setup OS and Apps Restore last Back up developer Hardware Recovery: 1 day 1 week OS Setup Time: 1-3 hours Apps Setup Time: 1-12 hours Restore Last Backup: 30 minutes- 1 hour sysadmin
Disaster Recovery Production (Linux 8GB) (Web Server, Application Server, Database) Production ( Windows 4GB) (Mail Server) Production ( Windows 6GB) (ERP) Test Development (Linux 4 GB) (Web Server, Application Server, Database) Virtual Machines Physical Servers Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 16 GB RAM, 200GB HDD, 4 Cores Hypervisor 8GB RAM, 200GB HDD, 6 Cores - ON SOS Our server has been melted down!!! OK I will start the last VM backups to the other servers until I find what happened developer Restore Last VM Backup: 15-30 minutes Hardware Recovery: 1 day 1 week sysadmin
PR Lab Story Pattern Recognition Laboratory 10 Researchers (Stuff, PHD and MSc Students) 3-5 Undergraduate Students 4 Servers 3 different research teams Applications 3 Web Servers + Databases (Lab and BioTools) 1 Developments Server 1 NAS Needs Run web applications Run experiments in standalone workstations Run Parallel Algorithms Experiments
Problems Not enough PCs to run experiments Old applications had to be deleted and backuped up 3 Web servers in 3 relatively overpowered machines Not having parallel computing infrastructure
Solution Install a virtualization infrastructure Xen hypervisor XenServer XenCenter for management All free Open Source Solutions
PR Lab Virtualization Infrastructure
PR Lab Virtualization Infrastructure 10G Switch XenServer 1 2 TB NAS XenServer 3 XenServer 2 XenServer 4
Now we have 14 Virtual Server in 1 physical server (50% of its resources) 5 Production Web Servers (Linux & Windows 2008) 1 Virtual Cluster of 4 VMS 1 Geo Server 5 Development Virtual Machines 2 inactive VMs from older research projects 3 physical metal servers for test purposes in 2 virtual pools Ability to easily test our applications in both Windows and Linux (Ubuntu, CentOS) Environments
Our Virtualization Management
Resources https://www.cs.indiana.edu/~achauhan/teaching/b649/2010- Spring/StudentPresns/Virtualization.pdf https://www.ibm.com/developerworks/library/l-hypervisor/ http://corensic.wordpress.com/2011/12/05/virtual-machines-virtualizing-virtualmemory/ http://www.ibm.com/developerworks/cloud/library/cl-hypervisorcompare/ http://wiki.xenproject.org/wiki/xen_overview#guest_types http://wiki.xenproject.org/wiki/virtualization_spectrum http://www.slideshare.net/xen_com_mgr/linuxcon-japan-13-10-years-of-xen-and-beyond http://en.wikipedia.org/wiki/hypervisor http://en.wikipedia.org/wiki/platform_virtualization http://www.cse.iitb.ac.in/~puru/courses/autumn12/cs695/classes/kvm-overview.pdf http://www.ibm.com/developerworks/linux/library/l-virtio/index.html http://www.cse.iitb.ac.in/~puru/courses/autumn12/cs695/classes/kvm-overview.pdf
Thank you!!! Any questions