Openstack Cloud computing with Openstack Saverio Proto saverio.proto@switch.ch Lugano, 23/03/2016
Agenda SWITCH role in Openstack and Cloud Computing What is Virtualization? Why is Cloud computing more than Virtualization? How Openstack implements Cloud Computing? 2
SWITCHengines Openstack public cloud with two regions in Switzerland https://www.switch.ch/engines/ SWITCHengines provides compute and storage services to Swiss academics. Users of participating institutions have on-demand access to create and manage virtual machines and storage. Two Regions Zurich Lausanne 3
Openstack Current status at SWITCH SWITCHengines has been in public beta and internal production since December 2014 Productive service since January 2016 Several SWITCH services run on it (SWITCHdrive, SWITCHfilesender, SWITCHtube) Over 630 individual users and around 40 research projects online Around 700 VMs running on the cloud right now (2200 cores, 4900 GB RAM, 400 TB of storage allocated) Started with Openstack Icehouse, we have upgraded the system to Juno (8/2015) and then to Kilo (3/2016) 4
The starting point: Virtualization Key idea: Hypervisor and virtual machines Commodity technology today Typically GUI driven Scale-UP approach Operators manage each Hypervisor independently Well known Hypervisors, and date of birth VMWare 2001 Xen 2003 Virtualbox 2007 KVM 2007 https://en.wikipedia.org/wiki/timeline_of_virtualization_development 5
Virtualization before Cloud Computing How to manage a datacenter with many hypervisors? Most of the management is by hand with GUI/WEB based tools What happens if I lose a physical host? Hypervisors are built with a lot of redundancy to prevent failures All the hypervisors in a legacy datacenter are Pets, you really have to make sure they are healthy How to manage multi-tenant use cases? Not all the VMs belong to the same project Separation is possible but configured by hand 6
Virtualization networking Hypervisor Network ETH0 br0 VM1 eth0 VM2 eth0 7
Datacenter Switch Virtualization Reference Network Network configuration can become complex and hard to manage by hand In case of multi-tenancy VLAN coordination needed by network-admin and server-admin 8
Legacy Virtualization - Cons Needs redundancy for hypervisors hardware Needs redundancy for networking hardware A lot of manual configuration No standard strategy for multi-tenancy 9
Cloud is already 10 years old AWS: Started in March 2006 (10 years ago) Openstack: Started in 2010 10
Public and Private Cloud In a Public cloud the services are offered to anyone over the Internet. Operators and users of the cloud are usually in different domains In a Private cloud the access is restricted to some users for private use Operators and users of the cloud are usually from the same company 11
What is new with Cloud Computing? It is design to fail Hardware is divided in two main categories, Pets and Cattle You will need to introduce redundancy only for your Pets Cattle are allowed to fail API based Control Plane The configuration of new VMs is done interacting with a Controller API interfaces make possible to automate the configuration. Requests for new virtual machines are processed by a controller that will schedule the machine to right hypervisor. 12
Pet and Cattle It is a change regarding Uptime responsibility. Hypervisors can fail, this means that VMs should be disposable. Developers are required to design the application in a way that a server can be lost. The application should exploit the new abstractions provided by the cloud. 13
Web Application Example: Legacy Scale-up mindset Make a server redundant and powefull Hypervisor2 backup RAID5 disks Hypervisor1 RAID5 disks Standby VM Web Server VM Hostname www.mysite.com Database Backup Database VM 14
Web Application Example: Cloud Scale-out mindset VMs are disposable FloatingIP LBaaS Hostname www.mysite.com Database is provided by the Cloud Infrastructure Object Storage is persistent vm01 vm02 vm03 Autoscaling VMs on load DBaaS Object Storage 15
API based Control Plane API can be accessed via CLI, Web based interface, and libraries are provided to implement own tools openstack server create --flavor c1.micro \ --image Ubuntu 14.04 \ --nic net-id=private \ myvmname 16
API based Control Plane The best hypervisor is selected automatically User starting a VM do not know how many hypervisors Multi-tenancy is by default Each user have visibility only of his own VMs VMs from the same user will appear on the same network 17
Scale out To make scale-out possible, a cloud computing provides the following abstractions: Web Services FloatingIPs and LBaaS Databases DBaaS : the user receives the endpoint and credentials Real DB is managed by Cloud Ops in multi master replication Storage Object store 18
New abstraction: Object Storage Scalable Access natively via HTTP Replication of objects User Object storage File System File System File System File System File System Server Server 19
New abstraction: floating IP Quickly move an IP address to another instance or to another service. The backend is independent. FloatingIP FloatingIP FloatingIP LBaaS vm01 vm01 Bigger VM vm01 vm02 vm03 20
Openstack components Main components Keystone: authentication and authorization Nova: Virtual Machine orchestrator Cinder: Volume manager Glance: Image manager Neutron: Network controller The openstack big tent Horizon Heat Trove Ceilometer 21
Keystone Keystone provides a token based authorization to the Openstack API services. export OS_USERNAME=saverio.proto@switch.ch export OS_PASSWORD=<secret> export OS_TENANT_NAME=saverio.proto@switch.ch export OS_AUTH_URL=https://keystone.cloud.switch.ch:5000/v2.0 export OS_REGION_NAME=LS Keystone publishes a catalog of the current API endpoint for services 22
Keystone Cloud Federation is possible with keystone However requires Keystone V3 API Federation enables Cloud Bursting Keystone development is today very active Token structures changed 23
Glance Glance is operating system image repository It works with different storage backends macsp:~ proto$ openstack image list +--------------------------------------+------------------------------------------+--------+ ID Name Status +--------------------------------------+------------------------------------------+--------+ 382099dd-61b5-4866-9956-bc68abb417dd RStudio Appliance (SWITCHengines) active 8bc628a2-a54c-4ee4-bda5-3325a1d12cb0 Ubuntu Trusty 14.04 (SWITCHengines) active a4e35b28-b1fe-40a0-ba33-f23f0a9a7440 CentOS 7.1 (SWITCHengines) active 1102bb9c-0256-4a4f-a104-f57c9715ae1b Debian Wheezy 7.8 (SWITCHengines) active 31a9beca-acc5-43de-9e52-01256ef211d3 Fedora release 20 (SWITCHengines) active 7ac1f5f9-d54c-43ef-9c4c-2b1e69603a03 Debian Jessie 8.1 (SWITCHengines) active 8e4c68d0-6d20-4767-9d5a-f72db43995b0 mybackup-19-jan-16 active 0a4daefd-83d0-48e9-b3ee-e1a5e6dcbf9d CoreOS (SWITCHengines) active dd0659cc-ee08-45c0-9658-7d5441d726b9 Windows 2012 Server EVAL (SWITCHengines) active +--------------------------------------+------------------------------------------+--------+ 24
Cinder Cinder is the volume manager in Openstack Because we said that VMs are disposable, we have two kind of volumes Ephemeral volumes Persistent volumes Ephemeral volumes Controlled directly by Nova, the disk of the VMs is destroyed when the VM is destroyed. Created when we boot from image. Persistent Is a volume independent from a VM. You can attach the volume to multiple VMs 25
Neutron VMs from the same project appear on the same network regardless of the hypervisor where they are scheduled Hypervisors are running OpenVSWITCH An Overlay network is built among the Hypervisors Different tunnels are possible (VXLAN, GRE) 26
Network architecture 27
Nova Nova is a cloud computing fabric controller It is software layer that hides to the user the hypervisor technology chosen to implement the cloud. The cloud can use even different hypervisor technology at the same time. Supports KVM, XEN, VMWare, Hyper-V, Bare-metal, Containers 28
Nova API User requests a VM to Nova Based in the flavor nova schedules the VM to the appropriate hypervisor User NOVA API XEN hypervisor KVM hypervisor VM VM VM VM VM 29
Reference scenario - Hardware http://docs.openstack.org/liberty/networkingguide/scenario-classic-ovs.html 30
Reference scenario - Network 31
Reference scenario Service 32
Conclusion Openstack is a great Open Source software ecosystem It makes possible to anyone to deploy a cloud Application should be designed to exploit the new abstractions Cloud Native Applications Cloud design with Pets and Cattles reduces infrastructure costs 33
Questions? 34
Use case on SWITCHengines ATLAS Experiment at CERN, UNIBE is using SWITCHengines 24/7 with 300 Cores ATLAS Instances on SWITCHengines 8cores 16GB RAM 20 GB Disk 35