KVM, OpenStack and the Open Cloud SUSECon November 2015 Adam Jollans Program Director, Linux & Open Virtualization Strategy IBM
Agenda A Brief History of Virtualization KVM Architecture OpenStack Architecture KVM and OpenStack Case Studies NTT Com Intel IT CERN Next Steps 2
A Brief History of Virtualization KVM hypervisor LXC / Docker x86 hardware virtualization Xen hypervisor for x86 VMware hypervisor for x86 Virtualization on Unix systems Virtualization on mainframes 1960s 1980s 1990s 2000s 2010s 2015 3
KVM Architecture
Conceptual Framework User Interface Applications Management Tools Storage Compute Networking 5
Introduction to KVM User Interface Applications ovirt Kimchi libvirt Management Tools KVM Storage Compute Networking 6
KVM Architecture Open source hypervisor Based on Linux Virtual Machine Linux Applications Virtual Machine Other Applications KVM Kernel module that turns Linux into a Virtual Machine Monitor Merged into the Linux kernel Linux Guest OS QEMU Other Guest OS QEMU Linux Applications QEMU Emulator used for I/O device virtualization KVM Linux Processors supported x86, POWER, z Systems, ARM x86 with virtualization extensions POWER8 IBM z Systems ARM64 7
KVM Performance SPECvirt_sc2013 VMWare, Intel Xeon E5-16 cores (HP) KVM, Intel Xeon E5-32 cores (HP) KVM, Intel Xeon E5-16 cores (IBM) KVM, Intel Xeon E5-24 cores (IBM) KVM, Intel Xeon E5-24 cores (IBM) KVM, Intel Xeon E5-60 cores (IBM) KVM, Intel Xeon E5-60 cores (IBM) KVM, Intel Xeon E7-120 cores (Lenovo) KVM, Intel Xeon E7-72 cores (Lenovo) KVM, Intel Xeon E5-36 cores (HP) KVM, Intel Xeon E7-72 cores (HP) KVM, Intel Xeon E5-16 cores (HP) KVM, Intel Xeon E5-36 cores (Huawei) KVM, Intel Xeon E7-60 cores (Huawei) PowerVM, IBM POWER8-24 cores (IBM) 0 500 1000 1500 2000 2500 3000 3500 4000 Source: SPECvirt_2013 Published Results - http://www.spec.org/virt_sc2013/results/specvirt_sc2013_perf.html 8
KVM Security SELinux Virtual Machine Linux Applications Virtual Machine Other Applications Mandatory Access Control (MAC) integrated into Linux Provides need to know security between processes svirt Linux Guest OS QEMU Other Guest OS QEMU Linux Applications Combines SELinux and KVM Delivers need to know security between virtual machines KVM Linux Certifications x86, POWER, z Systems, ARM EAL4+ certification for KVM in SLES 11 SP 2 and RHEL 6 on various x86 64-bit Intel and AMD64-based hardware from Dell, HP, IBM and SGI 9
KVM Management - libvirt User Interface Library Open Source project Command Line Remote Management Manages multiple hypervisors Command Line Powerful libvirt Library Network Daemon Complex to use Network Daemon KVM Xen LXC. Compute Enables remote management Base for other management tools virt-manager, Kimchi, ovirt OpenStack 10
KVM Management - Kimchi Command Line User Interface Remote Management Kimchi Open Source project Manages KVM on x86, Power User Interface Easy to use libvirt KVM Xen LXC. Library Network Daemon Access from HTML5 web browser Servers managed Single digits Compute 11
KVM Management - ovirt User Interface ovirt Open Source project Web Portals Command Line User Interface Web portals Command line, API ovirt ovirt Engine ovirt Node VDSM + libvirt KVM ovirt Engine Manages VMs Configures storage, network ovirt Nodes Run virtual machines Storage Compute Servers managed Tens to hundreds 12
KVM Futures Heterogeneous processor support ARM POWER System z GPUs Network Function Virtualization Additional Performance Improvements Nested Virtualization Containers with Virtualization 13
OpenStack Architecture
Building Open Clouds Security Resilience Performance Scalability thousands of nodes Heterogeneity Interoperability 15
Introduction to OpenStack User Interface Applications Open Stack Horizon Ceilometer Command Line Management Tools Keystone Heat Cinder Swift Glance Nova Neutron Sahara Trove Choice of storage Choice of hypervisor Choice of network Storage Compute Networking 16
OpenStack Design Principles Open Open Development Model Open Design Process Open Community General Purpose Balancing Compute, Storage, Network Massively Scalable Multi-site Resilient and recoverable 17
Nova Compute Service Manages VM lifecycle Starting and stopping VMs Horizon Command Line Scheduling and monitoring VMs Key Components Swift Glance Keystone Nova API Database Scheduler VM Images Storage Choice of hypervisor Compute Compute node and plug-ins Authentication Keystone Access to VM images Glance Swift 18
Keystone - Authentication Service Manages security Service for all other modules Horizon Command Line Authentication Authorization Keystone Key components API Backends Token Catalog Policy Identity 19
Cinder Block Storage Service Manages persistent block storage Provides volumes to running instances Horizon Cinder Command Line Keystone Pluggable driver architecture High Availability Key components API Queue Choice of Block Storage Database Scheduler Storage Storage plug-ins Authentication Keystone 20
Neutron Networking Service Manages networking connectivity Provides volumes to running instances Horizon Cinder Command Line Keystone Pluggable driver architecture Supports range of networking technologies Key components API Queue Choice of Block Storage Database Scheduler Storage Agent Networking plug-ins Authentication Keystone 21
Glance Image Service Manages VM images Catalog of images Horizon Command Line Search and registration Fetch and delivery Swift Glance Keystone Key components API Registry VM Images Database Authentication Storage Keystone Storage of VM images Swift Local file system 22
Swift Object Storage Service Manages unstructured object storage Horizon Swift Command Line Keystone Highly scalable Durable three times replication Distributed Key components Proxy / API Object Storage Rings Accounts Containers Storage Objects Data stores Authentication 23 Keystone
Provisioning a VM User Interface Applications 7 Horizon 2, 10 Command Line 1 Management Tools Keystone Cinder Swift 4 Glance Nova 9 8 Neutron 6 3 5 Storage Compute Networking 24
OpenStack New Features - Kilo Horizon Updated user interface Glance Additional artifacts beyond just images Ironic Bare Metal Provisioning Zaqar Messaging and Queuing System 25
KVM and OpenStack
KVM and OpenStack KVM excels at choice criteria for Hypervisor Cost Scale & Performance Security Interoperability Development Affinity Both open source projects KVM is default hypervisor for OpenStack development Deployment Affinity KVM is best supported, easiest to deploy, with most full-featured driver 27
OpenStack and Hypervisor Usage Source: OpenStack User Survey May 2015 - http://superuser.openstack.org/articles/openstack-users-share-how-their-deployments-stack-up 28
Case Studies
NTT Com s OpenStack Deployment NTT Com Leading global carrier headquartered in Japan Early adopter of both KVM and OpenStack Basing one of its public cloud offerings on OpenStack and KVM NTT involvement Actively involved with the OpenStack and KVM communities Continues to contribute to the development of both projects, with an emphasis on the cloud service provider use case Use of OpenStack Flexible plug-in infrastructure used as a unified orchestrator of both computing and networking resources Integrate software-defined-networking (SDN)-powered enterprise VPN service, allowing customers to create virtual datacenters that can span two or more physical ones GUI portal for its cloud services using OpenStack native APIs, letting customers provision and manage virtual machines, networks, and storage without having to know the OpenStack APIs Source: IDC white paper KVM Open Source Virtualization for the Enterprise and OpenStack Clouds on OVA website 30
Intel IT s Cloud Goals 80% Effective Utilization Efficiency through federation Velocity Increase Agility through automation & self service Pervasive virtualization (> 75%) > 90% new land in cloud Enterprise app virtualization Secure virtualization Larger pools in fewer data centers On-demand self-service the norm Provision VMs within minutes Innovative idea to production < day External cloud for burst demand Zero Business Impact Reduce MTTR App design for failure Increase availability Source: Open Virtualization Alliance presentation by IBM and Intel at LinuxCon North America 2015 31
Intel IT & OpenStack / KVM Deployment History Public Public Initial Deployment 2012 Today OpenStack Essex ~1000 virtual instances for external services qemu-system-x86_64 1.0 OpenStack Havana (Juno upgrade soon) ~4000 instances for multiple services (~70:1, ~100 vcpu) qemu-system-x86_64 1.4.2 Source: Open Virtualization Alliance presentation by IBM and Intel at LinuxCon North America 2015 32
Intel IT & OpenStack / KVM KVM Benefits Performance 2012 Study on standard cloud workloads (database) Par or better vs. marketplace HV realm is seemingly near-stable on straight performance Stability Open Source, tight OpenStack and Linux kernel integration Hypervisor efficiency Drinking our own champagne - we ve got a few KVM devs :-) Performance Check flags lots of features/options Windows guest updates Keep your images current KVM Lessons Learned Stability Oversubscribing & big multi-vcpu instances Windows guest can be sensitive IO interruptions Its not good enough to have a cloud environment, applications need to evolve to become more cloud aware Source: Open Virtualization Alliance presentation by IBM and Intel at LinuxCon North America 2015 33
CERN Private Cloud CERN Fundamental research into particle physics Large Hadron Collider seeking to find new particles Massive need for scalable computing resource on demand CERN Private Cloud Production since July 2013 with OpenStack using KVM, MySQL and RabbitMQ Currently 3,200 hypervisors with 83,000 cores Expected to reach over 100,000 cores by 2Q 2015 Key Requirements Scale Technology and Developer ecosystem Interaction with existing IT services Source: CERN OpenStack public reference on www.openstack.org 34
Next Steps
Additional Resources Open Virtualization Alliance https://openvirtualizationalliance.org IDC White Paper KVM Open Source Virtualization for the Enterprise and Open Stack Clouds Linux Foundation Training Course LFS540 Linux KVM Virtualization KVM Forum August 19-21 http://events.linuxfoundation.org/events/kvm-forum OpenStack Foundation http://www.openstack.org 36
Visit www.openvirtualizationalliance.com Try out KVM on your choice of hardware architecture Thank you. 37
Unpublished Work of SUSE. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary, and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.