Today. 1. Private Clouds. Private Cloud toolkits. Private Clouds and OpenStack Introduction

Transcription

1 Today Private Clouds and OpenStack Introduction 1. Private Clouds 2. Introduction to OpenStack What is OpenStack? Architecture and Main components Demo: basic commands Luis Tomás Department of Computing Science 1. Private Clouds Private Cloud toolkits IaaS infrastructure Built for exclusive use of one client full control over infrastructure, data, quality of service, etc. Private Company owns infrastructure may be located at enterprise or at colocation Local (if in-house) Built and managed by enterprise IT or cloud provider Most have stack in their name: Covers the stack from layer 1 (physical) to layer 7 (applications) of the OSI (Open Systems Interconnection) model Quite a few toolkits, and even more debate Examples OpenStack CloudStack OpenNebula Eucalyptus 1

2 Private Cloud toolkits OpenStack Started by Rackspace (storage files) and NASA in 2010 Both Ubuntu and Red Hat distributions Hypervisors: KVM, Xen, VmWare,,even baremetal CloudStack Apache project OpenStack owner (Citrix) now own CloudStack (2009) and dropped OpenStack More Amazon like Hypervisors: KVM, vsphere, XenServer, Oracle VM More private Cloud toolkits OpenNebula Developed 2008, Complutense University of Madrid, Spain Xen, KVM, VmWare Interfaces: EC2, OGF OCCI, vcloud Large open-source community Nimbus Developed 2009 EC2/S3 compatible Xen, KVM Combine with OpenStack, Amazon, others Even more Eucalyptus Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems Developed 2008 VMware, Xen, KVM Ubuntu, RHEL, and other flavors of Linux Only project based on GPL (GNU general SW license) and not ASL (Apache SW license) Implement AWS API on top of Eucalyptus AWS agrees to support Eucalyptus, users can migrate workloads between the two, applications compatible with both Agenda 1. Private Clouds 2. Introduction to OpenStack What is OpenStack? Architecture and Main components Demo: basic commands 2

3 2. Introduction to OpenStack 2. Introduction to OpenStack Initiated by Rackspace Cloud and NASA in 2010 Supported by an industry consortium IBM Canonical AT&T Rackspace HP RedHat The Cloud Operation System The biggest open source project in the world > people, >400 companies, >140 countries, >20M lines of code, What is OpenStack? What is OpenStack? Mission statement To produce the ubiquitous open source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implement and massively scalable. Open Open source Open design Open development Open community OpenStack Foundation OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a data center, all managed through a dashboard that gives administrators control while empowering users to provision resources through a web interface. 3

4 OpenStack: Overview Written (mostly) in Python All code available as open source Use Devstack available for more convenient development Other tools available, such as RDO from RedHat Ongoing projects for quick/easy installation/ configuration (TripleO) OpenStack inside OpenStack Ongoing effort Changing rapidly DevStack template SERVICE_TOKEN=azertytoken ADMIN_PASSWORD=nomoresecrete MYSQL_PASSWORD=stackdb RABBIT_PASSWORD=stackqueue SERVICE_PASSWORD=$ADMIN_PASSWORD # ``HOST_IP`` should be set manually for best results if the NIC configuration # of the host is unusual, i.e. ``eth1`` has the default route but ``eth0`` is the # public interface. It is auto-detected in ``stack.sh`` but often is indeterminate # on later runs due to the IP moving from an Ethernet interface to a bridge on # the host. Setting it here also makes it available for ``openrc`` to include # when setting ``OS_AUTH_URL``. # ``HOST_IP`` is not set by default. #HOST_IP=w.x.y.z # Logging # LOGFILE=$DEST/logs/stack.sh.log # Using milestone-proposed branches #CINDER_BRANCH=milestone-proposed #GLANCE_BRANCH=milestone-proposed DevStack template ENABLED_SERVICES=rabbit,mysql,key ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch,n-novnc,n-cauth ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-lbaas ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account ENABLED_SERVICES+=,g-api,g-reg ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch,c-bak ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond ENABLED_SERVICES+=,horizon ENABLED_SERVICES+=,ceilometer-acompute,ceilometer-acentral,ceilometercollector,ceilometer-api Q_PLUGIN=ml2 ENABLE_TENANT_VLANS=True Q_AGENT=openvswitch ML2_VLAN_RANGES=physnet1:100:400 PHYSICAL_NETWORK=physnet1 OVS_PHYSICAL_BRIDGE=br-ex RDO template # Set to 'y' if you would like Packstack to install OpenStack Compute # (Nova) CONFIG_NOVA_INSTALL=y CONFIG_NEUTRON_INSTALL=y CONFIG_HORIZON_INSTALL=y CONFIG_SWIFT_INSTALL=y CONFIG_CEILOMETER_INSTALL=y CONFIG_HEAT_INSTALL=n CONFIG_CLIENT_INSTALL=y... # The IP address of the server on which to install OpenStack services # specific to controller role such as API servers, Horizon, etc. CONFIG_CONTROLLER_HOST= # The list of IP addresses of the server on which to install the Nova # compute service CONFIG_COMPUTE_HOSTS= , , , PUBLIC_INTERFACE=eth1. 4

5 RDO template OpenStack: Overview CONFIG_NEUTRON_ML2_TYPE_DRIVERS=vlan # A comma separated ordered list of network_types to allocate as # tenant networks. The value 'local' is only useful for single-box # testing but provides no connectivity between hosts. CONFIG_NEUTRON_ML2_TENANT_NETWORK_TYPES=vlan # A comma separated ordered list of networking mechanism driver # entrypoints to be loaded from the neutron.ml2.mechanism_drivers # namespace. CONFIG_NEUTRON_ML2_MECHANISM_DRIVERS=openvswitch # A comma separated list of physical_network names with which flat # networks can be created. Use * to allow flat networks with arbitrary # physical_network names. CONFIG_NEUTRON_ML2_FLAT_NETWORKS=*. Design and coding standards Scalability and elasticity Asynchronous Horizontally scalable Share-nothing (or sharding) Distribute everything Accept eventual consistency Test everything Enforce PEP-8 6 month release cycle OpenStack: Overview Release history OpenStack: Overview Austin Bexar Cactus Diablo Essex Folsom Grizzly Havana Icehouse Juno Kilo? October 2010 February 2011 April 2011 September 2011 April 2012 September 2012 April 2013 October 2013 April 2014 October 2014 April 2015 October 2015? Why Service Providers Are Adopting? Originated by Rackspace and NASA Designed to scale cost effectively Emerging standard backed by large ecosystem Open source with no lock-in or license No desire to build proprietary clouds 5

6 OpenStack: Overview Contributors OpenStack: Overview Contributors Agenda Conceptual Architecture 1. Private Clouds 2. Introduction to OpenStack What is OpenStack? Architecture and Main components Demo: basic commands 6

7 Architecture Architecture Consists on several (main) modules: Compute (Nova) Network (Neutron) Image storage (Glance) Block storage (Cinder) Object storage (Swift) Identity (Keystone) Metering (Ceilometer) Orchestration (Heat) Dashboard (Horizon) And many more: Database Service (Trove) Deployment (TripleO) Common Libraries (Oslo) Bare metal (Ironic) Queue service (Marconi) Data processing (Sahara) Key management (Barbican) Logical (Core) Architecture: Icehouse Modules Compute - Nova Controls the cloud computing fabric VM management through the hypervisor Launch VM Start/Stop/Suspend/Resume/Reboot VM Resize/Migrate/LiveMigrate VM Format for describing VMs (Flavor) Supports popular Hypervisors KVM, Xen, or VMWare LXC (Container-based) 7

8 Modules Modules Compute - Nova Technology agnostic: Virtual machine or container virt. Virtualization agnostic Libvirt (KVM, QEMU, Xen, LXC), XenAPI, Hyper-V, VMware ESX, PowerVM, Docker, Bare-metal Initiates most of the orchestration activities, such as running or migrating an instance Networking - Neutron Network infastructure management Concepts & User configurable topology Networks, Routers, Subnets, Ports Multiple technologies OpenvSwitch, Linux Bridge, Vendor plugins Floating IP addresses Assigning fixed external IPs to VMs Modules Modules Block Storage - Cinder Persistent block storage, named Volumes Multiple storage backends eg LVM, RBD, Gluster, Sheepdog,...+ more... Exposed to compute host via iscsi: Internet Small Computer System Interface Internet protocol (IP)-based storage networking standard for linking data storage facilities Carry SCSI commands over IP netowrks Identity Management - Keystone Single point of integration for OpenStack policy, catalog, token and authentication Central authentication mechanism Directory of users Supports user and system policies Catalog of OpenStack services Can interact with (among others) LDAP Oauth PAM 8

9 Modules Modules Object Storage - Swift Redundant storage system for static data VM Images Backups Archives Data replication and distribution Multi-datacenter distribution Horizontally scalable Add more storage servers if needed Similar to amazon S3 Object Storage - Swift Stores and servers objects (files) Accessed via API or HTTP requests Employs object level replication to safeguard data Maintains distributed account and container databases Modules Modules Image Service - Glance Support for VM images Discovery Registration Activation Snapshots/Backup Write-once, read-many storage of images Interacts with backend storage OpenStack Object storage Image copied on use by Nova Image Service - Glance Supports most common formats RAW, VHD, VDI, VMDK, qcow2, OVF, Metadata properties eg specify virtual hardware preferences 9

10 Modules Modules Dashboard - Horizon Django application that users can access in their web browser Web UI for administrators Manage projects, users, and resources Communicates with each OpenStack service via their API Metering - Ceilometer Billing and monitoring Orchestration - Heat Template based And many more Nova Nova Architecture Main, central component VM scheduling VM creating/migration/ termination/ Concepts Instances Flavours Virt drivers Security group Dual APIs OpenStack REST EC2 compatible REST 10

11 Nova Scheduler Places instances on compute hosts Filter Scheduler: Pluggable filtering rules CPU model / architecture Virtualization type PCI device availability CPU, RAM, Disk usage +more... Nova Compute The nova-compute process is primarily a worker daemon that creates and terminates virtual machine instances via hypervisor s APIs The process by which it does so is fairly complex but the basics are simple: Accept actions from the queue And then perform a series of system commands (like launching a KVM instance) to carry them out while updating state in the database (through nova-conductor) Instance Boot Instance Boot 11

12 Instance Boot Instance Boot Instance Boot Instance Boot 12

13 Agenda 1. Private Clouds 2. Introduction to OpenStack What is OpenStack? Architecture and Main components Demo: basic commands OpenStack Demo Basic commands A few commands to: List VMs, volumes, network, flavors Create keys, volumes and networks Create VMs Attach/Detach volumes to VMs Attach floating IPs to VMs Stop/Remove VMs Live-migrate VMs OpenStack demo List existing VMs Volumes Networks OpenStack demo Flavor list Flavor attributes

14 OpenStack demo Create and show keys OpenStack demo Create volumes OpenStack demo Create virtual networks OpenStack demo Create virtual networks

15 OpenStack demo Create VMs OpenStack demo Network connectivity and floating IPs 57 OpenStack demo Network connectivity and floating IPs OpenStack demo Attach/Detach Volumes

16 Upcoming Next (and last) lecture Cloud Application Development #2 (2*45 min) Course summary for exam (45 min) Exam Register no later than (tomorrow!) 13-17, lecture room N450 Old example exam to be added to course web Assignment 3 (project) to be posted today. 3 deliverables: (Short) project plan. Due: Presentation and live demo. Due: Project Report. Due