OCCI and Security Operations in OpenStack - Overview

Transcription

1 Allocation of VMs: A primer Alex Glikson (IBM), John M. Kennedy (Intel), Giovanni Toffetti (IBM) FI-WAE Cloud Hosting Chapter June 6th,

2 Agenda Overview Web-based portal Live demo using FI-WAE testbed DCM extensions to OpenStack APIs OpenStack OCCI Summary 1

3 Functionality Trust and Security Operations FI-WAE Core Platform Overview Service delivery Cloud Hosting Internet of Things Support Services Developer tools I2ND = Interface to the Network and devices 2

4 FI-WAE Cloud Generic Enablers & Partners UI Identity Management & Access Control GE [Security WP] PaaS Management GE Software Deployment and Configuration (SDC) GE Monitoring GE IaaS Service Management (SM) GE Edgelet Management GE Object Storage GE IaaS Data Center esource Management (DCM) GE Job Scheduler GE Cloud Hosting Cloud Proxy GE [joint with I2ND WP] 3

5 FI-WAE Cloud Generic Enablers Overview UI FI-WAE Cloud GEs offer capabilities to host cloud-based applications (provisioning, management, etc) All the GEs are available free of charge in the FI-WAE Testbed, for experimentation purposes within the FI-PPP All the GEs provide EST APIs, some provide WebUI Most of the capabilities available in the 2 nd release (* = new in 2 nd release) Secure access via common Identity Management and Access Control mechanisms Identity Management & Access Control GE [Security WP] PaaS Management GE* Software Deployment and Configuration (SDC) GE* Monitoring GE* IaaS Service Management (SM) GE Edgelet Management GE* Object Storage GE IaaS Data Center esource Management (DCM) GE Job Scheduler GE* Cloud Hosting Cloud Proxy GE [joint with I2ND WP] 4

6 Infrastructure-as-a-Service (IaaS) Data Center esource Management (DCM) GE Offers cloud-based hosting of virtual machines, provisioned from a library of pre-configured images, as well as storage volumes and virtual network connectivity Provided by IBM (with contributions by Intel) Based on OpenStack leading open source IaaS API and implementation, widely adopted by the industry Will be used to host all the FI-WAE GEs in the Testbed, as well as Open Innovation Lab UI Identity Management & Access Control GE [Security WP] PaaS Management GE Software Deployment and Configuration (SDC) GE Monitoring GE IaaS Service Management (SM) GE Edgelet Management GE Object Storage GE IaaS Data Center esource Management (DCM) GE Job Scheduler GE Cloud Hosting Cloud Proxy GE [joint with I2ND WP] 5

7 Cloud Hosting Design Principles On-Demand, Self-Service: Broad Network Access. esource Pooling. apid Elasticity. Measured Service / Pay-As-You-Go. Dependability Availability: readiness for correct service eliability: continuity of correct service Safety: absence of catastrophic consequences on the user(s) and the environment Integrity: absence of improper system alteration Maintainability: ability for a process to undergo modifications and repairs Confidentiality: absence of unauthorized disclosure of information 6

8 DCM GE Design Principles When applied to DCM, the general design principles translate into the following key design goals: Fully-automated provisioning and life cycle of compute, storage and network resources, requested, managed and released via a standardsbased EST API High resource utilization, while providing the necessary levels of isolation, availability and performance of provisioned resources Ability to dynamically control the amount of allocated resources, as well as to monitor the actual resource usage High availability and scalability of the management stack Non-disruptive, automated administrative tasks (e.g., infrastructure maintenance) Avoid non-authorized access to resources and workloads 7

9 DCM GE Basic Concepts Virtual server: a virtualized container that can host an arbitrary Operating System and an arbitrary software stack on top, installed within the virtual server Virtual disk: represents a persistent virtual disk that can be potentially attached to an arbitrary virtual server Virtual network: represents a logical network abstraction that would typically represent an L2 segment Virtual image: a pre-packaged virtual server image. 8

10 DCM Architecture (2nd elease) Identity Management Service OpenStack / OCCI OpenStack + Open Cloud Computing Interface (OCCI)++ API OpenStack++ Virtual Image Mgmt Service Virtual Server Mgmt Service Virtual Disk Mgmt Service Virtual Network Mgmt Service images Image epository Placement Service Message Fabric Policy Service Configuration epository 9

11 OpenStack in a Nutshell OpenStack is a global collaboration of developers & cloud computing technologists working to produce an ubiquitous Infrastructure as a Service (IaaS) open source cloud computing platform for public & private clouds. Platinum Sponsors Gold Sponsors OpenStack Foundation: exponential growth 30 UGs 2600 Individuals 47 UGs Individuals OpenStack Compute (Nova) Provision and manage large networks of virtual machines OpenStack Storage (Cinder, Swift) Object and Block storage for use with servers and applications OpenStack Networking (Quantum) Pluggable, scalable, API-driven network and IP management OpenStack Image Service Catalog and manage massive libraries of server images OpenStack Identity Service Unified authentication across all OpenStack projects and integrates with existing authentication systems. Code available under Apache 2.0 license. Design tenets scale & elasticity, share nothing & distribute everything 10

12 Live Demo using FI-WAE Cloud Portal Use instructions at FI-WAE Catalogue to request an account Guidelines on taking snapshots in OpenStack: 11

13 DCM-specific features With respect to the OpenStack baseline, DCM provides in addition the following set of high-level advanced features: Shared storage configuration enabling live VM migration and related scenarios VM High Availability Adaptive scheduling for optimized resource utilization Support for QoS guarantees for workloads Support for placement policies Support of concurrent management and deployment workflows in a scalable consistent manner Unified management of heterogeneous environments Support for policy-based virtual network connectivity 12

14 DCM Extended capabilities Host failure detection (Zookeeper) Advanced Scheduler: Flexible global resource optimization based on large and extensible set of metrics Ongoing placement optimization using live migration and a solver Placement support for automated HA of VM instances: in an HA scenario, if a host fails, the placement logic will automatically provide correct placement for all recovered VMs Host evacuation Support for anti-affinity / placement policies HA-aware admission control Unified support for heterogeneous performance of the underlying HW Fine-grained compatibility verification Flexible resource allocation policies and adaptive resource over-commit based on idleness detection 13

15 DCM eferences DCM Specification DCM APIs: OpenStack: OCCI: 14

16 OpenStack API Walk-through 15

17 Authentication EQ: curl -d '{"auth":{"passwordcredentials":{"username": "giovanni", "password":"secret"}}}' -H "Content-type: application/json" ES: {"access": {"token": {"expires": " T09:50:28Z", "id": "fb4a4642f4b848f48d78052c98782ba3"}, "servicecatalog": {}, "user": {"username": "giovanni", "roles_links": [], "id": "dcdef6b4d8644d209d066ce1ebcf7994", "roles": [], "name": "giovanni"}}} 16

18 Get TenantId EQ: curl -k -D - -H "X-Auth-Token: fb4a4642f4b848f052c98782ba3" -X 'GET' -v -H 'Content-type: application/json' ES: {"tenants_links": [], "tenants": [{ } "enabled": true, "description": "FI-WAE Project", "name": "FIWAE", "id": "3bb69a bfc98e0199cb33000ea }] 17

19 Discover endpoints EQ: curl -d '{"auth":{"passwordcredentials":{"username: "giovanni", "password":"secret"}, "tenantid": "3bb69a bfc98e0199cb33000ea"}}' -H "Content-type: application/json" ES: "servicecatalog": [{ }], }, "endpoints": [{ "adminul": " "region": "egionone", "internalul": " "publicul": " "endpoints_links": [], "type": "compute", "name": "nova" 18

20 List images EQ: curl -k -D -H "X-Auth-Token: $token" -X 'GET' -H 'Content-type: application/json ES: { "images": [ { "id": "a5848d58-f a-9bf0-e8f1cf63256b", "links": [{ "href": ], "rel": "self"}, { "href": ", "rel": "bookmark" }, { } "href": ", "type": "application/vnd.openstack.image", "rel": "alternate" "name": "Ubuntu_12.04_cloudimg_amd64_VNC-r2 }, 19

21 List flavors EQ: curl -k -D - -H "X-Auth-Token: $token" -X 'GET' -H 'Content-type: application/json ES: { "flavors": [{ "id": "3", "links": [{ "href": "rel": "self" }, { "href": ", "rel": "bookmark" } ], "name": "m1.medium" },. 20

22 Create instance EQ: curl -k -H "X-Auth-Token: d889a8fdf8fb4d52b6e1b8771d0f59b8" -X 'POST' -H 'Content-type: application/json' -d '{"server": {"flavoref": " "imageef": " e b65c-f f6e6", "metadata": {"My Server Name": "Demo_server" }, "name": "api-server-test"}}' ES: {"server": {"OS-DCF:diskConfig": "MANUAL", "id": "32e36513-dfc7-4dd6-bc31-206cbda85297", "links": [{"href": " dfc7-4dd6-bc31-206cbda85297", "rel": "self"}, {"href": " dfc7-4dd6-bc31-206cbda85297", "rel": "bookmark"}], "adminpass": "SBr5wXMqbqLj"}} 21

23 List instances EQ: curl -k -D - -H "X-Auth-Token: d889a8fdf8fb4b8771d0f59b8" -X 'GET' rvers -H 'Content-type: application/json' ES: { "servers": [{ "id": "32e36513-dfc7-4dd6-bc31-206cbda85297", "links": [{"href": " ervers/32e36513-dfc7-4dd6-bc31-206cbda85297", "rel": "self" }, { "href":..", "rel": "bookmark" } ], },. "name": "api-server-test" 22

24 Stop instance EQ: curl -k -H "X-Auth-Token: $token" -X 'DELETE' rvers/32e36513-dfc7-4dd6-bc31-206cbda H 'Content-type: application/json' ES: HTTP code

25 A Standardised Interface OpenStack Nova implements its own custom API See: There are alternatives: standard cloud management APIs Open Cloud Computing Interface a working group under the Open Grid Forum (OGF) a living, open specification ESTful, extensible by design proven interoperability in cloud management over 20 implementations providing interoperability insurance 24

26 Terminology Concept Category Kind Mixin Action Entity esource Link Query Interface (QI) CUD Description Way of describing type(s), attributes, actions and relationships for entities Type of a resource Ephemeral Type of a resource; Mechanism for Tagging & (esource/os) Templating A push the button operation A resource later discoverable through the interface A resource with its attributes, links and applicable actions Exposes relationship between resources A way of discovering Capabilities Create, etrieve, Update and Delete ops on QI and Entities 25

27 OCCI for OpenStack Easy to install Supports all mandatory OCCI functionality Complete VM management including Basic VM operations Block Storage Manipulation VM resizing Security group and rule manipulation Floating IP management VMs not created through OCCI (e.g. through Horizon Dashboard) Currently supports OpenStack Folsom, Grizzly support imminent OpenSource: Source code: Binary: 26

28 OCCI API Walk-through 27

29 Authentication equest: curl -d '{"auth": {"project": "admin", "passwordcredentials": {"username":"admin", "password": "},"tenantid":"d418851c6d294381bbe6e d6"}} ' -H"Content-type: application/json" esponse: {"access": {"token": {"expires": " T13:39:12Z", "id": "ea379b97b79c4f1bbaaedfa58ad48e82", [ ] [ ] 28

30 Discover capabilities equest: curl -v -X GET -H 'X-Auth-Token: '$KID esponse: < HTTP/ OK < Content-Length: < Content-Type: text/plain < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :21:15 GMT Category: Ubuntu_12.04_cloudimg_amd64_VNC; scheme=" class="mixin"; title="this is an OS Ubuntu_12.04_cloudimg_amd64_VNC VM image"; [ ] Category: m1.small; scheme=" class="mixin"; title="this is an openstack m1.small flavor."; [ ] Category: compute; scheme=" class="kind"; title="compute esource";[ ] [ ] 29

31 Create security group equest: curl -v -H 'X-Auth-Token: '$KID -H 'Content-Type: text/occi' -H 'Category: my_grp; scheme=" rel=" location="/my_sec_group/"' -X POST esponse: < HTTP/ OK < Content-Length: 187 < Content-Type: text/plain < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :43:10 GMT < Category: my_grp; scheme=" class="mixin"; rel=" location=" 30

32 Create security rule equest: curl -v -X POST :8787/network/security/rule/ -H 'Content- Type: text/occi' -H 'X-Auth-Token: '$KID -H 'Category: my_grp; scheme=" class="mixin"' -H 'Category: rule; scheme= [ ] esponse: < HTTP/ Created < Content-Length: 2 < Content-Type: text/plain < Location: < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :45:10 GMT < OK 31

33 Create VM equest: curl -v -X POST :8787/compute/ -H 'Category: compute; scheme=" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'Category: m1.small; scheme=" class="mixin"' -H 'Category: Ubuntu_12.04_cloudimg_amd64_VNC; [ ] esponse: < HTTP/ Created < Content-Length: 2 < Content-Type: text/plain < Location: fb89c89f5107 < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :48:45 GMT < OK 32

34 List VMs equest: curl -v -X GET :8787/compute/ -H 'Accept: text/uri-list' -H 'X- Auth-Token: '$KID esponse: < HTTP/ OK < Content-Length: 86 < Content-Type: text/uri-list < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :51:23 GMT < # uri:/compute/ 33

35 Create block storage device equest: curl -v -X POST :8787/storage/ -H 'Category: storage; scheme=" class="kind"' -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'X-OCCI-Attribute: occi.storage.size = 1.0' esponse: < HTTP/ Created < Content-Length: 2 < Content-Type: text/plain < Location: < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :53:15 GMT < OK 34

36 Link VM and Storage equest: curl -v -X POST :8787/storage/link/ [ ] -H 'X-OCCI-Attribute: occi.core.source=" fb89c89f5107"' -H 'X-OCCI-Attribute: occi.core.target=" [ ] esponse: < HTTP/ Created < Content-Length: 2 < Content-Type: text/plain < Location: 532a665cf1ad < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :56:20 GMT < OK 35

37 List Instance equest: curl -v -X GET :8787/compute/edaab8cd-20a ea2-fb89c89f5107 -H 'X-Auth-Token: '$KID esponse: [ ] Category: compute; scheme=" class="kind"; [ ] Category: m1.small; scheme=" [ ] Category: Ubuntu_12.04_cloudimg_amd64_VNC; scheme=" class="mixin"; [ ] Link: </storage/10>; rel=" self="/storage/link/6cec11d cc-a85d-532a665cf1ad"; [ ] X-OCCI-Attribute: occi.compute.memory="2.0" 36

38 Stop a VM equest: curl -v -X POST " :8787/compute/edaab8cd-20a ea2- fb89c89f5107?action=stop" -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'Category: stop; scheme=" class="action"' esponse: < HTTP/ OK < Content-Length: 4359 < Content-Type: text/plain < Server: pyssf OCCI/1.1 < Date: Wed, 07 Nov :01:37 GMT < Category: compute; scheme=" class="kind"; title="compute esource"; rel=" [ ] 37

39 In Summary DCM extends the OpenStack baseline with several advanced features (e.g., shared storage, VM HA, adaptive scheduling, placement policies, policy-based virtual networking) It can be accessed through 2 different APIs: Native OpenStack API OCCI Implementation for OpenStack Source code: Binary: Grizzly support imminent 38

40 Thank you! 39

41 Questions For additional questions: 40