Dirk Vogelsang Senior Storage Architect May OpenStack & Cloud. Harald Seipp Sean Dague Alexander Saupp Bernd Freitag

Transcription

1 Dirk Vogelsang Senior Storage Architect May 2014 OpenStack & Cloud With contribution of: Harald Seipp Sean Dague Alexander Saupp Bernd Freitag 2014 IBM Corporation

2 Engage with the IBM EMEA Storage Competence Center Business Partners PartnerWorld 2 IBM Storageneers Intranet Internet Offerings and information especially for BPs! Events listing further categories will follow soon! ESCC storage blog about various storage topics ibm.biz/bpecampus escc.mainz.de.ibm.c om ibm.biz/storageneers

3 Agenda Cloud Introduction: public/private; IBM solutions & reference architectures, standardization IBM and OpenStack IBM involvement in general: Motivation, Contribution, IBM Value-add IBM Storage & OpenStack: Cinder support for Storwize, SVC, XIV, GPFS, DS8k, TPC, TSM Swift Object Storage Live Demo - OpenStack & XIV 3 Discussion / Q&As

4 Cloud Computing a common definition Cloud Computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, servers, storage, applications, and services) that can be rapidly provisioned!and released! with minimal management effort or service provider interaction.* *definition found via google on nist.gov, ibm.com, cisco.com, microsoft.com and ~ other pages 4

5 The continuous journey towards Cloud Computing 5

6 Cloud drivers behind an industry change... Focus staff on core business Transparent upgrade / maintenance services Overcome staffing limitations Desired: Flexibility, faster purchase & deploy, less administration It takes 10 minutes to order and 24 hours to deliver Speed! Price! Virtualization, Standardization -> Picklist Capacity / Performance tier? Backup / Replication? Snapspace? Pre-built VM appliances in standardized image format Price! 6 Consumption models - High importance to name top consumers even with central IT purchase HW & Support cost

7 Picklist Example: OpenStack Instance Launch 7

8 Cloud Storage Values 8

9 The Cloud strategy and mix that works for You Private Cloud & IT Benefits: Fully customizable Robust management Secure by design Dynamic Hybrid + Best of both worlds. Better outcomes. Public Cloud Benefits: Low entry cost Pay-per-use Highly elastic 9 Hybrid Cloud Maximize ROI on existing IT, while moving select workloads to public Meet seasonal capacity w/o undue investments in infrastructure Quickly extend the value of existing business process & services with SaaS

10 Cloud Taxonomy 10

11 Cloud Taxonomy SaaS Applications Data Runtime PaaS Middleware OS Application and logic provided by vendor, some programmability may be provided Runtime / Middleware programmatic you are responsible for application and data on top Virtualization IaaS Servers Storage Infrastructure is programmatic you are responsible for OS on up Networking 11

12 Cloud Taxonomy - OpenStack SaaS Applications Data Runtime PaaS Middleware OS Application and logic provided by vendor, some programmability may be provided Runtime / Middleware programmatic you are responsible for application and data on top Virtualization IaaS Servers Storage Infrastructure is programmatic you are responsible for OS on up Networking 12

13 IBM embraces & invests in Open Source to foster innovation Application Servers Service Oriented Architecture Cloud Computing Systems of Interaction Service Orientation June 1998: IBM enters into an engineering agreement with The Apache Group for development of the open-source Apache HTTP server software eventually becoming the leader of the new Application Server market September 1999: IBM capitalizes on an untapped market trend and begins participating in the community development of Linux with a $60M annual investment 13 Social Business Open Cloud Architecture November 2001: IBM rallies 150 influential vendors and the development community around a new tools environment with a $40 Million software donation disrupting the leadership of the software development ecosystem September 2012: IBM orchestrates the launch of The OpenStack Foundation boasting $10 million in funding and 5,600 members changing the dynamics of the Cloud ecosystem

14 The IBM approach to interoperability in the cloud Standards-based. Flexible. Customer driven. Cloud computing is changing the economics of IT, open cloud standards build the foundation of innovative, flexible and interoperable cloud solutions Business success is not theoretical. Practical cloud computing is grass roots: leveraging real world open source & standards implementations Collaboration across clients, providers, vendors & developers helps create a cross-industry view into use cases and best practices Standards allow enterprises to manage change across market evolution cycles extending the value of customer investments IBM very actively develops, leverages and supports open standards and technologies for cloud architectures and solutions. Only open, flexible and interoperable cloud solutions as promoted and provided by IBM unlock the full potential of a cloud implementation. 14

15 An open cloud architecture is emerging IBM Open Cloud API Economy External External Ecosystem Ecosystem analytics API API API API Marketplace API API Cloud Operating Environment 15 API API location collaboration API API data API API services Software Software as as a a Service Service API API Solutions App API API API & Integration Services Services & Composition Patterns datastore Software Defined Environment commerce mobile middleware Services security ops dev Traditional Workloads Platform as a Service Workload definition, Optimization, & Orchestration Resource Abstraction & Optimization Software Defined Compute Software Defined Storage Software Defined Networking Infrastructure as a Service

16 Open technologies to deliver a Software Defined Environment Value Automate everything to deliver a production environment: multi node applications, storage, network, change and configuration Delivers services via repeatable and controlled process, simplified self service consumption, auditable Capability Orchestration Services Platform services - Patterns of Expertise Infrastructure Services Based on OpenStack Integrated Workload OSLC TOSCA Software Defined Environment Workload Orchestration Workload Optimization, Workload definition, definition, Optimization, && Orchestration OrchestrationOptimization definition Resource Abstraction, Optimization & Security Software Defined Compute 16 Software Defined Storage Software Defined Networking

17 What is OpenStack? 17 OpenStack & Cloud Dirk Vogelsang

18 More than cloud infrastructure software a global open source community a collaboration among technology vendors 18 Successful platforms have three forces: Technology Ecosystem Users OpenStack & Cloud Dirk Vogelsang

19 OpenStack Open Source IAAS platform & Global Collaboration Mission: APR 2012 Exponential growth in 1+ YR 150 Contributors 2600 Individuals Create an ubiquitous open source cloud computing platform that is simple to implement and massively scalable Oct Contributors 12,000+ Individuals Scalable Massive scale Design Goals 1 Million physical machines, 60 Million VMs Billions of Objects stored Controlled by the OpenStack Foundation IBM is proud to be a Platinum Sponsor Open All code is Apache 2 licensed Simple Architecture is Modular Composed of multiple projects around the four capabilities Gold Sponsors Compute Network Storage Shared services Latest Releases: 19 Platinum Sponsors Grizzly Havana Icehouse Juno 04/ / / /2014

20 IBM is working to accelerate OpenStack Foundation success OASIS/W3C: LinkedData, OSLC TOSCA OpenStack Open Source Reference Cloud Standards Customer Council (CSCC) Cloud Computing Reference Architecture (CCRA) Because an open interoperable Cloud is critical for flexible cloud deployment and customer success 14 IBM has 14 core contributors 2 IBM is #2 in contributions to Icehouse & Havana releases 142 Active contributors from STG, SWG, LTC, Research IBM s ecosystem approach to an Open Cloud Architecture IBMers working on OpenStack from formation of the Foundation to Code Quality & New Function

21 OpenStack Components Nova Compute Cinder Block Storage Neutron Network Swift Object Store Glance Image repository Keystone Authentication Dashboard/ Horizon WebUI Ceilometer Reporting & Billing Heat Orchestration Trove DbaaS... Horizon Neutron Nova Glance Swift Cinder Keystone 21

22 OpenStack Nova Hypervisor support 22

23 Rackspace view on OpenStack (1/2) My summary: either you re in Compute provisioning (public or private deployment)... 23

24 Rackspace view on OpenStack (2/2) or you re (large capacity) Object Store 24

25 Interim summary Nested set of (optional) components, including alternative frameworks for similar purpose Mostly Virtual Machine centric use cases which then require network, storage,..or API driven object storage acess (OpenStack SWIFT) Likely to be service driven engagement incl. hardware not so much drop a box deals 100th++ of (similar!) virtual machines more than single ESX replacement *all of this are personal statements from the speaker 25

26 IBM SmartCloud Foundations & OpenStack - Supporting an evolutionary approach IBM SmartCloud products that make use of a packaged, integrated OpenStack approach Related Standards & Organizations Product rename: SmartCloud Entry IBM Cloud Manager with OpenStack SmartCloud Orchestration IBM Cloud Orchestrator SmartCloud Orchestration Orchestrate Services across multiple environments and domains SmartCloud Provisioning SmartCloud Provisioning Automate Optimized Workloads Automate Optimized Workloads SmartCloud Entry SmartCloud Entry Automate IT Delivery CIMI & OVF Key Common Cloud Stack TOSCA Factory Integrated Bundle Option Automate Optimized Workloads Automate IT Delivery Customer integrated hardware 26 OSLC CCRA PureFlex System PureApplication System

27 IBM Products with special support for OpenStack Globalization and localization enablement Localization for Simplified Chinese Crowd-sourced translation capability API, quotas, Nova integration Drivers for IBM SVC, XIV, PowerVM driver Dynamic hypervisor support HA enhancements DS8k, GPFS, TPC, TSM CDMI support and generic WSGI support Membership Services from HSLT IBM System p IBM System z 27 IBM SVC/Storwize IBM GPFS IBM XIV Legal support for drafting bylaws Improvements to stability and quality Community bug squashing days Permission building in the China market IBM TSM IBM DS8000 IBM TPC (4Q2014)

28 Why using IBM Storage for OpenStack (1/2)? XIV Very easy to install XIV itself as well as OpenStack driver XIV dense packaging well suited for storage-intense cloud Start small and grow with the Advanced System Placement Program (aka Frame Ahead) All storage is deployed automatically with complete automation Direct attached to VM node Supports iscsi + FC attachment SVC/Storwize Only Storage solution that supports fully transparent live storage migration in OpenStack Havana No interaction with the host required All advanced Storwize features are supported and exposed to the Cinder ecosystem Real-time compression, EasyTier Supports iscsi + FC attachment

29 Why using IBM Storage for OpenStack (2/2)? GPFS Single scale-out dataplane for the entire datacenter Volumes & Snapshots are files in the unified namespace Localizes & optimizes all volume operations VM instantiation is copy-on-write (COW) Volume Image conversion is COW Volume auto-formatting and -labeling Between GPFS pools or GPFS-nodes Supports all enterprise GPFS features 29 GPFS Better performance than NFS Transparent migration support GPFS File system features GPFS Glance integration Snapshots & Clones space-efficient and high-performance GPFS clone operations Supports per-volume replication levels Supports VM Live Migration OpenStack Controller and Compute Nodes GPFS NSD Servers / SAN GPFS: Enterprise Shared File System for storing Compute images, Volumes, Objects, Files FPO, de-clustered parity (GNR), policybased ILM/HSM, Tape support

30 Lessons learned: IBM Storage Cinder Drivers - Easy to Setup XIV Cinder driver is set up in minutes Download & start the Installer on the node running Cinder Enter XIV mgmt IP address & credentials Enter Storage Pool for OpenStack usage Create XIV Cinder Volume type with provided script Done! GPFS Cinder support is very easy to configure Ensure that GPFS is mounted across all nodes On the Cinder node, add the GPFS mount point and driver ID to the configuration Create GPFS Volume type Done! For both, Cinder is only required to run on a single node XIV will get directly attached also to non-cinder nodes through Nova + KVM GPFS will ensure that all Cinder operations will be node-local Note: On nodes that do not serve NSDs, only GPFS client is required When using GPFS, be careful with Neutron (networking) 30 Network re-configuration through Neutron may cause inconsistent GPFS state Use separate network for GPFS (maybe the best idea) or write scripts to shut down services, then unmount GPFS before Neutron changes

31 Why Objects? End User Developer Anywhere access Thin client Mobile integration Simplicity Offload object mgmt to storage, e.g., CIO/Business Low TCA and TCO Easy scaling from TBs to PBs Auto-delete Time-to-Value Sys Admin With a single namespace Flexibility in acquisition of software and hardware Time-to-value 31 no need to provision Atomicity of metadata and Data Built-in features such as versioning, geo-distribution (DR), etc. Users access without provisioning requests Automatically works around failures

32 Swift.openstack.org Object Storage (Swift) OpenStack Object Storage (Swift) is a massively scalable redundant storage system. Objects and files are written to multiple disk drives spread throughout servers in the data center, with the OpenStack software responsible for ensuring data replication and integrity across the cluster. - Application vendors can name OpenStack Swift API as supported storage configuration - Service Providers / application hoster can use Swift to provide Amazon S3 like services to their customers - All media files in Wikipedia are stored in OpenStack Swift 32

33 Background: OpenStack Swift An open source, highly available, distributed, eventually consistent object store Two tier architecture consisting of client facing proxies and storage servers Information protected through three-way replication (by default) Supports geo-distribution The dominant design for scale-out object stores Storage Servers (account, container and object) store, serve and manage data and metadata partitioned based upon ring Proxy Layer (public Swift was developed as pure software disconnected from hardware Clients send REST requests 33 face) authenticates and forwards to appropriate storage server(s) using ring Typically implemented on storage rich servers, e.g. IBM x3630 M4 Internet Internet or or Intranet Intranet Private Network

34 OpenStack Swift used by: an IBM Company e

35 ESCC OpenStack Live Demo Setup OpenStack Havana, RHEL 6.5 GPFS 3.5 Private Networks: N/A x3690-x HT 530GB RAM Control Nodes Compute Nodes nova neutron cinder glance keystone horizon ceilometer heat VM VM nova neutron VM VM x3650-m HT 46GB RAM VM VM VM VM XIV_02 GPFS VM Firewall HA Active/Passive Scale Out iscsi Public Networks: eth1: /24 vm: /24 35

36 Summary IBM Storage is Driving Innovation with OpenStack IBM is a leading contributor to OpenStack in general and to Cinder in particular Storage products that are OpenStack-ready: IBM XIV IBM Storwize family IBM DS8000 IBM Tivoli Storage Manager IBM Tivoli Productivity Center IBM General Parallel File System Industry-leading features accessible through OpenStack IBM Storage leadership and value for OpenStack Managing IBM Storwize V Storage with OpenStack OpenStack volume migration and IBM Storwize systems Roark Hilomen - IBM Edge

37 ESCC helps you keep storage clients satisfied Use Further information page to request access to the SmartCloud Community: 37