Challenges and lessons learned with Openstack deployments and MySQL. Sandro Mazzio+a Sr Director Product Management IaaS

Transcription

1 Challenges and lessons learned with Openstack deployments and MySQL Sandro Mazzio+a Sr Director Product Management IaaS

2 Who I am o o o o o I am not a DB expert or a DBA I work in Product Management since 15+ years I work at enovance since 2 months I started with Openstack in July 2011 I ve been launching HP Cloud Services DBaaS offering

3 enovance loca>ons and figures Montréal Bangalore Founded in San Francisco Paris People Customers Growth in Revenue

4 Journey to the cloud Software Value Engineering Software Architecture : from centralized apps to distributed services Software Engineering : from Waterfall to Agile Client / Server Vertical Scaling Hardware is key Centralized Services / API Horizontal Scaling Software is key Distributed Infrastructure IT Infrastructure : from Mainframe to Cloud Expensive Hw Vertical Scaling Monolithic Centralized Cheap Hw Horizontal Scaling Distributed Cloud Infrastructure Engineering : from Ops to DevOps

5 Market & opportunity Large Enterprises Software for Private Cloud: $ 10 bn by 2017 Telcos xsp Software for Public Cloud : $ 8 bn by 2017 Not Scalable Expensive Not Cloud Not Scalable Expensive Not Cloud Rich Set of Features No Versioning Shared Infrastructure Containers Not Scalable Not Cloud enovance Cloud Solu/ons Seamless Build & Delivery of OpenStack Cloud Infrastructures Telecom Equipment Vendors NFV : $ 2bn by 2017 Migrate from Physical Appliance To Network Functions Virtualization

6 What we do? Build a Cloud Deliver your Private or Public Cloud on Premise with a real Cloud Approach : No Versioning Pay as you Grow AWS Features Match Enterprise Grade SLA 24/7 Support Use a Cloud Deliver Cloud-Ready applications on top of existing Cloud infrastructures : Elastic & Scalable Continuous Delivery Pay as you Grow Enterprise Grade SLA 24/7 Support

7 Openstack & enovance q Gold Member q Top Ten Contributor q Share DNA Open Cloud, Open Source

8 Openstack is not simple

9 Openstack is not cloud (Subbu Allamaraju ebay, July 25, 2013) infra onboarding, boostrapping, remediaron, config management, patching, packaging, upgrades, high availability, monitoring, metrics, user support, capacity forecasrng and management, billing or chargeback, reclamaron, security, firewalls, DNS, integraron with other internal infrastructure and tools, and on and on and on. All key requirements for producron: By Enterprise for Private Cloud By Service Providers for Public Cloud

10 Openstack Upstream does not address HA Openstack HA Guide h+p://docs.openstack.org/high- availability- guide/content/ index.html ApplicaRon on top Openstack OS Infra. Plane: Openstack API Message Queues DB VirtualizaRon Layer

11 enovance Openstack Reference Architecture V1.0 Ø Management cluster full highly available Ø OpenStack API, schedulers Ø RabbitMQ, Database, Cache Ø Network services Ø Compute nodes with KVM and Open- vswitch with GRE tunnels as overlay Ø Swia cluster Ø Ceph cluster, used as backend for Nova, Glance and Cinder

12 enovance Openstack product Contains all core projects of Openstack Supports HA in a Single DC mode Based on edeploy (enovance Bare Metal Deployment Opensource) Puppet as Soaware ConfiguraRon Management

13 What about DB in our product? DB Requirements: FuncRonal & Performance requirements: Galera cluster for MySQL 100 % uprme Performance DegradaRon is ok Business Requirements: Opensource

14 enovance Galera for MySQL usage: results Experience from ProducRon Environment: 1 cluster of 3 Management Nodes No Service InterrupRon Performance DegradaRon are OK Galera performed correctly

15 On the topic of Mul> Data Center East ASIA Goal is to address: West 1 EMEA Disaster Recovery Local Affinity

16 Not all services are ready for Mul>- DC Openstack has a concept of AZ, Host AggregaRon = good in a single DC. Not good If your DC are separated by a WAN Nova Cells is good for distriburon (but nobody else is taken care) MulRple Region Keystone in every region or one overlay Using Heat to orchestrate MulRple Region Galera 3 Galera SynchronizaRon over WAN

17 Keystone Our experience working at large customers is that: Token storage is a nightmare and and do not scale well on large scale deployment Key Value Store (memcache, redis, ) - > MySQL - > memcache (but we have a bug) euh

18 Everything is MySQL, almost VS

19 Ceilometer, we use mongodb Because we need scalibility and because of the nature of informaron to store But Requires MongoDB DBA, Runbook, skills,

20 Openstack is not cloud (Subbu Allamaraju ebay, July 25, 2013) infra onboarding, boostrapping, remediaron, config management, patching, packaging, upgrades, high availability, monitoring, metrics, user support, capacity forecasrng and management, billing or chargeback, reclamaron, security, firewalls, DNS, integraron with other internal infrastructure and tools, and on and on and on. All key requirements for producron: By Enterprise for Private Cloud By Service Providers for Public Cloud

21 Why we need update upgrade patching? Billing CRM Portal seongs changes IntegraRon mess User new release bug fixes driver firmware update

22 What is our vision Deliver Openstack and app on top of it using Openstack process and tools ConRnuous IntegraRon/Delivery

23 Leverage the Openstack process How Openstack handle Contribu/on of 400 developers / month? Developers from 250 companies? 10+ integrated projects? and yet release a new version every 6 months with 400 new features without major inconsistencies?

24 and tools Everything in Process descrip/on and code Infrastructure used to handle the process Code to deploy the infrastructure 100 Rmes be+er than documentaron of a manual process that is never up to date... Reproducible environments Dynamic Provision test env when needed by Jenkins Allow dev to generate a test env for debug at will Store validated releases and customer specific env (in Openstack Swi^)

25 Benefits Improve Efficiency Nothing is local, everything is shared Control access rights Contribute model No need for a full release team Factorize hardware needs Reproducible Env Dynamically provision test environments when needed by Jenkins Allow developers to generate a test environment for debugging at will Store validated releases and customer specific environments (in Swi^)

26 What we are describing is Devops Validate code changes and mulrple deployments environments at once Accommodate more devs working on the same projects Limit wait Rmes Industrializes development and...

27 Conclusion Openstack is not easy. It is ok at small scale in a single DC mode. enovance has a product to address Single DC HA mode deployment is applying Openstack CD/CI to Openstack deployment itself is working on the MulR DC mode

28 Ques>ons? Sandro