Automation & Open Source How to tame the Cloud?
Who are we? François Deppierraz Mathieu Bornoz System and Network engineer System and Software engineer Reluctant to type the same command twice Automation obsessed Provides consulting services Head of the Infrastructure Solutions Department
Agenda Context Why automation? Why did we build our own private cloud? Use cases Puppet Modules: continuous integration Deploy & Manage Virtual Infrastructure (demo) Conclusion 3
Context Geospatial & Business solutions development Need to provide standard solutions platform for development, QA, prod, CI (TDD, BDD) RedHat/Debian/Ubuntu ready Flexible hosting (Cloud, virtual, bare-metal) Scalable and evolutive platform Deployed/managed by us or by customers 4
Hybrid Platform
Why automation? Share working knowledge as code Repeatability Reuse components Decreased manual configuration DevOps Philosophy Change management Ability to record changes in a VCS 6
Why build a private cloud? Benefits Keep control on the full stack Data locality and sovereignty Useful for internal infrastructure/services Customer choice (privacy vs. capacity/elasticity) Leverage existing infrastructure (Data Center, Network, Hardware) -> cost effectiveness Help keep internal developments provider agnostic 7
Camptocamp s private cloud OpenStack Icehouse (started with Havana) Deployment automated with Foreman and Puppet 30+ physical servers Ceph for volumes and objects storage Upgrades are somewhat tricky... 8
Use Case #1: Puppet Modules Dev & QA
Puppet Modules: Dev & QA Improved development workflow Source code hosted on Github At each commit, Travis CI triggers Unit tests Acceptance tests (server spec) on our private cloud For each new tag commited Upload the release to the Puppet Forge 11
Puppet modules
14
1 66 86 Using worker: worker-linux-docker-1872a3f1.prod.travis-ci.org:travis-linux-5 $ git clone --depth=50 --branch=master git://github.com/camptocamp/puppet-openldap $ export BEAKER_set="debian-8-x86_64-openstack" 218 219 228 Provisioning OpenStack Provisioning debian-8-x64 (...) OpenStack host debian-8-x64 (...) assigned ip: 128.179.66.57 580 889... (debian-8-x64) 19:27:51$ apt-get install -y puppet... (debian-8-x64) 19:30:04$ puppet apply --strict_variables... 1148 Port "389" should be listening 1178 Port "636" should be listening 1251 Finished in 4 minutes 1.1 seconds (files took 2 minutes 4.9 seconds to load) 1252 11 examples, 0 failures
Unit / Acceptance Tests Benefits Test / Behavior Driven Development Pluggable inside your Continuous Delivery Pipeline Non-regression secured by tests in change requests Simplifies multi-platform validation More confidence as a result 17
Use Case #2: Deploy & Manage Virtual Infrastructure Terraform demo
Terraform Infrastructure as Code One layer above typical CM tools (Puppet, Chef, ) Manages servers, networks, DNS records,... Simple file based configuration JSON-like language Idempotency Missing feature from other tools (CloudFormation, OpenStack Heat, ) http://terraform.io/ 19
Provisioning from scratch 21
Live Demo
diff(1) your infrastructure 24
diff(1) your infrastructure 25
Apply changes to the infrastructure 26
Conclusion Taming the cloud requires... appropriate tooling (automation) embracing DevOps principles agility and willingness to learn The FOSS ecosystem provides software for Bootstrapping and managing your own private cloud Automate the deployment of all your infrastructure All that with the great support of FOSS communities! 28
Thank you for your attention! Any questions?