Adventures & Challenges building an OpenStack public cloud Walter Heukels, Koert van der Veer en Pim van Riezen
The Sysadmin Experience Walter Heukels
About Me Walter Heukels Senior Engineer at CloudVPS Working on OpenStack Infra
About OpenStack Free sojware (Apache License) WriOen in Python Big project Very flexible
OpenStack AssumpSons Flexible, but it does have a philosophy CaOle servers, not pets Makes certain assumpsons We came up against some of those Don't go against the grain
OpenStack AssumpSons Flexible, but it does have a philosophy CaOle servers, not pets Makes certain assumpsons We came up against some of those Don't go against the grain Unless you really want to
OpenStack Structure Lots of sub- projects Nova SwiJ Quantum / Neutron Keystone... Distributed architecture
Our Cloud Object Store Since April 2013 Very Cool Compute Started free public beta ZFS Storage KVM VirtualisaSon
Lessons Learned: Technical People are mostly running private clouds at the moment Not much informason available on running a public cloud Examples MulSple external networks Keystone (authenscason) performance
Lessons Learned: ExpectaSons Customers make assumpsons Especially our customers Examples: IP spoof protecson ( my VPN router doesn't work ) HA for VM's..we're working on this!
Security groups?!? Bitcoins!! Lessons Learned: ExpectaSons Some customers don't know what to expect Will my Wordpress site scale automascally?
Distributed system Race condisons can occur Lessons Learned: Debugging Hard to find the logging you need Graph everything
The Road Ahead New features LBaaS VPNaaS Database as a Service PaaS
The Dev Experience Koert van der Veer
About Me Koert van der Veer Senior Developer at CloudVPS Working on OpenStack features
My Role Before OpenStack Development responsible for every detail of cloud management system Large CompeStors are moving incredibly quickly High pressure to add new features No Sme to fix technical debt
My Role With OpenStack Responsible for custom features only Bugs are usually fixed by others Large acsve community helps diagnosing problems ContribuSng is very sassfying and results in goodwill
Our Work on OpenStack Core features (contributed) ZFS block storage SwiJ features Bugfixes Deployment Billing Interfacing
Development Tools Python with geventlet, kombu, sqlalchemy, etc. DevStack Unit tests Tempest Grenade
Development Environment ProducSon close to git head Rapidly re- deployable testcluster pip instell e.
Development Work Flow PreparaSon Write code Review Merge Maintain
Step 1 - PreparaSon Launchpad blueprints Launchpad bugs IRC Mailinglist
Step 2 - Write Code Create feature branch Write code Write unit tests Run unit tests and stasc analysis Commit
Step 3 - Review Submit to Gerrit Jenkins tests Other reviews Core reviewer approves
Zuul reviews and audits code Jenkins merges code Step 4 & 5 Merge and Maintain
Bug detected (Jan 7th) Bug fixed (Jan 9th) Case Study 1: Bug in Cinder- Rootwrap SubmiOed to Gerrit (Jan 9th, Jan 10th) Approved (Jan 14th) Zuul rejected (Jan 15th) SubmiOed to Gerrit (Jan 16th) Approved (Jan 17th) Zuul accepted (Jan 19th)
Goal Challenges Case Study 2: Custom Bug in cinder- rootwrap Feature completeness Unit tests Progress Cinder Driver
High availability for VMs Per- port IP spoofing control Extra security msg queue Requests? Future Plans for ContribuSon
The Frontend Experience Pim van Riezen
About Me Pim van Riezen Senior developer at CloudVPS Working on OpenStack GUI
Interfacing Goal Goal: Make it easy to get started with a first VM
Interfacing Challenges Lots of dependencies: Create a keypair Create a private network Create a NAT router Create security groups and rules Create Server
Interfacing Challenges Decisions to make: Networking Key management..
Interfacing Challenges Security group abstracson: The double funcson as membership tag and access rule grouping confuses users It takes a lot of words to actually explain the concept
Interfacing Challenges API documentason: Hard to figure out what extensions are relevant Different parts of an openstack cloud may be out of sync A lot of perculiar choices made in v1 APIs ssll leak through in v2 Most command line tools also default to v1 APIs Lots of documentason lacks basic descripsons of parameters CombinaSon of tracing command line client, making wild guesses, luck
Interfacing SoluSons SoluSons: Comprehensive wizard Clear choices Image metadata Predefined security groups