Building a CloudStack UI for the Enterprise David Grizzanti, Bill Jones Sungard Availability Services 2014 Sungard AS. All rights reserved. CCCNA 14 - APRIL 10TH, 2014 - DENVER, CO
Overview! Who are we? David Grizzanti (Software Engineer @ Sungard AS) Bill Jones (Software Engineer @ Sungard AS)! Cloud Engineering @ Sungard Availability Services Current Cloud Offering & why CloudStack is the future! What s unique about our UI Features Technologies we re using! Overall Architecture! Integrating with CloudStack! Demo! Continuous Integration 2
About Sungard Availability Services Sungard Availability Services provides IT Operations support to IT departments that need to ensure business viability by keeping mission critical information and applications up and running. Sungard AS provides tailored enterprise cloud services, as well as innovative solutions including:! Disaster Recovery! Managed Hosting! Software! Consulting As of March 31 st, 2014 Sungard AS was split from Sungard and became a separate, independent company.
Cloud Engineering at Sungard AS Current Cloud Platform! Enterprise Cloud Services we operate a shared, multi-tenant infrastructure! We provide a fully managed Virtual Data Center environment for our customers! Our customers get cloud economics and agility without needing to re-architect their applications! We currently use traditional network isolation and security techniques! We have developed our own orchestration platform for our fully automated service provisioning Issues with this approach! Runs on complicated hardware! Difficult to perform upgrades, general lifecycle issues! Need for a self-service option 4
Cloud Engineering at Sungard AS Future/Choosing CloudStack! We are growing, so scaling is a concern! Customers want more flexibility and provide more customization! Improve reliability of provisioning automation! Provide more features to customers in an agile fashion! Need for a Public cloud as well as enterprise offerings What UI do we offer our customers? 5
UI: Necessary Features Self Service! Ability to enter credit card and spin up a VM within minutes Utility Billing! Customer shouldn t be blocked from adding more capacity Cost Control! Customers can see projected spend and manage budgets across their entire company Make it straightforward and easy to use! 6
UI: What sets us apart Real Time Notifications to the UI! Integrates with Event Bus/RabbitMQ Elevated Permissions on top of API! Allow more than just ROOT Admin to administer accounts Roles! Allow for multiple user Roles above what CloudStack provides Customer centric UI! Workplace concept to integrate CloudStack resources and billing 7
Overall Architecture Client/Browser Unified Cloud Portal CloudPlatform Hermes Bunny Ears SockJS Web Single Sign On Billing Integration API
Let s talk about the tech!! Ruby (Bunny Ears) Worker that s reading from RabbitMQ! Ruby on Rails Main web app (running under nginx + unicorn)! Node.js (Hermes) Notifies client/browser of changes CloudStack resource state! Backbone.js/Marionette.js + SockJS(WebSockets) Javascript frameworks used to drive UI changes in the browser Websockets keep connection open to Node.js for updating client s browser! Redis Storing information related to each async job User invites! Bootstrap Responsive front-end framework 9
Working with the CloudStack API Benefits! Fully featured API! Async Jobs Web Job Management! Storing request parameters + Job ID in Redis for retrieval and correlation Areas for improvement! Better permissions model (this is already in progress)! A Truly RESTful interface! Consistency around responses for all requests (async and non-async) 10
Example
Working with the Event Bus Benefits! Real time notifications (async jobs results)! No polling required Hermes Client/Browser Bunny Ears Node.js + SockJS (Websockets)! Worker process reads from RabbitMQ! Queries CloudStack/Redis! Sends events to Hermes -> Client s browser Areas for improvement! Multiple events types can sometimes be confusing! Data spread across more than 1 event! Incorrect information included in events 12
Overall Architecture Client/Browser Unified Cloud Portal CloudPlatform Hermes Bunny Ears SockJS Web Single Sign On Billing Integration API
Let s see a Demo!
VM Creation
VM Events
Network Configuration
Testing is Hard Try as hard as possible to TDD Tried to use VCR in the beginning The rise and fall of Smokestack! Built a replica of CloudStack for integration testing (well, almost)! Became too much work to maintain! Went with simulator (quicker on dev cycle) Test Coverage! Code Climate! Quality & security analysis for Ruby on Rails and Javascript 18
Continuous Integration with CloudStack Typical Flows! Run master after each push! Job in jenkins to build new projects for each new branch we push, auto-build on intial push or update! Both master and feature branches run full unit and integration tests Run full test suite after every code push! Challenge of trying to run integration tests on every push! Using CloudStack Simulator for feature tests Pool of CloudStack instances Vagrant/VirtualBox + Rails app to handle resetting each instance after use 19
Simulator Pool
Next Steps Move to Docker? Docker CloudStack Simulator! github.com/dgrizzanti/docker-cloudstack-simulator 21
Tools for sharing Built a fully featured ruby client for CloudStack (StackerBee)! Exception handling! Idiomatic Ruby formatting for names (snake_case vs mixedcase)! Handling of map parameters! Configurable API version! CloudStack REPL! Faraday HTTP lib (configurable middleware)! Exposes console access URL! Tests! github.com/promptworks/stacker_bee Vagrant CloudStack Simulator! github.com/promptworks/vagrant-cloudstack-simulator 22
Logging Tracing requests while testing! Need a reliable way to trace a request all phases of the application! UI -> CloudStack! CloudStack -> RabbitMQ -> BunnyEars -> Hermes -> Client 23
Credits/Acknowledgements 24
Questions? David Grizzanti @dgrizzanti david.grizzanti@sungardas.com Bill Jones bill.jones@sungardas.com