Taking Drupal development to the Cloud Karel Bemelmans
About me Working with Internet based services since 1996 Working with Drupal since 2011 Currently the devops guy @ Nascom
Case Study: Nascom Genk, Belgium Service Design Agency Drupal & Symfony2 dev About 12-16 devs
Fase 1: Local development
Local dev: Con s Hard to setup properly properly (http://localhost) Hard to work on a shared files Hard to show your work to someone else Not a good reflection of the production environment
New setup requirements 1. Perfect copy production 2. Isolated, no more shared solutions 3. But still easy to use
Fase 2: virtual project servers
Server Setup One virtual Linux server per project, just like production (LAMP + varnish, solr, memcached, ) Automated setup (puppet) Hosted on local xen cloud + ceph distributed filesystem LDAP accounts + sudo for everyone
Workspaces Apache (nginx) Virtual hosts with dedicated php-fpm pool DocumentRoot: /data/sites/$user http://$user.project.nascom.local Shared content: /data/shared (no chmod 777 -> setfacl) Mailcatcher on every server
Setup new workspace createworkspace.sh $user cd /data/sites/$user git clone symlink /data/shared/files to www/sites/default/files create env.settings.php Done!
Drupal settings settings.php: project-specific information generic site settings (Drupal hash, ) module includes (memcached) committed in git
<?php $update_free_access = FALSE; $drupal_hash_salt = 'RANDOMSTRING'; ini_set('session.gc_probability', 1); ini_set('session.gc_divisor', 100); ini_set('session.gc_maxlifetime', 200000); ini_set('session.cookie_lifetime', 2000000); $env_config = DRUPAL_ROOT. '/sites/default/env.settings.php'; if (file_exists($env_config)) { require_once($env_config); }
Drupal settings (2) env.settings.php: environment-specific information included at the end of settings.php database info memcached prefix $conf[ foo ] = bar ; not commited in git
<?php $databases = array ( 'default' => array ( 'default' => array ( 'database' => 'dbname', 'username' => 'dbuser', 'password' => 'secret', ), ), ); $conf[ memcache_key_prefix ] = karel ;
sshfs mount Local machine: mkdir $HOME/Projects/foo sshfs user@server:/data/sites/user $HOME/Projects/foo SublimeText opens project in $HOME/Projects/foo
Remote commands work on the server via ssh, not local (git, compass, grunt, ) you can work on the shared git (e.g. GitX), but slow! IDE: disable fs scan plugins! (git plugins) leet mode: develop with vim/emacs on the virtual server remote xdebug: possible with ssh tunnels
Virtual project servers: Pro s Identical to production setup Independent of the dev s client (Mac, Win, Linux, ) Once the server is running, adding extra workspaces is fast Everyone can access it, easy to look at other people s work Easy staging setup: http://staging.project.nascom.local
Virtual project servers: Con s Takes time to setup new project (puppet!) Grows fast on hardware needs (100GB disk, 2 CPU, 4GB RAM per server), we had about 30 projects running at one point Backup and monitoring is complex Local hosted on a Xen cloud with ceph filesystem: lots of hardware problems, high operational cost
Setup 3: shared hosting on an external hosted private cloud.
VMWare platform VMWare vsphere Essentials Plus with vcenter 3 physical machines with dual octocore CPU and 128GB RAM iscsi storage over 10gigE fiber Nascom office: 100/100 fiber connection
Plesk Parallels Plesk Panel 11.5 & 12 Intended for shared hosting setups Looks good Works even better: built on standard CentOS / Ubuntu distro (apache, nginx, mysql, php-fpm, )
Shared hosting setup Linux distribution (CentOS 6.5, Ubuntu 14.04 LTS) Local users (managed by puppet) Plesk Local tools: compass, grunt, xml tools, xhprof, New Relic,
Keeping it all private? Firewall per server: iptables via ferm (puppet) dev: closed except vpn acc, ci, prod: open port 80 +.htaccess OpenVPN: masquerade vpn ip push routers for 3rd party servers = easy access
Plesk Customers: Customers (e.g. Nascom) Subscriptions: Projects (e.g. Nascom-Corporate): http://nascom-corporate.dev01.nascom.be Subdomain: Workspaces http://user.nascom-corporate.dev01.nascom.be
Working on the shared server The same as our virtual project server: bitbucket git sshfs commands via ssh
Extra daemons Install it on the dev server e.g. memcached, solr via tomcat on port 8080, Install it on another virtual server e.g. varnish on 80, connects to dev on 7080 normal: http://user.project.dev01.example.org varnish: http://user.project.varnish-dev01.example.org
Extra tools mailcatcher install it on a separate server dev/acc postfix: relayhost $ip:1025 (or PHP sendmail_path) http://mailcatcher.nascom.be:1080
Acceptance, CI, Production Same server setup as dev (puppet!) Same Plesk setup, just no subdomains: Customer: Nascom Project: Nascom-Corporate Main workspace: nascom-corporate.acc01.nascom.be Plesk htaccess setup: not firewalled, but still a password
Support servers in our cloud backup server (cheap SATA storage): NFS + rsync monitoring & alerting: only for server + applications not hardware munin, nagios3, zabbix virtual varnish server (dev, acc, ci, prod) puppet master support applications: mulesoft, Bamboo buildserver,
External shared hosting: Pro s Fast to set up: 5mins initial setup, 1min workspace Easy to backup and maintain: Plesk New production setup? Make a new dev server. Exotic setup? Make a new dev server. Still a good reflection of our production environment
External shared hosting: Con s Shared server down? All workspaces down. Solution: shard your dev servers + use provisioning Expensive. But so are humans. Bandwidth and latency requirement: 100/100mbit for Nascom office
Conclusion After 9 months: 0 downtime. Everyone happy! Dev server is about 90% of the production environment, problems: varnish, exotic server installs, FTP hosting :( Works great with automated deployment (Bamboo) Next step: automation, testing,
Statistics Latency over VPN: 8-10ms roundtrip 2TB iscsi storage 200GB traffic/month Costs: 3500 euro/month
Questions?