Jenkins Cloud with Apache Mesos Klaus Azesberger Reinhard Kiesswetter Infonova GmbH
Agenda Our Jenkins Reasons to adopt our approach Pains of a static slave cloud Live Demo of setup and common ops use cases
Standing on the shoulders of giants Vinod Kone ebay PaaS team
Our Jenkins 300 Developers 35 + Projects ~ 2500 Jobs One Jenkins to rule them all
Features you need with centralized approach Jenkins Master High Availability Maintenance without downtime 100 % reliable pre-prod environment Rollbacks Isolation of builds (through reused VMs)
Should you adopt our approach? Costs of this setup Size (of your Jenkins) doesn't matter SysAdmin background helps
What we want A single, unified scalable HW resource pool (Mesos)......to run any kind and size of slaves s should be instantly available and lightweight (Docker)
Setup listing Jenkins Mesos Plugin Mesos Master Cluster (with Zookeeper) Mesos s Jenkins Docker Images CoreOS Monitoring
Problems with static approach Managing Instances Installs Updates / Versioning Backward compatibility Balancing between Labels Sizing Static slaves have a history
Problems with static approach Managing Instances Balancing between Labels Sizing Label 2 Label 3 Label 4 Label 1
Problems with static approach Managing Instances Build queue Balancing between Labels Sizing Label 2 Label 3 Label 4 Label 1
Problems with static approach Managing Instances Build queue Balancing between Labels Sizing Label 2 Label 3 Label 4 Label 1
Problems with static approach Managing Instances Balancing between Labels Sizing Waste of Resources VS. Long or even failed builds
Setup listing Jenkins Mesos Plugin Mesos Master Cluster (with Zookeeper) Mesos s Jenkins Docker Images CoreOS Monitoring
Some Mesos context Scheduler Launches tasks Receives resource offers Executor Executes tasks Launched by the Mesos slave Framework
Mesos Jenkins master Standby master Mesos master Standby master Mesos slave Mesos slave Mesos slave Docker executor Docker executor Docker executor Jenkins slave Jenkins slave Jenkins slave
Mesos Jenkins Plugin explained Jenkins Mesos Plugin Scheduler cloud Jenkins Master Job Queue Jenkins (Task) Mesos master Job Mesos slave Docker executor 18
M S Private Docker Registry 19
Sharing is caring http://github.com/kazesberger/jenkins-mesos-setup https://github.com/kazesberger/jenkins-docker-slaves 20
Ansible in short 21 Automate setup Automate maintenance Execute commands on a set of hosts yml + python + ssh No agents
Demo: Use Cases Scale out HW Update SW (firefox) One can also provide legacy support Change Sizing of slaves One can also provide a set of different sizingvariants
Migration without new server Build queue Label 1 Label 2 Label 3 Label 4
Jenkins slaves Mesos slaves Label 1 Label 2 Label 3 Label 4
Load during business hours Label 1 Label 2 Label 3 Label 4
Load during the night Label 1 Label 2 Label 3 Label 4
Lessons learned When configuring systemd services keep in mind that the coreos node could be autoupdated at any time When building docker containers for your jenkins slaves respect settings like Timezone and character encoding You should do a soft migration to the mesos cloud with projects that can provide some meaningful feedback in case of problems
Klaus Azesberger Technology Architect INFONOVA GmbH A BearingPoint Company Seering 6 A-8141 Unterpremstätten/Graz Austria Tel: +43 316 8003 2106 Fax: +43 316 8003 1080 Mobile: +43 664 81 61 963 www.infonova.com klaus.azesberger@infonova.com Reinhard Kiesswetter Senior System Engineer INFONOVA GmbH A BearingPoint Company 32 Seering 6 A-8141 Unterpremstätten/Graz Austria Tel: +43 316 8003 1173 Fax: +43 316 8003 1080 Mobile: +43 664 81 61 886 www.infonova.com reinhard.kiesswetter@infonova.com
Thank You Sponsors