Developing Plugins for Cloud Scale

Transcription

1 Developing Plugins for Cloud Scale

2 Who I am? 2

3 Who I am?

4 I am? Oscar Sanjuan Engineering Director Twitter: twitter.com/elasticbox Blog: elasticbox.com/blog

5 What does Elasticbox? ElasticBox makes it very easy for devops and IT ops to collaborate, develop, build and deploy any application on any cloud - private, public or hybrid at Scale. AUTOMATED APPLICATION DEPLOYMENT CONTINUOUS INTEGRATION CONTINUOUS DELIVERY SELF- SERVICE CATALOG

6 How do we use Jenkins today?

7 How do we use Jenkins today? CI/CD Environment Hybrid Multicloud Environment Dynamic and Fast changing environment.

8 Why an Elasticbox Plugin? 8

9 What means for us cloud Scale? We need slaves to be able to work with heterogeneous services in: Diferent regions Different public and private clouds Avoid The Explosion on the number of configurations Infrastructure changes should not imply changes in jenkins in cloud everything can be dynamic- 9

10 Jack of all trades, master of none. Our Goal the plugin is don t burn jenkins because of the infrastructure changes: It does not matter if you want slaves in one VPC or the other It can be GCE, AWS or any other. You can build slaves with cloud formation or slaves, it does not matter.

11 Design Challenges 11

12 Agenda / Design Challenges Let s Make Jenkins Infrastructure Independent. Cloud Scale Management (tags). Easy Setup and Slaves Definition. Scale Workloads Up and Down.

13 Challenge 1 Let s Make Jenkins Infrastructure Independent

14 Let s Make Jenkins Infrastructure Independent Changes in the infrastructure should not need to change jenkins, jenkins should be a pure consumer, we achieve this thanks to: ElasticBox becoming a Jenkins Cloud. Making Infrastructure configuration independent of jenkins (deployment Policies). 14

15 1. ElasticBox as a cloud in Jenkins. Need to distribute jobs across multiple clouds: ElasticBox already deploy boxes across multiple cloud. Asimilating ElaticBox to the jenkins concept of cloud, let jenkins deploy slapves in any cloud

16 1. EB as a cloud in Jenkins.

17 Challenge 2 Cloud Scale Management (tags).

18 2. Tags for scalable management and deployments. PETS vs CATTLE VS The traditional workload based on the pet approach will go away over time, to be replaced with workloads based on the cattle approach. 18

19 2. Tags for scalable management and deployments. This is usefull in both CI/ CD but also on one click deployments of clusters and multilayer applications. 19

20 2. Tags for scalable management and deployments. Managing LifeCycle Deploy Install Configure Start Stop Shutdown Terminate 20

21 2. Tags for scalable management and deployments. New Builds Steps 21

22 2. Tags for scalable management and deployments. Schedule instances 22

23 2. Tags for scalable management and deployments. Selecting instances 23

24 2. Tags for scalable management and deployments. Updating boxes as part of the LifeCycle 24

25 Challenge 3 Easy Setup and Slaves Definition.

26 3. Defining your slaves with boxes. Boxes are the templates that store application automation. Boxes contain scripts, variables, and metadata to automate processes when instantiated on cloud infrastructure. Stitched together, boxes model complex processes like deploying or upgrading multi-tier enterprise scale applications. 26

27 3. Defining your slaves with boxes. Boxes can also define and deploy a Jenkins server itself. (Check Jenkins EB Public Box) 27

28 3. Defining your slaves with boxes. 28

29 3. Defining your slaves with boxes. EXAMPLE: 29

30 3. Defining your slaves with boxes. 30

31 Challenge 4 Scale Workloads Up and Down.

32 4. Auto Scaling of Slaves. Scaling the Slaves is not just about scaling slaves but also about scaling down and releasing the unused resources. 32

33 4. Auto Scaling of Slave How it works? We calculate how many slaves we already have deployed for that cloud and recalculate the workload taking into account nodes that are still deploying or are deployed but not connected to Jenkins. We look for the slave configuration that match the criteria and we deploy as many as nodes are needed, taking into account the number of executors configured for them. This means that if the exceed workload is 3 and the slaves are configured with 1 executor, we deploy 3 nodes, but if they are configured with 3 executors, we only deploy 1 slave. Finally, to remove all nodes that have reached the maximum number of builds specified on the slave configuration section inside an ElasticBoxCloud. We also respect the minimum number of slaves that need to be available. 33

34 What we have earned / Conclusions Manking Jenkins independent from the Infrastructure Full Automation and Consistency! Easy and Fast Setup More control over up Sacaling and down scaling Replicability and Trazability Using jenkins at Cloud Scale J 34

35 How to get the EB Plugin? Signup for an Elasticbox account at: elasticbox.com Get the publically accesible plugin at:

36 Best way to try: Public boxes 36

37 Demo Come to our booth 37

38 Questions?? Oscar Sanjuan

39 Thanks to our Sponsors! 39

40 Developing Plugins for Cloud Scale