Containers with Google: from Borg to Kubernetes

Transcription

1 Containers with Google: from Borg to Kubernetes Jens Bussmann Cloud Platform Lead, NACE Google Filip Grzadkowski Warsaw Kubernetes Team Lead Google

2 Building what s next 3 We set out to build a better cloud for us. Now, we re giving it to you. 3

3 Building what s next 4 4

4 Google Innovations in Software Kubernetes FlumeJava Borg Spanner Colossus Bigtable Dremel 2015 MapReduce 2012 GFS

5 Google Innovations in Software Kubernetes FlumeJava Borg Spanner Colossus Bigtable Dremel 2015 MapReduce 2012 GFS

6 Google Innovations in Software Kubernetes FlumeJava Borg Spanner Colossus Bigtable Dremel 2015 MapReduce 2012 GFS

7 Google Innovations in Software Kubernetes FlumeJava Borg Spanner Colossus Bigtable Dremel 2015 MapReduce 2012 GFS

8 Containers At Google, everything runs in a container

9 This is vanilla virtualization Guest environment app code libraries guest kernel Hypervisor 10

10 It has downsides: Not portable & opaque Guest environment app code libraries machine image locked into a platform guest kernel Hypervisor 11

11 It has downsides: No Isolation Guest environment app code app code??? dependency libraries guest kernel Hypervisor 12

12 It has downsides: Little Reuse redundant Guest environment Guest environment Guest environment app code app code app code libraries libraries libraries guest kernel guest kernel guest kernel Hypervisor 13

13 Containers create a better abstraction layer Guest environment app code cut here libraries guest kernel Hypervisor 14

14 Containers at Google Google is using container technology for more than 10 years Everything at Google runs in a container 15

15 Containers at Google Google is using container technology for more than 10 years Everything at Google runs in a container We launch over 2 billion new containers per week 16

16 Containers Google and containers Google and container technology Released CGroups Limited isolation 2004 Kubernetes Released LMCTFY Confidential & Proprietary 17

17 Containers Google and containers Google and container technology Released CGroups Limited isolation Kubernetes Released LMCTFY Borg years of experience Confidential & Proprietary 18

18 Kubernetes Container orchestrator Builds on containers (Docker, Rocket) Handles container and node failure Multiple cloud and bare-metal environments 100% Open Source, written in Go Let users manage applications, not machines 19

19 Open Source 5k Commits in % Unique Contributors Top 0.01% of all Github Projects External Projects Based on K8s Companies Contributing Companies Using 20

20 Kubernetes Components Confidential & Proprietary 21

21 Pods A group of one or more containers sharing fate Containers in a pod run in a shared context (tcp/ip, sockets) Can communicate via localhost Considered to be relatively ephemeral (rather than durable) entities Pod Container 1 Container 2 22

22 Pod example apiversion: v1 kind: Pod metadata: name: wordpress spec: containers: - image: wordpress name: wordpress env: - name: WORDPRESS_USER value: wp_user Public image from Docker Hub. Private images have the form gcr.io/<your-project>/<your-image> 23

23 Nodes Pod 1 A worker machine in Kubernetes that will host one or many pods May be a VM or physical machine, depending on the cluster Restarted automatically after failure on GCE Pod 2 Node 1 24

24 Services Pod 1 Your load balancers, so to speak A stable endpoint that hides the implementation behind (pods or external) All pods will see your services as env variables or DNS entries Different Types: ClusterIP NodePort LoadBalancer Pod 2 Node 1 wordpress-service :80 Pod 3 Pod 4 Node 2 25

25 Labels type=wordpress Pod 1 Any combination of key-value pairs Pod 2 data-center=east release=production version=1.2 tier=front-end Can be applied to pods, services, nodes Node 1 wordpress-service :80 Pod 3 Pod 4 Node 2 26

26 Service example apiversion: v1 kind: Service metadata: name: wpfrontend spec: type: LoadBalancer ports: - port: 80 targetport: 80 protocol: TCP selector: name: wordpress Type of service: ClusterIP, NodePort, LoadBalancer Label to select the pods that will receive traffic 27

27 Replica Sets (since 1.2) Pod 1 Ensures a specified number of pod replicas are running with the specified labels Will react if a pod dies unexpectedly Can autoscale Pod 2 Node 1 wordpress-rc replicas: 4 Pod 3 Pod 4 Node 2 28

28 Replica Sets (1.2) vs Replication Controller (1.1) Replica Sets will replace Replication Controller The only difference is more flexible label selector support Not to be used directly. Use Deployment instead # equality-based environment = production tier!= frontend # set-based environment in (production, qa) tier notin (frontend, backend) partition!partition 29

29 Deployments (since 1.2) Provides declarative updates for Pods and Replica Sets Describe the desired state in a Deployment object, and the Deployment controller will get there at a controlled rate Pause, Resume, Rollback deployments Deployments keep a history 30

30 Deployment example apiversion: extensions/v1beta1 kind: Deployment spec: replicas: 3 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerport: 80 if too few, start new replicas if too many, kill some 31

31 Deployments $ kubectl create -f docs/user-guide/nginx-deployment.yaml deployment "nginx-deployment" created $ kubectl get deployments NAME DESIRED nginx-deployment 3 CURRENT 0 UP-TO-DATE 0 AVAILABLE 0 AGE 1s $ kubectl get deployments NAME DESIRED nginx-deployment 3 CURRENT 3 UP-TO-DATE 3 AVAILABLE 3 AGE 18s 32

32 Update a Deployment apiversion: extensions/v1beta1 kind: Deployment spec: replicas: 3 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerport: 80 apiversion: extensions/v1beta1 kind: Deployment spec: replicas: 3 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.9.1 ports: - containerport: 80 33

33 Update a Deployment $ kubectl apply -f docs/user-guide/new-nginx-deployment.yaml deployment "nginx-deployment" configured $ kubectl get deployments NAME DESIRED nginx-deployment 3 CURRENT 3 UP-TO-DATE 0 AVAILABLE 3 AGE 20s $ kubectl get deployments NAME DESIRED nginx-deployment 3 CURRENT 3 UP-TO-DATE 3 AVAILABLE 3 AGE 36s 34

34 Resources: Requests / Limit Set up required and maximum resources for the pod The pod can only be scheduled in a node with enough capacity for all its containers spec: containers: - name: db image: mysql resources: requests: memory: "64Mi" cpu: "250m" limits: memory: "128Mi" cpu: "500m" If a pod requests cpu: 0 it can always be scheduled 35

35 Ranking potential nodes Prefer node with most free resource left after the pod is deployed Minimise number of Pods from the same service on the same node CPU and memory are balanced after the Pod is deployed Node Affinity (since 1.2): use label filters to alter node selection 36

36 Other components Persistent Volume: attach a persistent disk to a pod. Secrets: inject sensitive information into your pods, reducing the risk of accidental exposure Namespaces: scope for object names. Names need to be unique within a namespace. Daemon Set: All nodes run a replica of a pod Jobs: creates pods and ensures that a specified number of them successfully terminate. 37

37 Host it in the cloud or on premise Confidential & Proprietary 38

38 Google Container Engine The easiest way of getting started with Kubernetes Managed by Google Cloud Logging and Monitoring integrated out-of-the-box Once connected, it s just Kubernetes # Create cluster gcloud container clusters create hello-world --num-nodes 3 # Connect Kubernetes gcloud container clusters get-credentials hello-world # Use Kubernetes kubectl get pods 39

39 (Coming soon) A collaboration between Google and Red Hat Deploy your Docker containers in a IaaS platform managed by Red Hat Migrate your application seamlessly between Google Cloud and your own premises Portable: move your application to your own Data Center using Open Shift Online or Open Shift Enterprise. Confidential & Proprietary 40

40 Hosting Kubernetes Hosting for Kubernetes in the hybrid cloud Online, Enterprise or Dedicated Managed by Red Hat or hosted yourself Hosting for Kubernetes in Managed by Google Scale with a click Confidential & Proprietary 41

41 Build What s Next