Building a big IaaS cloud with Apache CloudStack

Transcription

1 Building a big IaaS cloud with Apache CloudStack David Nalley PMC Member Apache CloudStack Member, Apache Software Foundation [email protected] New slides at:

2 #whoami Apache Software Foundation Member Apache CloudStack PMC Member Recovering Sysadmin Fedora Project Contributor Zenoss contributor Employed by Citrix in the Open Source Business Office

3 My questions for you.

4 Agenda Justification - 'so what' Overview of Apache CloudStack Break Design choices

5 Why use cloud? From a dev point of view the process looks like: Start new project File ticket for resources...wait...wait...wait Get resources, that aren't configured...wait... Get network access...get permission...wait Get things done.

6 Why use cloud? What IT Ops provides is not what developers want. Does not maximize value for the business

7 Get rid of the waiting! Remove constraints - developers empowered to get things done. Agility Enforce automated process instead of manual ones

8 Orchestration/Automation Sysadmins and network admins, still do so much manually. IaaS does not solve all the problems.

9 Overview

10 Overview CloudStack is an open source Infrastructure-as a-service (IaaS) orchestration platform that enables users to build, manage and deploy compute cloud environments.

11 Overview - GUI CloudStack offers an administrator's Web interface, used for provisioning and managing the cloud, as well as an end-user's Web interface, used for running VMs and managing VM templates.

12 Overview - API CloudStack Web Services Query HTTP API is loosely based on the REST architecture and allows developers to create new management solutions or integrate existing systems with CloudStack. It returns both XML and JSON response formats. Documented at:

13 Overview API - EC2 CloudStack also has a native but separate EC2 API Interface. Documented at:

14 Overview API Google Compute Engine A few CloudStack developers created a lightweight GCE API translation layer as well. Currently a separate project. Downloadable from:

15 VM Provisioning Select Operating System Windows, Linux Select Compute Offering CPU & RAM Select Data Disk Offering Volume Size Select Network Offering Network & Services Create VM

16 Dashboard VM Counts Public IPs Networks Latest Events

17 Virtual Machine Management Users VM Operations Console Access VM Status Change Service Offering Start Stop Restart Destroy CPU Utilized Network Read Network Writes 2 CPUs 1 GB RAM 20 GB 20 Mbps 4 CPUs 4 GB RAM 200 GB 100 Mbps

18 Volume Management Add / Delete Volumes VM 1 Volume Create Templates from Volumes Volume Template Schedule Snapshots Now Hourly Daily Weekly Monthly View Snapshot History. 12/2/ am 2/2/ am

19 Network Management Create Networks and attach VMs Acquire public IP address for NAT & load balancing Control traffic to VM using ingress and egress firewall rules Set up rules to load balance traffic between VMs Configure multi-tier networks

20

21 Management Server Secondary Storage MySQL Cloud_db Zone Zone Zone

22 Management Server Deployments User API Management Server Admin API Load Balancer Management Server MySQL DB Replication Back Up DB Infrastructure Resources

23

24 Understanding the Role of Storage and Templates Primary Storage Cluster level storage for VMs Connected directly to hosts NFS, iscsi, FC and Local Secondary Storage Zone level storage for template, ISOs and snapshots Host Host Primary Storage Cluster Pod NFS or Object Store via CloudStack System VM Templates and ISOs Imported into CloudStack Can be private or public Templat e Secondary Storage Zone

25 Provisioning Process 1. User Requests Instance 2. Provision Optional Network Services 3. Copy instance template from secondary storage to primary storage on appropriate cluster Host Host VM Primary Storage Cluster 4. Create any requested data volumes on primary storage for the cluster 5. Create instance 6. Start instance Templat e Secondary Storage Pod Zone

26 System VMs System VMs optimize and scale the datapath on behalf of CloudStack Stateless, can be destroyed and recreated from database state Highly Available Communicates with Management Server over management network Usually have 3 interfaces: control, guest and public

27 System VMs Virtual Router VM Provides multiple network services IPAM (DHCP), DNS, NAT, Source NAT, Firewall, PF, VPN User-data, Meta-data, SSH keys and password change server Redundancy via VRRP MS configures VR over SSH Proxied via the hypervisor on XS and KVM

28 System VMs Console Proxy VM Provides AJAX-style HTTP-only console viewer Grabs VNC output from hypervisor Scales out (more spawned) as load increases Java-based server Communicates with MS over message bus

29 System VMs Secondary Storage VM Provides image (template) management services Download from HTTP file share or Object Storage Copy between zones Scale out to handle multiple NFS mounts/object Stores Java-based server communicates with MS over message bus

30 Networking.is the bane of every cloud operators existence... Advanced Basic Everything else

31 Networking - Advanced VLANs for isolation All VLANs in a Pod trunked to hypervisors Each account has a dedicated virtual router More services (VPN, Firewall, LB, etc.)

32 Networking - Basic Simple, flat, Layer-2 network Bridge-based Layer-3 filtering/firewall Massively scalable

33 Networking Everything else GRE Tunnels VMware NSX (nee Nicira NVP) Midokura Midonet Stratosphere BigSwitch VXLAN (in a release this winter) Juniper Contrail (in a release this winter)

34

35 Installation Add yum/apt repo yum -y install cloudstack-management cloudstack-setup-databases cloudstack-setup-management Configure...

36 Design Choices

37 Self service UI API Some external tool

38 Self service UI API Some external tool

39

40 API or Command-line cloudmonkey> deploy virtualmachine serviceofferingid=d8611d07-acf5-4cd4-a630-5c4d937ef043 templateid=081358ff f8-adcc-1bb002fab361 zoneid=d06193b ad1-bd8-7b2f2eda63c3 curl ' command=listusers'

41 Config Management deployment

42 { "name": "hadoop_cluster_a", "description": "A small hadoop cluster with hbase", "version": "1.0", "environment": "production", "servers": [ { "name": "zookeeper-a, zookeeper-b, zookeeper-c", "description": "Zookeeper nodes", "template": "rhel-5.6-base", "service": "small", "port_rules": "2181", "run_list": "role[cluster_a], role[zookeeper_server]", "actions": [ { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] } ] }, { "name": "hadoop-master", "description": "Hadoop master node", "template": "rhel-5.6-base", "service": "large", "networks": "app-net, storage-net", "port_rules": "50070, 50030, 60010", "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]" }, { "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c", "description": "Hadoop worker nodes", "template": "rhel-5.6-base", "service": "medium", "port_rules": "50075, 50060, 60030", "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]", "actions": [ { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] }, { "http_request": " } ] } }

43 Use a tool

44 Usage Jevons Paradox Plenty of waste possible as well - will developers always destroy a machine when they are done with it? Important to show what projects and groups are consuming resources as well as how they are using those resources

45 Storage Commodity storage if you can get away with it. Local storage tends to be the best mix of cheap and performant No failover - do you need it? If so, use something enterprise-y.

46 Commodity Networking Layer 3 isolation - (aka Security Groups) VLANs - (not as commodity, but still relatively cheap on a small scale, but not at a large scale) Virtual routers (provide DHCP, DNS, LB, Firewall, PF, NAT, etc)

47 Commodity Hypervisor If your scale is below 100 hypervisors use what you know and if you don't know use KVM If you have more than 100 hypervisors you should be seriously evaluating XenServer there's a reason Amazon, Rackspace, and Google use Xen-based hypervisors. Use VMware if you already know it. Or have some demands. Easy to mix and match if necessary.

48 Limiting Resources Limit the number of VMs, snapshots, IP addresses, etc. Use 'projects' to share resources This means most folks will never have problems, but heaviest users will not be able to interrupt service for others.

49 Questions

50 Resources #cloudstack on irc.freenode.net