The CC1 system Solution for private cloud computing 1
Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2
What is CC1? The CC1 system is a complete solution to manage computing resources in the form of IaaS in any institution or company. It helps to organize and virtualize heterogeneous compute and storage resources. Good for small (100 cores) and large installations (>10000 cores). Final user can create and save virtual machines in easy way through the Web interface or EC2 compatible API. 3
Features For the end users: CC1 provide Web interface and EC2 programming interface which is easily accessible by Python scripts. Disk space for users is provided as a space for virtual machine images and virtual storage disks there is no user space like a cloud object storage. With CC1 system user gets management options for virtual machines (VM) and virtual farms, which are group of Vms forming a computing cluster with one head node. 4
Features For the CC1 administrator Dedicated administrator panel to manage users and resources. Administrator can organize scattered clusters unified with one CC1 cloud controller. Elastic way to extend/narrow number of physical nodes in the system. 5
Features: administrator module List of clusters added to CC1 cloud List of storages added to CC1 cloud Adding new storage Adding new cluster List of physical nodes for one of cluster 6
Features CC1 administrator can easily set and modify resources limits for particular users, also can set monthly credits to use resources. User can check usage of credits on chart. 7
Features User can monitor each VM 8
Technical details 9
Technical details 10
Technical details Software: Tested under latest stable Debian release. Uses virtualization package KVM (libvirt) and web application framework Django. New version is released every few months. Uniform Python environment Interface WWW (Django with Apache). CLM (Cloud Manager) and CM (Cluster Manager). Databases (PostgreSQL or MYSQL). Communication (more RESTful communication since version 2.0 instead of XML RPC). All CC1 services are recommended to be installed on separate virtual machines for easy backup and fast recovery. 11
Technical details Virtual Machines Libvirt is used as low level virtualization tool (not a bare metal solution). The KVM is default hypervisor. Libvirt supports more hypervisors, but were not tested. VM image is copied from physical storage to a local disk on the node and from this clone of image VM is started. It is recomended to keep the size of VM images small (10 GB for Linux). Additional space can be provided via virtual disks. Possibility to create preconfigured virtual computing clusters (Farms) with batch system. 12
Technical details Virtual storage disks User storage is provided via virtual disks that can be attached to VM at the start or to a running VM (USB and virtio drivers). Virtual disks are created by users on the physical storage through the network protocol. Disks are accessed directly from the physical storage. The disks are permanent i.e. survive after destruction of VM. 13
Technical details User's private networks Each user has its own network/networks. Range of private IP addresses may be booked by users from the total pool. Inter-VM communication is done by routing OSPF protocol (Quagga). Users virtual machines deployed on physical nodes 14
Technical details Network topology and administration CC1 Installation packages configure all necessary services required to start cluster's network. Network solution for VMs is based on small routed networks (not bridged). Each virtual network for VM: Is created using Libvirt. Is connected to physical node. Has a local Libvirt's dnsmasq (basic DNS and DHCP). All these networks are joined by routing protocols (OSPF). Private IP address is provided to VM by dnsmasq service (DHCP) managed by libvirt. Routing from the internal cluster network to the Internet is done on Routing server with proper quagga configuration. 15
Technical details Communication to VM from the outside VM Public IP can be dynamically attached to VM, then released and attached to another VM. Private to public IP mapping on the node where VM is running. VNC port mapping CC1 Cluster Manager forwards VNC ports to nodes, so needs to have its own public IP. 16
Use cases: by the Scientist Web interface allows to create of machine groups (so called farms). Scientist can process calculations on such pre configured computing cluster. For now, SLURM batch system is available. Easy to add other kinds of clusters. 17
Use cases: by HEP experiment HEP experiments like Belle II, LHCb, ATLAS, can use computing centers based on cloud computing paradigm. In particular, clouds with EC2 compatible interface (which is available in CC1 system). In the Belle II experiment, distributed environment is lead by the DIRAC system. DIRAC can handle jobs submitted to the grid and the cloud sites as well. 18
Use cases: by HEP experiment 19
Use cases: by HEP experiment 20
Use cases: by HEP experiment 21
Use cases: by HEP experiment 22
System requirements CC1 is a free Linux software build using modern virtualization and Internet technologies. CC1 System requirements: Full system can be installed on one powerful node. For bigger installations is recommended to split system to a few operational machines with the following parts: routing service, interfaces, cloud manager, cluster manager, databases. Router server to route packages between the Internet and cluster network (Quagga configuration) Hardware requirements: Physical nodes A few multi core nodes with hardware virtualization and Debian system installed Underlying host OS does not matter for end users Debian has good hardware support (including enterprise blade Systems) Disk storage Mounted to nodes as remote location (NFS by default, other distributed file systems also possible) For large installations disk array is recommended to have proper performance. Recommended network is 10 Gb/s. Optional: Mail server to sent activation request during user registration DNS to translate domain names for elastic (public) IP of VM. 23
System installation To install all CC1 components on the same machine Add CC1 repository to the operating system. Install deb packages: cc1-cm cc1-clm cc1-wi cc1-ec2 cc1-common Modify configuration files according to the installation instruction on http://cc1.ifj.edu.pl and restart services. Visit your machine address by Internet browser to see CC1 web portal. 24
Technical details CC1 installation at Institute of Nuclear Physics PAN in Krakow 1000 cores (Blades IBM HS22 Xeon L5640 @ 2.27 GHz) Disk storage 100 TB Network: 10 Gb/s external link 10 Gb/s backbone of internal network 25
How to get CC1? You can test the system ONLINE! Just send us request about access to CC1 Web and EC2 interfaces to cc1@cloud.ifj.edu.pl and we will provide a temporary access to computing resources in Krakow. CC1 project page is www.cc1.ifj.edu.pl There, in 'download' section you can find disk images with pre installed CC1 system. So you can test CC1 Web portal on your local machine. Do not hesitate to contact me! Rafal.Grzymkowski@ifj.edu.pl 26