Hsted Private Clud Open surce clud cmputing with penqrm by Rene Buest INSIGHTS
Abstract Cmpanies have recgnized the benefits f the flexibility f their IT infrastructure. Hwever, the recent past has reinfrced the cncern t avid the path t the public clud fr reasns f data prtectin and infrmatin security. Therefre alternatives need t be evaluated. With a private clud ne is fund, if this wuld nt end in high up-frnt investments in wn hardware and sftware. The middle way is t use a hsted private clud. This type f clud is already ffered by sme prviders. Hwever, there is als the pssibility t build it up and run themselves. This INSIGHTS reprt shws hw this is pssible with the pen surce clud cmputing infrastructure slutin penqrm.
Why a Hsted Private Clud? Cmpanies are encuraged t create mre flexible IT infrastructure t scale their resurce requirements depending n the situatin. Ideally, the use f a public clud is meeting these requirements. Fr this n upfrnt investments in wn hardware and sftware are necessary. Many cmpanies dread the way int public clud fr reasns f data prtectin and infrmatin security, and lk arund fr an alternative. This is called private clud. The main advantage f a private clud is t prduce a flexible self-service prvisining f resurces fr staff and prjects, such as in a public clud, which is nt pssible by a pure virtualizatin f the data center infrastructure. Hwever, it shuld be nted that investments in the IT infrastructure must be made t ensure the virtual resurce requirements by a physical fundatin fr building a private clud. Therefre, an apprpriate balance needs t be fund that allws a flexible resurce btaining fr a self-service, but at the same time must nt expect any high investment in the wn infrastructure cmpnents and withut t waive a self-determined data prtectin and security level. This balance exists in hsting a private clud at an external (web) hster. The necessary physical servers are rented n a hster wh is respnsible fr their maintenance. In rder t secure any physical resurce requirements, apprpriate arrangements shuld be made with the hster t use the hardware in time. Alternatives include standby server r similar appraches. On this external server-/strage-infrastructure the clud infrastructure sftware is then installed and cnfigured as a virtual hsted private clud. Fr example, accrding t their needs this allws emplyees t start wn servers fr sftware develpment and freeze and remve them after the prject again. Fr the billing f the used resurces, the clud infrastructure sftware is respnsible, which prvides such functins. penqrm Clud Basically, an penqrm Clud can be used fr the cnstructin f a public and private clud. This cmpletely based n penqrm s appliance mdel and ffers fully autmated deplyments that can be requested by clud users. Fr this penqrm Clud supprts all the virtualizatin and strage technlgies, which are als supprted by penqrm itself. It is als pssible t prvide physical systems ver the penqrm Clud. Based n the penqrm Enterprise Clud Znes, a fully distributed penqrm Clud infrastructure can als be build. Thus, several separate data centers may be divided int lgical areas r the cmpany tplgy can be hierarchically and lgically cnstructed safely separated. Mrever penqrm Enterprise Clud Znes integrates a central clud and multilingual prtal including a Ggle Maps integratin, s an interactive verview f all sites and systems is created.
Structure f the reference envirnment Fr the cnstructin f ur reference setup a physical server and multiple public IP addresses are required. There are tw ptins fr installing penqrm: Recmmended: Cnfiguratin f a private class C subnet (192.168.xx/255.255.255.0) in which penqrm is perated. penqrm required an additinal public IP address fr access frm the utside. Optin: Install penqrm in a virtual machine. In this variant penqrm cntrls the physical server and receives the virtual machines frm the physical hst fr subsequent peratins f the clud. Fr the assignment f public IP addresses clud NAT can be used in bth scenaris. This penqrm Clud functin will translate the IP addresses f the private penqrm Class C netwrk int public addresses. This requires pre-and pstruting rules n the gateway / ruter using iptables, cnfigured as fllws: iptables -t nat -A POSTROUTING -s 192.168.0.0/24 - br0 -j MASQUERADE iptables -t nat -A POSTROUTING -s 192.168.0.0/24 - eth0 -j MASQUERADE Mre infrmatin n pre-and pst-ruting with iptables can be fund at http://www.karlrupp.net/en/cmputer/nat_tutrial Fr the cnfiguratin f cmplex netwrk envirnments, the IP management plugin is recmmended. This enterprise plugin allws t set any netwrk- and IP address cnfiguratins fr the managed servers. In the penqrm Clud, it als prvides a mapping f netwrks t clud users and grups and als supprts the autmated VLAN management. In additin, tw bridges are needed: One f the public interface with a public IP address. One fr the private interface dpe fr which DHCP is cnfigured. The data in the clud are later stred in the lcal strage f the physical server. Fr this purpse, there are tw variants: Recmmended: KVM-Strage LVM Deplyment (LVM lgical vlume deplyment) Requires ne r mre dedicated LVM vlume grup (s) fr the virtual machines. Fr mre cmplex setups a central iscsi target r a SAN is recmmended. Optin: KVM-Strage BF Deplyment (blckfile deplyment) Create a directry n the Linux server as /var/lib/kvm-strage/strage1 /var/lib/kvm-strage/strage2 (The strage directries can be set arbitrarily n the plugin cnfiguratin.) Fr mre cmplex setups, a central NAS fr the cnfigured munt pints shuld be used. At the end iptables must be cnfigured accrding t the rules abve and the desired wn safety. After that the installatin f penqrm fllws. Packages fr ppular Linux distributins are available at http://packages.penqrm.cm. After penqrm has been installed and initialized the cnfiguratin fllws.
Basic cnfiguratin f penqrm The first step after initializatin is editing the /usr/share/penqrm/plugins/dns/etc/penqrm-plugin-dns.cnf, by changing the default value t the wn dmain. Cnfigure dmain fr the private netwrk # please cnfigure yur dmain name fr the penqrm netwrk here! OPENQRM_SERVER_DOMAIN="qnet.rg" After that we activate and start the plug-ins via the web interface f the penqrm server. The fllwing plugins are abslutely necessary fr this: DNS Plugin Used fr the autmated management f the DNS service fr the penqrm management netwrk. DHCPD Autmatically manages the IP addresses fr the penqrm management netwrk. KVM Strage Integrates the KVM virtualizatin technlgy fr the lcal deplyment. Clud-Plugin Allws the cnstructin f a private and public clud cmputing envirnment with penqrm.
Further additinal plugins are recmmended: Cllectd A mnitring system including lng-term statistics and graphics. LCMC Integrates the Linux Cluster Management Cnsle t manage the high availability f services. High-Availability Enables autmatic high availability f appliances. I-d-it (Enterprise Plugin) Prvides an autmated dcumentatin system (CMDB). Lcal server Integrates existing and lcally installed server with penqrm. Nagis 3 Autmatically mnitrs systems and services. NVNC Puppet Prvides a remte web cnsle fr accessing virtual machines and physical systems. Integrates Puppet fr a fully autmated cnfiguratin management and applicatin deplyment in penqrm. SSHterm Allws secure lgin via a web shell t the penqrm server and integrates resurce Plugins which ffer mre cmfrt in the autmatic installatin f virtual machines as clud templates are: Cbbler Integrates cbbler fr autmated deplying f Linux system in penqrm. FAI Integrates FAI fr the autmated prvisining f Linux systems in penqrm. LinuxCOE Integrates LinuxCOE fr the autmated prvisining f Linux systems in penqrm. Opsi Integrates Opsi fr the autmated prvisining f Windws systems in penqrm. Clnezilla/lcal-strage Integrates Clnezilla fr the autmated prvisining f Linux and Windws systems in penqrm.
Basic cnfiguratin f the hst functin fr the virtual machines Case 1: penqrm is installed directly n the physical system Next, the hst must be cnfigured t prvide the virtual machines. Fr that an appliance type KVM Strage Hst is created. This wrks as fllws: Create appliance Base > Appliance > Create Name: e.g. penqrm Select the penqrm server itself as resurce Type: KVM Strage Hst This gives penqrm the infrmatin that a KVM strage is t be created n this machine. Case 2: penqrm is installed in a virtual machine running n the physical system Using the "lcal server" plugin the physical system is integrated int penqrm. T this the "penqrm-lcal-server" integratin tl is cpied frm the penqrm server n the system t be integrated, e.g. scp /usr/share/penqrm/plugins/lcal-server/bin/penqrm-lcal-server [ip-address f the physical system]:/tmp/ After that, it is executed n the system t be integrated: ssh [ip-address f the physical system]: /tmp/penqrm-lcal-server integrate -u penqrm -p penqrm -q [ip-address f the penqrm server] -i br0 [-s http/https] (In this example "br0" is the bridge t the penqrm management netwrk.) The integratin via "lcal server" creates in penqrm autmatically: a new resurce a new image a new kernel a new appliance frm the sub-cmpnents abve Next, the appliance f the currently integrated physical system must be cnfigured t prvide the virtual machines. Fr this the appliance is set as type KVM Strage Hst. That wrks as fllws: Edit the appliance Base > Appliance > Edit Type: Set KVM Strage Hst This gives penqrm the infrmatin that a KVM strage is t be created n this machine.
Basic cnfiguratin f the strage functin Nw, the basic cnfiguratin f the strage fllws. Fr this purpse, a strage bject f a desired type is created. This wrks like this: Create strage Base > Cmpnents > Strage > Create Case 1, select the resurce f the penqrm server Case 2, select the resurce f the integrated physical system Name: e.g. KVMStrage001 Select deplyment type This depends n the selected type at the beginning: KVM-Strage LVM deplyment r directry (KVM- Strage BF deplyment)
Preparatin f virtual machine images In rder t prvide virtual machine (VM) later ver the clud prtal as part f finished prducts, an image fr a VM must first be prepared. This wrks as fllws: Creating a new virtual machine with a new virtual disk and install an ISO image n it. Plugins > Deplyment > LinuxCOE > Create Templates The created images are autmatically stred in an ISO pl which each virtual machine within penqrm can access. Subsequently a base fr the master template is created. This serves as a basis t prvide users a prduct ver the rder prcess. Create a new appliance Base > Appliance > Create Create a new resurce KVM-Strage virtual machine Create a new VM Make settings Select an ISO image Create Select created resurce Create a new image Add image as KVM-Strage vlume Select KVM-Strage Select vlume grup n KVM-Strage Add a new lgical vlume Select an image fr the appliance Edit t set a passwrd The previusly chsen passwrd f the ISO is verridden. Select kernel Frm the lcal disk (LAN bt is als pssible) Start appliance The autmatic installatin can nw be tracked ver VNC. Further adaptatins can be dne itself. Please cnsider Misc > Lcal-Server > Help >Lcal VMs Lcal-Server fr lcal virtual machines
Cleaning up The created appliance can nw be stpped and deleted afterwards. The imprtant pint was t create an image that can be used as a master template fr the clud. The created image using the appliance includes the basic perating system which was created frm the ISO image. Cnfiguratin f the penqrm Clud We have nw finished all preparatins t start cnfiguring the penqrm clud. We find the necessary settings at Plugin > Clud > Cnfiguratin > Main Cnfig. All parameters which are adapted here have a direct impact n the behavir f the whle clud. Basically an penqrm Clud can be run with basic settings. Depending n the needs and the wn specific situatin, adaptatins can be make. The area descriptin in the right clumn f the table are helpful.
Hwever, there are parameter which are need t cnsider regardless f the wn use case. These are: Autmatic prvisining (aut_prvisin) Determines if systems are autmatically prvisined by the clud r if an apprval f a system administratr is needed. Prvisining f physical systems (request_physical_systems) This parameter defines if besides virtual machines even physical hsts can be prvisined by the clud. Clning f images (default_clne_n_deply) By default the clud rlls ut cpies (clnes) f an image. High-availability (shw_ha_checkbx) Enables t perate the penqrm clud including the high-availability f the prvided resurces. Billing f the used resurces (clud_billing_enabled) penqrm has an extensive billing system t determine wn prices fr all resurces t get a transparent verview f the running csts. Clud prduct manager (clud_selectr) Enables the prduct manager t prvide users varius resurces ver the clud prtal. Currency fr the settlement f resurces (clud_currency) Determines the lcal currency with which the resurces are t be settled. Exchange rati fr resurces in real currency (clud_1000_ccus) Determines hw many 1000 CCUS (Clud Cmputing Units) crrespnd t a previusly fixed real currency. Resurce allcatin fr grups (resurce_pling) Determines frm which hst an appinted user grup receive their virtual machines.
Creating prducts fr the penqrm Clud T prvide ur users the resurces ver the clud prtal we have t create prducts first which define the cnfiguratin f a virtual machine. The settings fr that we find at Plugin > Clud > Cnfiguratin > Prducts. The Clud prduct management is used t create varius prducts which users can chse later t build wn virtual machines itself ver the clud prtal. Prducts which are available fr us are: Number f CPUs Size f lcal disks Size f RAM Kernel type Number f netwrk interfaces Pre-installed applicatins Virtualizatin type If a virtual machine shuld be high-available
Over the status line by using +/- each prduct can be activated r deactivated t shw r hide it fr the user in the clud prtal. Please nte: Prducts which are deactivated but are still active within a virtual machine cntinue t be billed. T create a new CPU prduct we select the CPU tap and define in the area Define a new CPU prduct ur wanted parameter. The first parameter defines hw many CPUs (cres), here 64, ur prduct shuld have. The secnd parameter determines the value f the prduct and hw many csts ccur per hur during its use. In this example, 10 CCUs per hur fr 64 CPUs ccurs. With the arrw keys the rder n hw the single prducts are displayed in the clud prtal can be determine. The default value is abve ne. Please nte: In the clud prtal standard prfiles in the sizes small, medium and big exist. Accrding t the rder the prfiles are autmatically be determined under the respective prducts. That means that small is always the first value, medium the secnd and big the third.
penqrm als allws t rder virtual machines with pre-cnfigured sftware stacks. Fr this penqrm uses Puppet (Plugins > Deplyment > Puppet). Thus, fr example, it is pssible t rder the ppular LAMP stack. If we have cnfigured ur prduct prtfli, it s the user s turn t rder virtual machines. This is dne via the clud prtal.
penqrm Clud-Prtal T create a new virtual machine (VM) we click n the tap New. An input mask fllws n which we can create ur VM based n the prducts the administratr has determined and apprved in the backend. We chse the prfile Big and a LAMP server. Our virtual machine nw cnsists f the fllwing prducts: Type: KVM-Strage VM RAM: 1 GB CPU: 64 cres Disk: 8 GB NIC: 1 In additin the virtual machine shuld be high-available. This means, if the VM fails, autmatically a substitute machine with exactly the same cnfiguratin is started t wrk n with. Fr this cnfiguratin we will have t pay 35 CCUs per hur. This is equivalent t 0.04 eurs per hur r 0.84 per day r 26.04 per mnth. If we want t rder the virtual machine we select send.
Belw the tap Orders we see all current and past rderings we have made with ur user. The status active in the first clumn shws that the machine is already started. Parallel t this we receive an e-mail including the ip-address, a username and a passwrd, we can use t lg int the virtual machine.
The tap Systems cnfirms bth infrmatin and shws further details f the virtual machine. In additin we have the pprtunity t change the systems cnfiguratin, pause the virtual machine r t restart. Furthermre the lgin via a web-shell is pssible. If the virtual machine is nt needed any mre it can be paused. Alternatively it is pssible that the administratr dispses this due t an inactivity f the system r at a specific time. Creating a virtual machine with the Visual Clud Designer Besides the rdinary way f building a virtual machine, the penqrm Clud prtal enables the user t d that cnveniently via drag and drp. Here the Visual Clud Designer helps, which can be find behind the tap VCD. Using the slider n the left belw Clud Cmpnents it is pssible t scrll between the prducts. Using the muse allws t assemble the Clud Appliance (virtual machine) in the middle with the apprpriate prducts.
Our virtual machine Teststern we assembled in this case with KVM-Strage, Ubuntu 12.04, 64 CPUs, 1024 MB Ram, 8 GB disk, ne NIC, and sftware fr a webserver and the high-availability feature. With ne click n Check Csts, penqrm tells us that we will pay 0.03 EUR per hur fr this cnfiguratin.
T start the rdering prcess fr the virtual machine we click request. We get the message that penqrm starts rlling ut the resurce and we will receive further infrmatin int ur mailbx. The e-mail includes, as described abve, all access data t wrk with the virtual machine. In the clud prtal under systems we already see the started virtual machine.
Creating a virtual machine with the Visual Infrastructure Designer Besides the prvisining f single virtual machines the penqrm clud prtal als ffers the pprtunity t prvide cmplete infrastructures cnsisting f multiple virtual machines and further cmpnents, at ne click. Thus, we use the Visual Infrastructure Designer. This can be fund in the clud prtal behind the tap VID. Using the VID it is pssible t build and deply a cmplete WYSIWYG infrastructure via drag and drp. Fr this purpse, it is necessary t create ready prfiles with pre-cnfigured virtual machines at first, which include fr example webserver, ruter r gateways. These can be deplyed afterwards.
Cntact New Age Disruptin research analysis strategy advisry Rene Buest Dipl.-Infrmatiker (FH) M.Sc. in IT-Management and Infrmatin Systems Kernerstrasse 30 24103 Kiel, Germany Phne: +49 (0)431 28 93 42 52 Mbile: +49 (0)173 36 49 468 E-Mail: hell@newagedisruptin.cm Web: http://newagedisruptin.cm Twitter: @ReneBuest CludUser: http://cluduser.de Image surce cver: Paul-Gerg Meister / PIXELIO