Intel Cloud Builders Guide to Cloud Design and Deployment on Intel Platforms

Intel Cloud Builders Guide Intel Xeon Processor-based Servers NTT DATA BIZXAAS* Full OSS Cloud Solution Intel Cloud Builders Guide to Cloud Design and Deployment on Intel Platforms NTT DATA BIZXAAS* Full OSS Cloud Solution Audience and Purpose This document is for IT professionals in companies, governments, local governments, universities, and schools who are looking to build a private cloud infrastructure. Intel Xeon Processor 5500 Series Intel Xeon Processor 5600 Series When a company decides to build their own private cloud, one of the best solutions is to use open source software (OSS) packages. It is not always easy to choose the best packages for their requirements and set up these OSS packages. This reference architecture shows how a private cloud infrastructure is designed and constructed with BIZXAAS* full OSS cloud solution and Intel Xeon processor-based platforms. August 2011

Table of Contents Executive Summary... 3 Introduction... 4 Test Bed Blueprint Overview... 5 Hardware Description... 5 Physical Architecture... 6 System Design... 6 Server Requirements... 6 Network Topology... 7 Software... 7 Technical Review... 7 Adminstrators Interface... 7 Create user accounts... 7 Change parameters for VM... 7 Change max resource for each group... 10 Monitoring cloud status... 12 Cloud Client, a GUI-based Interface for End-Users... 14 Create new instance... 14 Terminate instance... 16 Create new VM image from a running instance... 16 Create multiple instances from a group of VM images... 17 Managing network... 18 Associate public IP... 20 Managing volume... 21 Use Case of BIZXAAS* Full OSS Cloud Solution... 22 edubase Cloud as an Educational Computing Environment... 22 Things to Consider... 23 Requirements for CPU and Number of NC... 23 Requirement for Storage Capacity... 23 Conclusion... 23 Glossary... 23 References... 23 2

Executive Summary When a company decides to build their own private cloud, one of the best solutions is to use open source software (OSS) packages. As various kinds of functions are required for a cloud, such as cloud management software, monitoring software, user account management software, and data backup software, the integration of many OSS packages is necessary. It is not always easy to choose the best packages for the requirements. It also requires a lot of time and knowledge of OSS packages to try integrating OSS as a cloud platform. BIZXAAS* full OSS cloud solution provides a complete set of the OSS packages required to build and manage a private cloud. With BIZXAAS full OSS cloud solution, users can build their own private cloud themselves. Detailed instructions of installations and settings of OSS packages are provided with the solution. Professional support for both design and operation phase is also provided. In a typical configuration of BIZXAAS, the cloud consists of one cloud controller (CLC), 1 cluster controller (CC), 15 node controllers (NC), 1 storage server, and 1 monitoring server. The cloud can support up to 120 virtual machine (VM) instances with 8-core CPU and 15-NC configuration. It can support even more VMs, if the assignment policy allows CLC to make more than one VM assigned to a core. With the graphical user interface (GUI), the end users of BIZXAAS cloud can control VMs, such as launching a VM, creating a cluster with VMs, installing software on VMs, and stopping VMs. No administrators are needed for these operations. The primary components of BIZXAAS consists of several OSS packages including Eucalyptus for cloud management, Open LDAP for user account management, Hinemos* for monitoring the status of hardware and software, and Amanda for data backup. BIZXAAS full OSS cloud solution supports Microsoft Windows* XP, Microsoft Windows* 7, Microsoft Windows Server* 2003, and Microsoft Windows Server* 2008, as well as Red Hat Linux and CentOS as guest VM images. It supports these operating systems (OS) to boot from both an object store (called Walrus) or from an Elastic Block Storage (EBS). When an OS image boots from Walrus, the change of disk image is ephemeral and is lost at the termination of the instance. When an OS image boots from EBS, on the other hand, the changes of disk images are persistent and no special operation is required to preserve the image. Figure 1: Architecture of BIZXAAS full OSS cloud solution test bed 3

Introduction Many organizations, such as private companies, governments, local governments, and universities, own various computer systems that are customized for their own needs. The systems are often based on different software architectures and different designs for operations, as they had developed progressively and were not designed as a comprehensive system. The total cost of ownership (TCO) of these systems, as well as their operational cost, tends to become higher as the systems need to be changed according to the requirements. One of solutions for this situation is to build a private cloud and consolidate the existing system into the cloud. It can lead to reduction of both hardware and operations costs. When a company decides to build its own cloud to consolidate its systems, one good choice is to make use of OSS. In order to build a private cloud by using OSS, the company would need several types of OSS packages, such as cloud management software, user account management software, monitoring software, and data backup software. This is because no onepackage solution exists for an OSS-based cloud. In design phase, it is often time consuming to try finding the best combination of OSS packages and find the best configuration of these OSS packages for the construction of private clouds. Deep knowledge of these OSS packages is required to use a set of OSS packages. In the operation phase, updating the OSS package is a hard task for administrators to achieve. BIZXAAS full OSS cloud solution provides a complete set of the OSS packages required to build and manage a private cloud. Detailed instructions of installations and settings of OSS packages are provided with the solution. Professional support for both design and operation phase are also provided. BIZXAAS full OSS cloud solution includes the following OSS packages: Cloud Management Software Eucalyptus-based cloud management software is used. The following functions are implemented on BIZXAAS full OSS cloud solution: Xen is selected as VM technology GUI-based original VM and VM image management software, named cloud client Support of Microsoft Windows as guest VM VM can boot from EBS Direct access of external iscsi storage device from node controller Account Management Open LDAP is used for the management of user account in the cloud. Monitoring Software Hinemos, NTT DATA s OSS based monitoring software, provides the following monitoring functions for the cloud: Monitoring of hardware, including servers and network switches Monitoring of system resources, such as CPU and memory usage in cloud servers, capacity usage of storage devices, and bandwidth of network traffic Monitoring of cloud services, such as process monitoring of important service, and monitoring of system log in servers Munin, a networked resource monitoring tool, is supported for the monitoring the resources used by VMs. Data Backup Software Amanda is an OSS package that supports data archive among networks. It is used for backup and restoration of data in a cloud. 4

Test Bed Blueprint Overview In this section, a typical configuration of BIZXAAS full OSS cloud solution is described as a test bed. Hardware Description The following table represents a set of hardware prepared for the test bed of BIZXAAS full OSS cloud solution. Server / Component Required Number Processor Configuration Additional Information Cloud Controller (CLC) 1 Intel Xeon E5620 16GB (4GBx4) DDR3-SDRAM 600GB HDD x 2 (10000 rpm, RAID1) Internal DVD-ROM RAID Controller Board (w/256mb, RAID 0/1) 1GBps Ethernet Port x 6 Cluster Controller (CC) 1 Intel Xeon E5620 16GB (4GBx4) DDR3-SDRAM 600GB HDD x 2 (10000 rpm, RAID1) Internal DVD-ROM RAID Controller Board (w/256mb, RAID 0/1) 1GBps Ethernet Port x 6 Node Controller (NC) More than 1 15 (typical) Intel Xeon E5620 32GB (4GBx8) DDR3-SDRAM 600GB HDD x 2 (10000 rpm, RAID1) Internal DVD-ROM RAID Controller Board (w/256mb, RAID 0/1) 1GBps Ethernet Port x 2 Monitoring Server 1 Intel Xeon E5620 32GB (4GBx8) DDR3-SDRAM 300GB HDD x 2 (10000 rpm, RAID1) Internal DVD-ROM RAID Controller Board (w/256mb, RAID 0/1) 1GBps Ethernet Port x 4 Cloud Storage Server (Recommended) More than 1 Network Storage Device 6TB (600GB x 12) 15000rpm SAS, RAID 10 Protocol: iscsi Memory Cache: 2GB / 1GB 1GBps Ethernet Port x 2 Backup Storage Server (Optional) Network Storage Device 9TB (1000GB x 12) 10000rpm SATA, RAID 6 Protocol: iscsi Memory Cache: 2GB / 1GB 1GBps Ethernet Port x 2 Network Switch 4 Layer 2 network switch 1GBps Layer 2 switch / 1GBps (require 1 switch for each segment. 3 switches in total) 1GBps Ethernet Port x 16 Table 1: Components of BIZXAAS full OSS cloud solution test bed Note that readers are expected to consider the following items and modify the test bed according to their requirements. Compute (CPU) requirement affects number and specification of node controller (NC) Storage requirement affects capacity, speed, and redundancy of specification of cloud storage server Backup policy affects the requirements for backup storage server 5

Physical Architecture Figure 2 represents architecture of BIZXAAS full OSS cloud solution test bed. Details of components are described in the following subsections. Figure 2: Architecture of BIZXAAS full OSS cloud solution test bed System Design Server Requirements The following servers are used for BIZXAAS full OSS cloud solution test bed. 1. CLC and CC A primary function of CLC is the management of cloud resources, such as CPU, IP address, and storage. Allocations of CPU resources over NC are determined by CLC, for example. CC provides network access for VMs. All of the traffic between external network and NC is handled by CC. In the minimum configuration, it is possible to integrate CLC and CC in one physical machine. 2. NC NC provides CPU resources for the VM. At least one NC is required on a BIZXAAS full OSS cloud solution. As one CPU core supports more than one VM, an 8-core NC can support more than 16 VMs, for example. However, as parts of physical memory in NC are assigned for each VM, and virtual memory is not used in NC, the size of physical memory must be larger than total memory assigned for VMs. 3. Monitoring server Monitoring of physical hardware and services in BIZXAAS full OSS cloud solution is provided by the monitoring server. 4. Cloud storage server All of the VM images are stored in cloud storage. 5. Cloud backup storage server The cloud backup storage server is used to keep several sets of copies of the cloud storage server. The cloud backup storage server is optional but recommended for building a cloud with high reliability systems. 6

Network Topology BIZXAAS full OSS cloud solution consists of four independent network segments. 1. Service network segment The service segment is used to access VMs from external networks. For the separation of each VM, a VLAN tag is allocated and assigned for each VM. 2. Cloud network segment The cloud network segment is used for the communication between CC, CLC, and NC. 3. Management network segment The management network segment is used for monitoring and the controlling the physical machines on the cloud. 4. Storage network segment The storage network segment is used to provide access to iscsi cloud storage and cloud backup storage. Note that all of the network segments should use a 1GbE switch. Support of High Availability (HA) infrastructure is optional. At minimum the cloud should consist of one CLC/CC, one NC, one backup server, and one cloud storage server. Technical Review Administrators Interface Create User Accounts 1. Log on to CLC and edit a property file of LDAP. (/etc/eucalyptus/cloud.d/ authenticatioin.properties) 2. In the file, find the line of URL address (in this case, line 4, URL= `) and edit it. Similarly, find the line of account information represented by ( Account= `, at line 6 in this case,) and edit it. 3. Restart CLC to reflect the changes. 1. #Authentication Class 2. AuthenticationClass=jp.co.nttdata.rd.rdhc.transport.auth. ADAuthentication 3. # LDAP URL 4. URL=ldap://172.16.64.111:389 5. # LDAP Account for UIM 6. Account=CN=%s,OU=Members,OU=deps,DC=company,DC=jp Change Parameters for VM 1. Launch Internet Explorer on the management console and gain access to https://<clc-ip>:8443/, where <CLC-IP> is IP address of CLC. 2. The following screen will appear on the Web browser. Log on the Eucalyptus Web Console with privileged ID. Software The following list shows a summary of software used in BIZXAAS full OSS cloud solution. 1. Eucalyptus-based cloud management software 2. LDAP is used for the management of user account 3. Hinemos is used for monitoring of the status of servers, network devices, and software service on cloud 4. Amanda for backing up data 7

3. Choose a Configuration tab. 4. On top of the tab, the IP address of CLC, CC, and DNS server are shown. 8

5. Scroll down to the bottom of the page. Find Walrus Configuration section. A B 6. Change the following parameters. Item A B Meaning Max number of VM image per group Max size of VM image 7. Click Save Walrus configuration to save the changes. 8. Scroll down the page and find EBS Volume setting. C 9. Change the following parameters. Item C Meaning Max size of EBS volume 9

10. Click Save cluster configuration to save the change. 11. Scroll farther down the page and locate Settings of VM Types. D E F 12. Change the following parameters. Item D E F Meaning Number of CPU for each machine type Max memory size for each machine type Max Hard Disk size for each machine type 13. Click Save VmTypes to save the changes. Change Max Resource for Each Group 1. Connect the CLC database named eucalyptus_general with sqleonardo on the management console. 10

2. On the left pane of the sqleonardo window, expand the entry named vclc0001_eucalyptus_general and select TABLE. 3. Find a line on the right pane, where the TABLE_NAME is USERCONFIG, and then double click the line. USERCONFIG is shown on screen. 4. Locate the following entry and change the column named VALUE. Parameter Column 'KEY' 'USER_NAME' (A) Max number of CPU for the group 00 <Group ID> (B) Max size of volume for the group 01 (C) Max number of Public IP address for the group 02 5. Unselect the cell edited above and click save changes button. 6. The changes are committed to DB. 11

Monitoring Cloud Status 1. On the management console, launch Hinemos Client from Windows START menu. Login window will appear on screen. 12

Intel Cloud Builders Guide: NTT DATA BIZXAAS* Full OSS Cloud Solution 2. Type in user id and password. 3. Type in IP address of backup server at connection field. 4. Choose login button. User and Connection ID are shown here A B C 13

View View Name Information A Summary View The most important events, such as those with critical status, are listed on this view. Details of those events are shown on Status View and Events View. B Status View The Status of all machines is shown on this view. Unlike event view, only the latest information is shown on this view. C Event View All of the events received are listed in chronological order. Table 2: Summary of views on Hinemos Client Importance Criteria Color Meaning Critical RED Monitoring system received a critical message and an immediate action is required. Alert YELLOW Alert messages should be notified by admin immediately. Information GREEN Important messages that admin should know. Unknown BLUE The status of node is unknown. This status is caused by malfunction of the monitoring system. Table 3: Summary of message criteria on Hinemos Cloud Client, a GUI based Interface for End Users Create New Instance 1. Run Cloud Client on a user PC and connect to CLC. 14

2. Locate Centos 5.4 VM image on VM view pane in Cloud Client. 3. In the pull-down menu on the VM image, select Launch VM item. 4. A dialogue box will appear on screen. Select version number, instance type, name of keypair, number of instances, and security group from each pull-down menu on the dialogue box, followed by finish to launch VM. 5. Find the entry of the new VM appearing on the list of instances pane, and then wait until the status of the VM becomes running.' Select ssh from the pull down menu on the VM. 15

Terminate Instance 1. Locate the instance on list of instances pane on Cloud Client. 2. Select terminate instance from pull-down menu. 3. Wait until the VM instance disappears from list of instances pane. Create New VM Image from a Running Instance 1. Locate the VM on list of instances pane on Cloud Client. 2. Select Save Image from pull-down menu on the VM. 3. New image dialogue box appears on the screen. 4. Type in the name of new VM image, VM group, version number of VM, comments for the VM, and disk image size of VM, followed by finish button. 16

5. Wait until the VM image appears on the list of VM image pane on Cloud Client. Create Multiple Instances from a Group of VM Images 1. Launch Cloud Client on user PC. 17

2. Locate a VM group on list of VM images pane, and select run instances from pull-down menu on the VM group. The following dialogue box is shown. 3. Set up parameters for each VM images. The parameter includes: (A) version of VM to use, (B) instance type, (C) keypair to use, (D) number of instances to launch, and (E) security group for the instance. 4. Push finish button for launch all of instances. 5. Wait until all of instances are ready. Managing Network 1. Launch Cloud Client on user PC and connect to CLC. Add a new security group. 2. On Security Group tab on top right pane, recall a pull-down menu with right mouse-click and choose new security group. 18

3. The following dialogue box appears on the screen. 4. Type in the name of security group and comments for the group. Assign a set of access policies for the security group. 5. On top right pane, choose the security group and recall a pull-down menu with right mouse-click and choose add permission. 6. A dialogue box appears on the screen. Type in protocol, port range, and network mask to define permission. 19

7. To allow SSH access to the security group, for example, set protocol to TCP, port to 22, and network mask to required range. 8. Confirm the new security policy is defined on screen. Associate Public IP 1. Launch Cloud Client on user PC and connect to CLC. 2. Allocate public IP 3. A new public IP is allocated and added on the list. Associate public IP to instance. 4. On list of instances pane, find an instance and recall a pull-down menu with right mouse-click. Choose associate public IP followed by public IP address. 5. Confirm the public IP is associated for the instance on list of instances pane. 20

Managing Volume 1. Launch Cloud Client on user PC and connect to CLC. Create EBS volume. 2. On Volume/Snapshot tab on top right pane, recall a pull-down menu with right mouse-click, and choose create new volume. The following dialogue box will appear on the screen. 3. Choose create an empty volume or create a volume from snapshot with radio button. Type in the size of volume for the former case. Create an empty volume Create a volume from a snapshot 4. Click OK to finish. Confirm that the volume is on the list of Volume/Snapshot tab. Attach EBS volume. 5. On list of instances pane, find an instance and recall a pull-down menu with right mouse-click. Choose attach volume followed by Volume name. 21

6. On Volume/Snapshot tab, confirm that the volume is attached to the instance. Mount EBS volume from the instance. 7. Login to the instance with route account. 8. Create file system and mount the volume to the instance. # mkfs -t ext3 /dev/sdh # mkdir /mnt/ebs/ # mount /dev/sdh /mnt/ebs 9. Confirm the volume is mounted on the instance and ready to use. Use Case of BIZXAAS Full OSS Cloud Solution BIZXAAS full OSS cloud solution is used for a wide variety of applications. Major applications include education, workflow management, and software development and test. In this section, a private cloud for education at National Institute of Informatics (NII) in Japan is described as a use case. NII s private cloud is called edubase Cloud, which is designed for Information Technology (IT) education. edubase Cloud as an Educational Computing Environment edubase Cloud can create an environment in which users can acquire necessary IT resources when needed, and can test ideas without restraint. Utilization of cloud in the actual training grounds is expected, from the basic technology of cloud computing to Project-Based Learning (PBL) in the IT field. Also, edubase Cloud is a place to learn essential advanced technological capabilities to manage and develop the cloud platform. edubase Cloud offers the most advanced IT environment and serves as the driving force to accelerate the development of leading IT specialists. Four major features of edubase Cloud are as follows. Exclusiveness: One can acquire a set of personal IT environments for experiment. Interoperability: On edubase Cloud, one can easily work with external cloud services by utilizing edubase Cloud s open interface. Alteration: The full open-source cloud environment enables customization and tuning, including to the modification of the platform itself. Storage Stability: edubase Cloud offers an archive function to save VM images in a reusable form in the long term. Utilization of edubase Cloud in IT education includes: Lectures Training PBL Figure 3: educase Cloud 22

Things to Consider Requirements for CPU and number of NC The number of NC can vary depending on the requirements of the computing resource. In the minimum configuration, one NC is required on a cloud. In the following configurations, one NC can support up to eight VMs without over committing. Requirement for Storage Capacity Cloud storage is used to store VM images. The storage device can be DAS (Direct Attached) or a SAN (Storage Area Network). iscsi is recommended as LAN storage protocols. When calculating capacity of storage requirements, assume each VM requires approximately 5GB of storage space. Requirements for the capacity for cloud backup storage can vary depend on backup requirements. Conclusion BIZXAAS full OSS cloud solution is one of the best cloud solutions with Intel Xeon processor-based systems and open source software. Glossary ATA over Ethernet (AoE) - A network protocol designed for simple, highperformance access of SATA storage devices over Ethernet networks. Amanda An OSS package which supports data archive among network. Cloud Controller (CLC) - A software component that implements Amazon EC2 API and provides Web UI. Cluster Controller (CC) - A software component which manages a set of node controllers. Elastic Block Storage (EBS) Raw block storage devices which can be attached to VM instances. VM instances are also launched from VM images stored in EBS. Hinemos An OSS package which monitors the status of servers, network devices, and software services. It sends messages to the administrators of a cloud if the trigger conditions are fulfilled. It handles SNMP, syslog, http, and smtp. Internet Small Computer System Interface (iscsi) - An Internet Protocol (IP)-based storage networking standard for linking data storage facilities. Lightweight Directory Access Protocol (LDAP) - An application protocol for reading and editing directories over an IP network. Node Controller (NC) - A software component that runs on a node and manages instances on the node. Open LDAP An OSS which implements of Lightweight Directory Access Protocol (LDAP). Open Source Software (OSS) - Computer software that is available in source code form: the source code and certain other rights normally reserved for copyright holders are provided under a software license that permits users to study, change, improve and at times also to distribute the software. Simple Mail Transfer Protocol (SMTP) Simple Network Monitoring Protocol (SNMP) Virtual Machine (VM) - A software implementation of a computer that executes programs like a physical machine. Virtual Machine (VM) Instance - Individual VMs running on physical nodes. Virtual Machine (VM) Image A disk image that is used to start VM. Walrus A software component which implements network storage device like Amazon S3. VM images are stored and executed from Walrus. Xen - A virtual-machine monitor for IA-32, x86-64,and Itanium architectures, which allows several guest operating systems to execute on the same computer hardware concurrently. Xen is an OSS package. References Grace Center, National Institute of Informatics: Full open-source educational Cloud http://grace-center.jp/en/prj_ educloud.html edubase Cloud: An Open-source Cloud Platform for Cloud Engineers, Nobukazu Yoshioka, Shigetoshi Yokoyama, Yoshionori Tanabe, and Shinichi Honiden,National Institute of Informatics, Japan, Workshop on Software Engineering for Cloud Computing (SECLOUD 2011) BIZXAAS Cloud Service: http://bizxaas. net/ (In Japanese) News Release: NTT DATA Rolls Out Full Cloud Service Lineup http://www.nttdata. com/media/2010/040800.html Hinemos : http://www.hinemos.info/ (In Japanese) Amanda: http://www.amanda.org/ Eucalyptus: www.eucalyptus.com/ Open LDAP: http://www.openldap.org/ Intel Cloud Builders: www.intel.com/ cloudbuilders 23

Disclaimers Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. See www.intel.com/ products/processor_number for details. INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROP- ERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR. Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked reserved or undefined. Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information. The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order. Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or by visiting Intel s Web site at www.intel.com. Copyright 2011 Intel Corporation. All rights reserved. Intel, the Intel logo, Xeon, Xeon inside, and Intel Intelligent Power Node Manager are trademarks of Intel Corporation in the U.S. and other countries. *Other names and brands may be claimed as the property of others.