Parallels Cloud Storage

Parallels Cloud Storage White Paper Best Practices for Configuring a Parallels Cloud Storage Cluster www.parallels.com

Table of Contents Introduction... 3 How Parallels Cloud Storage Works... 3 Deploying Parallels Cloud Storage... 4 Best Practices for Parallels Cloud Storage... 5 Hardware Recommendations... 5 Choosing a Cluster Configuration... 5 Defining a Cluster... 7 Creating Clusters with Fewer Than 10 Servers... 8 Forming a Cluster of Three Servers... 8 Expanding the Cluster to Five Servers... 9 Creating Clusters with 10 or More Servers... 9 Forming a Cluster of 15 Servers... 9 Expanding the Cluster to 25 Servers... 10 Expanding the Cluster to 45 Servers... 11 Monitoring the Cluster... 11 Parallels Cloud Storage: a Component of Parallels Cloud Server... 12 Conclusion... 12 Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 2

Introduction Parallels Cloud Storage is a distributed, shared storage solution that decouples computation from storage. By doing so, it enables virtual machines and containers to be instantly migrated to an operational physical server whenever the original server becomes unavailable. This approach also enables storage to be distributed across multiple physical machines, rather than being limited to locally attached disks so if one data source becomes unavailable, virtual machines and containers can easily access the data they need from another source. With Parallels Cloud Storage, you can: Store and run Parallels virtual machines and containers. Eliminate VPS downtime due to hard drive failures. Deliver high-availability cloud servers to your customers, with zero-downtime server migration and non-disruptive replication. Achieve performance levels that meet or exceed those of local disks. Save money by turning your unused disk space into a resilient cloud storage pool. Built on SATA direct-attached storage and a 1 GigE networking infrastructure, Parallels Cloud Storage provides SAN-like capabilities for data centers. Its storage can be used for running both containers and virtual machines, as well as for storing snapshots, backup images, large log and media files, and shared hosting scenarios. This paper explains how to set up a Parallels Cloud Storage cluster consisting of anywhere from 3 to 45 servers, based on a network infrastructure from 1 GigE to 10 GigE. It also discusses how you can boost I/O performance and IOPS (input/output operations per second) using solid state drive (SSD) caching How Parallels Cloud Storage Works As shown in Figure 1, Parallels Cloud Storage consists of three major components: a metadata server (MDS), a chunk server (CS), and a client (the virtual machine or container). The system works by splitting the image file of a container or virtual machine into fixed-size data chunks, which it then distributes and replicates among participating servers in the cluster. For fastest possible access, the system always places the first copy of each chunk close to the container or virtual machine ideally, on the same server. The components have the following functions: MDS: The metadata server tracks the replicas of the chunks and their location in the cluster, storing this metadata in an in-memory database for fast access. The only time the metadata is needed is in the initial communication between the client and chunk server, to determine the location of a particular data chunk. The system will operate with just a single MDS, but you will need at least three MDS instances to ensure high availability. CS: The chunk server stores the data chunks and performs read/write operations on them. Client: The client is the container or virtual machine that accesses the stored data. Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 3

Parallels Cloud Storage Architecture Clients Ethernet MDS 1 MDS 2 MDS 3 Meta Data Server (MDS) Stores metadata in memory Tracks data chunks and their versions Is highly available Can run on the same server as the chunk server and client CS CS CS CS...N Chunk Server (CS) Stores data chunks Manages data chunks Performs read/write operations on data chunks Can run on the same server as the client Figure 1. Parallels Cloud Storage architecture overview Communication between all three components takes place through conventional Ethernet switches. Parallels Cloud Storage doesn t require any special switch configuration, further minimizing your infrastructure investment. Parallels Cloud Storage is very flexible, enabling you to place its components on servers in any combination. If a cluster has fewer than 10 servers, we recommend that you place all three components on the same server. If a cluster has 10 or mores servers, we recommend that you place each MDS component on a dedicated server to avoid performance degradation. Parallels Cloud Storage will deliver optimal performance with any cluster size. You can add storage on the fly to accommodate additional nodes and devices, and you can just as easily remove unneeded storage from the cluster all without any downtime. When you introduce new storage servers into a cluster, or when servers come back online after a failure, the system will automatically optimize its chunk placement and correct any imbalances across the total cluster. Deploying Parallels Cloud Storage Parallels Cloud Storage was designed to maximize the efficiency of your existing infrastructure, without requiring any additional hardware investment. However, to use Parallels Cloud Storage, you will need to make sure that any existing servers that will be in the cluster are running Parallels Cloud Server and that all server disks that will be participating in the cluster are formatted using the EXT4 file system. We recommend limiting the storage cluster to a maximum size of 1PB, which will require 16GB of memory per MDS server. We also recommend using a dedicated storage network to prevent customer requests from negatively affecting storage traffic. To further ensure top performance, the cluster should be limited to a single rack, with only one switch hop between any two nodes in the cluster. For improved performance and reliability, you may want to install a small Solid State Drive (SSD) into each CS and client node. The SSD is used for read/write caching, data checksums, and scrubbing (periodic data verifications that perform background reads of cluster data to make sure that it is still available and readable). To avoid any performance degradation, you can limit the amount of bandwidth used by scrubbing operations. By storing the file system journal for the CS on the SSD, you will improve data commit latency and allow the CS to burst random I/O speeds until the journal is full. The journal will also track all data checksums, verifying them during both data reads and scrubbing. If you use an SSD, for best performance we recom mend deploying 10 GigE Ethernet interfaces and switches for the storage network. However, even with a 1 GigE Ethernet, an SSD will significantly improve IOPS. Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 4

Best Practices for Parallels Cloud Storage This section describes the recommended configuration for cluster sizes ranging from 3 to 45 machines. For additional information about installation, please see the Parallels Cloud Storage Guide, available at http://www.parallels.com/products/pcs. HARDWARE RECOMMENDATIONS Although Parallels Cloud Storage will perform well on any existing infrastructure, as you expand your server deployment, we recommend the configurations shown in Table 1 to deliver optimal performance. The table illustrates the recommended hardware configurations both for dedicated MDS servers and for servers with shared MDS, CS, and client components. CHOOSING A CLUSTER CONFIGURATION As mentioned above, you can install the MDS on the same machine as the CS and clients in the case of clusters consisting of nine or fewer CSs and clients. For larger clusters, however, we recommend that you install MDS components on dedicated machines. Parallels Cloud Storage is designed to grow on demand, so you can easily add another dedicated MDS machine to an existing cluster if needed. The MDS requires the availability of a quorum (i.e., a majority of MDS nodes) to recover from a node failure, so to ensure high availability, you will need a minimum of three MDS servers. Table 1: Recommended Hardware Configurations for Parallels Cloud Storage Server Type System Dedicated MDS DELL PowerEdge R420 CPU 1x Intel Xeon E5-2440 RAM HD NIC 4x 4GB Registered DIMM, 1333 MHz, Dual Rank 1x 100GB Solid State Drive, SATA, MLC, 3G, 2.5in Hot-Plug, 3.5in 1x Intel X540 DP 10GBASE-T Server Adapter 1x On-Board Dual Gigabit Network Adapter Server Type System Shared MDS/CS/Client DELL PowerEdge R720xd CPU 2x Intel Xeon E5-2660 RAM HD SSD HD SATA NIC 16x 8GB Registered DIMM, 1333 MHz, Low Volt, Dual Rank 2x 100GB Solid State Drive, SATA, MLC, 3G, 2.5in Hot-Plug, Flex Bay 12x 2TB 7.2K RPM SATA 3.5in Hot-Plug Hard Drive RAID0 1x Intel X540 DP 10GBASE-T Server Adapter 1x Intel I350 DP 1Gb Server Adapter 1x Broadcom 5720 DP 1Gb Network Adapter Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 5

The cluster configuration example shown in Table 2 is based on the recommended hardware configuration shown in Table 1, with 12 hard drives per shared MDS/CS/client server. In the hypothetical example in Table 2, the hoster has deployed 40 servers. Given the typical annual hard drive failure rate of 4%, in any given year, this hoster can expect to experience 19.2 hard drive failures (40 X 12 X.04). Of course, not all the failures would affect the quality of service for example, some might be redundant drives in a RAID configuration. For the purposes of this example, we assume conservatively that just 20% of the failures will have an impact on service. Based on the recommended configuration shown in Table 1, we also assume that each node can support 80 customers (i.e., 80 virtual servers) 1 ; that each customer has an average lifespan of 52 months, with half of that lifespan remaining at the time of failure; that each customer brings in monthly revenues of $30; and that the profit margin on those revenues is 30%. Since it can often take a day or two to get a failed hard drive running again and the data copied to a new drive, each failure can easily result in one to two days of virtual private server (VPS) downtime. And because most customers are not going to stay with a hoster when their site has been down for that period of time, we assume that the customers affected by a disk failure will most likely leave. Therefore, as Table 2 shows, even with our conservative assumptions, 80 hard drive failures per year X 20% (the number affecting service) X 40 customers per node X 26 months (average customer lifespan remaining) X $30 per month adds up to nearly $240,000 in lost revenues and over $70,000 of lost profits. With Parallels Cloud Storage, however, VPSs will instantly and automatically be switched to a different storage location when a hard drive fails, thereby completely eliminating downtime due to hard drive failures. Table 2: Annual Lost Revenues Due to Churn Resulting from HDD Failures Hardware Assumptions No. of servers 40 No. of HDDs per server 12 Replacement rate per year 4% No. of HDD failures per year (40 X 12 X.04) 19.2 Lost Revenue and Profit Due to Churn % of HDD failures that have a service impact (assumption) 20% Typical no. of customers per hardware node (assuming 80 VMs per node) 80 Average customer life span (per industry norms) 52 months Average remaining life span at time of failure 50% Average monthly revenue per customer (per industry norms) $30.00 Lost Revenue/Yr (19.2 X.2 X 80 X 26 X $30) $239,616.00 Profit Margin 30% Lost Profits /Yr ($239,616 X.30) $71,884.80 1 Note that the actual number of virtual servers that a given node can support will vary based both on the hardware configuration of the physical server and the size and load that each virtual server places on it. Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 6

Based on the recommended hardware configuration shown in Table 1 and the lost revenue assumptions shown in Table 2, Table 3 shows the amount of revenue you will save as you grow a cluster from 3 to 45 servers, as well as the recommended configurations in each case. As the table illustrates, both the effective capacity and the amount of saved revenues will grow as the cluster size increases. Table 3: Incremental Capacity Gains and Saved Revenue With Clusters of Different Sizes No. of Servers No. of Dedicated MDS Servers No. of CS + Client + MDS Servers No. of CS + Client Servers No. of Replicas Effective Capacity No. of Drive Failures 1 Saved Revenues 2 3 Î 3 Î 3 24TB 1.44 $ 17,971 5 Î 3 2 3 40TB 2.40 $ 29,952 15 5 Î 10 3 80TB 4.80 $ 59,904 25 5 Î 20 3 160TB 9.60 $119,808 45 5 Î 40 3 320TB 19.20 $239,616 1 Calculated by multiplying the number of CS machines X 12 drives per server X 4% replacement rate per year. 2 Calculated by multiplying the annual number of drive failures (column 7) X 20% service impact X 80 customers per node X 26 months average remaining customer lifespan X $30 average customer monthly revenue. DEFINING A CLUSTER Parallels Cloud Storage is easy to install and configure. The following example describes the recommended process and the command-line instructions to set up a Parallels Cloud Storage cluster. The first thing you need to do is define a cluster. All servers in the cluster must be named pcs1, pcs2, pcs3, pcs4, etc., and must be configured in your Domain Name System (DNS). That s because DNS service (SRV) records are needed to configure auto-discovery and auto-discovery is what new MDS, CS, and client components use to automatically obtain information about the MDS components running in the cluster. Make sure that you set SRV records for all machines participating in the cluster, using the following format: _pstorage._tcp.cluster SRV 0 1 2510 pcs1 _pstorage._tcp.cluster SRV 0 1 2510 pcs2 _pstorage._tcp.cluster SRV 0 1 2510 pcs3 _pstorage._tcp.cluster SRV 0 1 2510 pcs4 The Parallels Cloud Storage cluster will require authenticated access to execute cluster-related commands, such as adding or removing additional MDS and CS machines. You can change the password from any machine participating in the cluster, using the following command: --change-password Parallels Cloud Storage requires a valid license to create containers and virtual machines on-top. You can install it, using the following command: pstorage -c CLUSTER load-license -p <key> Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 7

If you have obtained the license in form of a file, you can install it, using the following command: pstorage -c CLUSTER load-license -f <filename> The following sections describe the best practices for creating two types of clusters: those with fewer than 10 servers, and those with 10 or more servers. CREATING CLUSTERS WITH FEWER THAN 10 SERVERS Forming a Cluster of Three Servers 1. To create a cluster of three servers, you need to specify the cluster name (which, in our example, is CLUSTER); identify the listening IP address (the IP address of the storage network), and name the directory where the MDS will store its files. 2. Then you can start the MDS service. To do so, you ll execute the following command on the pcs1 server: pstorage c CLUSTER make-mds -I -p -a <ip>:2510 -r /pcs/cluster-mds service pstorage-mdsd start This command tells the cluster to create its first MDS instance, which will immediately become the master server. At this point, the system will ask you for a password to protect the cluster from unauthorized access. 3. Once you create the password, create two more MDS servers, by executing the following commands on the pcs2 and pcs3 servers: pstorage -c CLUSTER make-mds -a <ip>:2510 -r /pcs/cluster-mds service pstorage-mdsd start 4. The next step is to add the CS components to the three servers in the cluster, so that the system can store replicas of the cluster s data. To do this, execute the following commands on the pcs1, pcs2, and pcs3 servers: pstorage c CLUSTER make-cs -r /pcs/cluster-cs service pstorage-csd start 5. Now, to enable client machines to access the cluster, mount the distributed storage system across all three machines. To do so, execute the following commands on the pcs1, pcs2, and pcs3 servers: mkdir /pcs/cluster pstorage-mount -c CLUSTER /pcs/cluster echo pstorage://cluster /pcs/cluster fuse.pstorage rw 0 0 >> /etc/ fstab 6. The next step is to set up multiple replicas, so the cluster will survive even if one of its servers goes down. Right now, because the cluster was initially created from one machine, you have only one instance of the data. To set up the cluster with three replicas, for example, you would execute the following command on the pcs1 server: pstorage c CLUSTER set-attr -R /pcs/cluster replicas=3:1 Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 8

7. You will also need to modify the default storage locations of the containers and virtual machines in the cluster, so they will store their virtual disk data inside Parallels Cloud Storage. 8. For containers, you will do this by editing the /etc/sysconfig/vz configuration file on the pcs1, pcs2, and pcs3 servers with the following setting: VE_PRIVATE= /pcs/cluster/$veid 9. For virtual machines, you will do this by executing the following command on the pcs1, pcs2, and pcs3 servers: prlsrvctl user set --def-vm-home /pcs/cluster At this point, all three machines are participating in the cluster, with all three components (MDS, CS, and client) present on each machine. The existence of three replicas across the cluster means that the storage is now redundant and can survive the loss of one machine. Expanding the Cluster to Five Servers To expand the cluster to five servers, you ll go through all the steps in Forming a Cluster of Three Servers, above, and then add two more servers with just CS and client components no MDS. To add the two additional servers, execute the following commands on the pcs4 and pcs5 servers: pstorage c CLUSTER make-cs -r /pcs/cluster-cs service pstorage-csd start mkdir /pcs/cluster pstorage-mount -c CLUSTER /pcs/cluster echo pstorage://cluster /pcs/cluster fuse.pstorage rw 0 0 >> /etc/ fstab sed s,^ve_private=.*,ve_private= /pcs/cluster/$veid,g -i /etc/ sysconfig/vz prlsrvctl user set --def-vm-home /pcs/cluster You have now successfully formed a cluster of five servers. In this cluster, the pcs1, pcs2, and pcs3 servers each have MDS, CS, and client components, while the pcs4 and pcs5 servers have only the CS and client components. CREATING CLUSTERS WITH 10 OR MORE SERVERS Forming a Cluster of 15 Servers If a cluster will consist of 10 or more CS machines, the MDS components must be set up on dedicated machines to avoid performance degradation. In configuring a cluster of this size, you will need to specify the cluster name (in our example, CLUSTER), the listening IP address, and the directory where the MDS will store its files. Then you can start the MDS service. 1. To begin execute the following command on the pcs1 server: pstorage c CLUSTER make-mds I -p -a <ip>:2510 -r /pcs/cluster-mds service pstorage-mdsd start This command tells the cluster to create the first MDS instance, which will immediately become the master server. At this point, the system will ask you for a password to protect the cluster from unauthorized access. 2. Once you create the password, you will need to create four more MDS instances, which you will do by executing the following commands on the pcs2, pcs3, pcs4, and pcs5 servers: pstorage c CLUSTER make-mds -a <ip>:2510 -r /pcs/cluster-mds service pstorage-mdsd start Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 9

3. The next step is to add the CS components to the remaining servers in the cluster. You ll do this by executing the following commands on servers pcs6 through pcs15: pstorage c CLUSTER make-cs -r /pcs/cluster-cs service pstorage-csd start 4. Now, to enable clients to access the cluster, you ll need to mount the distributed storage system across all 10 CS machines. To do so, execute the following commands on servers pcs6 through pcs15: mkdir /pcs/cluster pstorage-mount -c CLUSTER /pcs/cluster echo pstorage://cluster /pcs/cluster fuse.pstorage rw 0 0 >> /etc/ fstab Because you initially created the cluster from a single machine, you have only one instance of the data at this point, so the cluster would not be able to survive a lost machine. 5. To create the minimum necessary three replicas, execute the following command on the pcs6 server: pstorage -c CLUSTER set-attr -R /pcs/cluster replicas=3:1 6. You will also need to modify the default storage locations of the containers and virtual machines in the cluster, so they will store their virtual disk data inside Parallels Cloud Storage. 7. For containers, you will do this by editing the /etc/sysconfig/vz configuration file on servers pcs6 through pcs15 with the following setting: VE_PRIVATE= /pcs/cluster/$veid 8. For virtual machines, you will do this by executing the following command on servers pcs6 through pcs15: prlsrvctl user set --def-vm-home /pcs/cluster Now you have a cluster of 15 machines, in which servers pcs1, pcs2, pcs3, pcs4, and pcs5 are dedicated MDS machines, and servers pcs6 through pcs15 have the CS and client components. The storage is redundant, can survive the simultaneous loss of two machines, and can be extended on the fly. Expanding the Cluster to 25 Servers To create a cluster with 25 servers, you ll first go through all the steps in Forming a Cluster of 15 Servers, above. Then you will need to add 10 more servers with the CS and client components. To do so, execute the following commands on servers pcs16 through pcs25: pstorage c CLUSTER make-cs -r /pcs/cluster-cs service pstorage-csd start mkdir /pcs/cluster pstorage-mount -c CLUSTER /pcs/cluster echo pstorage://cluster /pcs/cluster fuse.pstorage rw 0 0 >> /etc/ fstab sed s,^ve_private=.*,ve_private= /pcs/cluster/$veid,g -i /etc/ sysconfig/vz prlsrvctl user set --def-vm-home /pcs/cluster You have now successfully formed a cluster of 25 servers, in which servers pcs1, pcs2, pcs3, pcs4, and pcs5 are dedicated MDS machines, and servers pcs6 through pcs25 have the CS and client components. Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 10

Expanding the Cluster to 45 Servers To create a cluster with 45 servers, you ll first go through all the steps in Forming a Cluster of 15 Servers, above. Then you will need to add 30 more servers with the CS and client components. To do so, execute the following commands on servers pcs16 through pcs45: pstorage c CLUSTER make-cs -r /pcs/cluster-cs service pstorage-csd start mkdir /pcs/cluster pstorage-mount -c CLUSTER /pcs/cluster echo pstorage://cluster /pcs/cluster fuse.pstorage rw 0 0 >> /etc/ fstab sed s,^ve_private=.*,ve_private= /pcs/cluster/$veid,g -i /etc/ sysconfig/vz prlsrvctl user set --def-vm-home /pcs/cluster You have now successfully formed a cluster of 45 servers, in which servers pcs1, pcs2, pcs3, pcs4, and pcs5 are dedicated MDS machines and servers pcs6 through pcs45 have the CS and client components. MONITORING THE CLUSTER From the command line, you can view the state of the cluster to learn if it is healthy, how much space is available, and what the current I/O rates are, as well as detailed information about chunk placement. To access this data, execute the following command on the pcs1 server: pstorage c CLUSTER top Figure 2 shows an example of the data this command will display. In this example, the screen tells you that the cluster consists of three machines, all healthy, each with MDS, CS, and client components. The cluster has 0 data chunks (replicas) because there is no data inside clustered storage yet. The command also displays an event log that informs you about recent activity in the cluster, such as data rebalancing and adding or removing a node. Figure 2. Cluster state data displayed by Parallels Cloud Storage Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 11

Parallels Cloud Storage: a Component of Parallels Cloud Server Parallels Cloud Storage is delivered as a component of Parallels Cloud Server 6 the only solution available today that permits the coexistence of servers running container and hypervisor technology, as well as migration between these servers. By eliminating virtualization lock-in, Parallels Cloud Server lets you take advantage of the scalability and streamlined operations of operating system virtualization (containers) while also making use of high-performance virtual machines (hypervisors) for isolated applications or simultaneous heterogeneous environments. Parallels Cloud Server 6 comes with two virtualization options: Parallels Containers for Linux, which sets the hosting industry standard for profitable, secure, and flexible VPS and cloud server offerings. Uniquely suited to cloud server virtualization, it enables nearinstant provisioning and on-the-fly modification of hosting and cloud server plans, while delivering up to three times better density and up to 350% better performance than leading hypervisors. Parallels Hypervisor, which gives you the flexibility to create virtual machines for customers who prefer to deploy and maintain a variety of operating systems simultaneously. By combining both virtualization technologies into a single solution, Parallels Cloud Server provides you with the freedom to choose the most efficient technology for each virtual server. Conclusion By decoupling computation from storage, Parallels Cloud Storage lets you instantly migrate virtual machines and containers to an operational physical server whenever the original server becomes unavailable, as well as providing you with multiple replicas of stored data. The result is virtually no downtime due to server or hard drive failures. In addition, Parallels Cloud Storage saves you money by turning your unused disk space into a resilient cloud storage pool, thereby increasing the value of your hardware investment. For more information about Parallels Cloud Storage, please visit http://www.parallels.com/products/pcs/ Contact Us For more information about Parallels hosting and cloud solutions, please contact: Parallels, Inc. 500 SW 39th St. Suite 200 Renton, WA 98057 +1 425 282 6448 www.parallels.com Copyright 2013 Parallels IP Holdings GmbH. All rights reserved. Parallels and the Parallels logo are registered trademarks of Parallels IP Holdings GmbH. Other product and company names are the trademarks or registered trademarks of their respective owners. Parallels Best Practices for Configuring a Parallels Cloud Storage Cluster 12