Moving Drupal to the Cloud: A step-by-step guide and reference document for hosting a Drupal web site on Amazon Web Services

Moving Drupal to the Cloud: A step-by-step guide and reference document for hosting a Drupal web site on Amazon Web Services MCN 2009: Cloud Computing Primer Workshop Charles Moad <cmoad@imamuseum.org> Robert Stein <rstein@imamuseum.org> Ari Davidow <adavidow@jwa.org> INTRODUCTION... 2 CONVENTIONS USED IN THIS DOCUMENT... 2 EC2 CONSOLE... 3 PREREQUISITES TO LAUNCHING AN EC2 INSTANCE... 4 CREATING A KEY PAIR... 4 CREATING A SECURITY GROUP... 5 SELECTING A STARTER AMI... 6 LAUNCHING AND CONNECTING TO YOUR EC2 INSTANCE... 7 STARTING YOUR EC2 INSTANCE... 7 CONNECTING TO YOUR EC2 INSTANCE... 8 Preparing Your SSH Key Pair for Putty... 8 CONFIGURING YOUR EC2 INSTANCE... 9 INSTALLING PREREQUISITES... 9 CREATING AN EBS VOLUME... 10 ATTACHING YOUR EBS VOLUME... 11 FORMATTING AND MOUNTING YOUR EBS VOLUME... 11 CONFIGURING MYSQL... 12 CONFIGURING APACHE... 13 INSTALLING AND CONFIGURING DRUPAL... 14 APPENDIX... 15 CONFIGURING CRON FOR DRUPAL... 15 ASSIGNING A STATIC IP TO YOUR EC2 INSTANCE... 16 SNAPSHOTTING YOUR EBS... 17 BUNDLING YOUR EC2 INSTANCE INTO AN AMI... 18 Page 1 of 19

Introduction This document aims to give attendees a step-by-step guide on how to host a Drupal website using the Amazon Web Services. It should also serve as a valuable reference document. The Drupal instance will be hosted on a LAMP stack and steps will be shown on how to backup and snapshot your EC2 instance. Conventions Used in this Document Normal text Filename or itemname Command line Page 2 of 19

EC2 Console This guide uses the EC2 Console, a web interface provided by Amazon which allows complete control of running and managing EC2 instances in the cloud. Page 3 of 19

Prerequisites to Launching an EC2 Instance Creating a Key Pair You must specify a security key pair to use when launching EC2 instances. A key pair is actually used as your SSH key pair for connecting to your instance. There is no password access. You can technically launch an instance without a key pair, but then you would have no means to log into the running instance. In the EC2 console, select Key Pairs from the Navigation section. Click the Create Key Pair button to create a new key pair. In this example we will name our key pair, mcn09. Once you create your key pair, your browser will download the actual private key file. In this example the file will be named, mcn09.pem. Keep this file secure. Page 4 of 19

Creating a Security Group Think of security groups simply as your firewall rules for EC2 instances. You use it to say which ports are open, and who they are open to. You must specify a security group when launching an EC2 instance. At a minimum, you will want to allow access on port 22 so you can SSH and port 80 to serve web pages. In the EC2 console, select Security Groups from the Navigation section. Click the Create Security Group button to create a new group. In this example we will name our security group, mcn09. Optionally enter a description for the group. Configure the ports as shown in the image below. Fill in each line and click Save. The Source item should be set to 0.0.0.0/0 to allow access to allow machines. Reference CIDR Notation to learn more about this setting or click the Help button seen in the interface below. Page 5 of 19

Selecting a Starter AMI You ll need to select an Amazon Machine Image to use as a basis for your EC2 instance. There exists AMI s preconfigured with Drupal. This guide however will use a stock, Amazon provided Fedora 32-bit AMI. Specifically we are using, ami-48aa4921, as shown below. EC2 has a variety of instance sizes that you can launch your AMI s on. They vary in storage, memory, priority, processing power, and most namely, price. m1.small and c1.medium instance sizes are 32-bit and can only run 32-bit AMI s. All remaining instance sizes are 64-bit. Several studies have shown that the c1.medium instance size is by far the best bang for your buck. It is strongly recommended that you use this size for moderate traffic LAMP web sites. Page 6 of 19

Launching and Connecting to Your EC2 Instance Starting Your EC2 Instance You are now ready to launch your EC2 instance. After selecting the AMI (ami- 48aa4921), click the Launch button. Fill in the form as shown in the image below. Most importantly, select the mcn09 key pair and mcn09 security group that you have already configured. In this example, we are launching a single m1.small instance. In the EC2 console, click the Instances link in the Navigation section. Here you can monitor the progress of your instance. Proceed once the status says, running. In this example, our launched instance has the identifier, i-58d96330. Page 7 of 19

Connecting to Your EC2 instance First you must get the public DNS name assigned to your new instance. You can find this in the instance details of the EC2 Console. In this example, our public DNS is ec2-67-202-9-72.compute-1.amazonaws.com. If you are on a Linux or OSX machine, you can now SSH to your machine using the following command in a terminal. If you are using Windows, follow the instructions for preparing your SSH key pair for Putty. ssh i mcn09.pem root@ ec2-67-202-9-72.compute-1.amazonaws.com Preparing Your SSH Key Pair for Putty If you are using Windows, you must first convert your key pair file to work with Putty. If you are on a Linux or OSX machine, simply skip this section. From the Start Menu, launch the PuTTYgen program under the PuTTY menu item. Select Load private key from the File menu. Navigate and select the private key pair file you downloaded earlier. In this example, the file name was mcn09.pem. Select Save private key from the File menu. In this example, we name the new file mcn09.ppk. Now you can connect to your running EC2 instance from Putty by providing this key pair file under Connection SSH Auth. Use the username, root. Page 8 of 19

Configuring Your EC2 Instance Installing Prerequisites The AMI you started with have next to nothing installed. You must now install the traditional LAMP stack as well as a few additional tools for Drupal. Now you should run the following two commands to install our LAMP stack and some other useful tools. yum groupinstall "MySQL Database" "Web Server" yum install php-gd xfsprogs phpmyadmin This will install Apache and MySQL as well as the library php-gd to make Drupal happy. phpmyadmin is an extremely useful web interface for administering MySQL. Additionally we install xfsprogs so we can manipulate XFS file systems. All web and database files will be hosted on an Elastic Block Store (EBS) volume that you will attach to your running instance. The XFS file system is an ideal format for this volume since it can be resized in case you need more space later, and it can be frozen which prevents corrupt database files when snapshotting the volume. This will be discussed later in the document. Page 9 of 19

Creating an EBS Volume Storage on EC2 instances is considered ephemeral, meaning that it goes away when your EC2 instance goes away. If for some reason your instance crashed, you would lose all of your web and database files. To solve this problem, Amazon offers virtual block devices you can attach to your running EC2 instances. Elastic Block Store (EBS) volumes can range from 1GB to 1TB in size. These volumes are separate from your instance. They remain persistent if your instance is terminated. You can detach an EBS volume from one instance and attach it to another. Also, you can attach multiple EBS volumes to one instance and even setup RAID configurations amongst those volumes. You must first note which availability zone your instance is running in. EBS volumes can only be attached to instances running in the same zone. You can find this in the instance details of the EC2 Console. In this example, our instance is running in zone useast-1a. In the EC2 Console, click on Volumes in the Navigation section. Click the Create Volume button to start the creation process. Make the EBS volume a reasonable size and in the same zone as your EC2 instance. Leave Snapshot set to No Snapshot. Page 10 of 19

Attaching Your EBS Volume Once the EBS volume is finished creating you will attach it to your running EC2 instance. In the EC2 Console, make sure you are still in the Volumes section. Click the newly created EBS volume (vol-f5c6399c in this example) then click the Attach Volume button. Select the EC2 instance you just launched and the device location you want the volume to be mounted. In this example, we select our instance, i-58d96330, and the default device location, /dev/sdf. Formatting and Mounting Your EBS Volume You will now need to format and mount your EBS volume on your EC2 instance. In this example, you are mounting the volume at /ebsvol. We will also add an entry to /etc/fstab so the volume will be automatically mounted in the event of a reboot. Run the following commands in a terminal connected to your instance. mkdir /ebsvol mkfs.xfs /dev/sdf echo "/dev/sdf /ebsvol xfs noatime 0 0" >> /etc/fstab mount /ebsvol/ Page 11 of 19

Configuring MySQL The MySQL database and log files will reside on the EBS volume. You must now configure MySQL to know where to host these files. First, create the destination directories on the EBS volume. mkdir -p /ebsvol/mysql/lib/mysql mkdir -p /ebsvol/mysql/log/mysql Edit the file, /etc/my.cnf, to contain the following. [mysqld] #datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock user=mysql # Default to using old password format for compatibility with mysql 3.x # clients (those using the mysqlclient10 compatibility package). old_passwords=1 innodb_file_per_table datadir = /ebsvol/mysql/lib/mysql #log_slow_queries = /ebsvol/mysql/log/mysql/mysql-slow.log #long_query_time = 10 max_connections = 250 max_user_connections = 250 key_buffer = 36M myisam_sort_buffer_size = 64M join_buffer_size = 2M read_buffer_size = 2M sort_buffer_size = 3M table_cache = 1024 thread_cache_size = 286 interactive_timeout = 25 wait_timeout = 1800 connect_timeout = 10 max_allowed_packet = 1M max_connect_errors = 999999 query_cache_limit = 1M query_cache_size = 16M query_cache_type = 1 tmp_table_size = 16M [mysqld_safe] log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pid Configure MySQL to Start at Boot chkconfig --level 345 mysqld on Start the MySQL Service service mysqld start Set the MySQL root Password mysqladmin -u root password 'mcn09demo' Page 12 of 19

Configuring Apache The Apache web and log files will also reside on the EBS volume. Create the destination directories for these files. mkdir -p /ebsvol/apache/www mkdir -p /ebsvol/apache/log You should now create a virtual host entry for your Drupal web site. Create and edit an apache configuration file (/etc/httpd/conf.d/mcn09.conf in this example). Replace the text _MCN09_ with your custom values. NameVirtualHost *:80 <VirtualHost *:80> #ServerAdmin _MCN09_ DocumentRoot /ebsvol/apache/www #ServerName www._mcn09_.com #ServerAlias _MCN09_.com ErrorLog /ebsvol/apache/log/error.log TransferLog /ebsvol/apache/log/access.log <Directory /ebsvol/apache/www> AllowOverride All </Directory> </VirtualHost> Configure Apache to Start at Boot chkconfig --level 345 httpd on Start the Apache Service service httpd start You can now visit your EC2 instance in a web browser using the public DNS name. In this example, http://ec2-67-202-9-72.compute-1.amazonaws.com/. You will see the Fedora test page for Apache. Page 13 of 19

Installing and Configuring Drupal You will now download and extract the latest Drupal release into the Apache web server directory on the EBS volume. You will also make the apache user the owner of all the web files, and copy Drupal s sample settings. cd /ebsvol/apache/www/ wget http://ftp.drupal.org/files/projects/drupal-6.14.tar.gz tar -xzf drupal-6.14.tar.gz mv drupal-6.14/{*,.htaccess}./ rm -rf drupal-6.14* cp sites/default/default.settings.php sites/default/settings.php chown -R apache:apache. Create the destination Drupal database. In this example, the database is called mcn09db. mysql -uroot -pmcn09demo -e "create database mcn09db" Now visit your EC2 instance in a browser again and follow the typical Drupal installation process. Page 14 of 19

Appendix Configuring CRON for Drupal Edit the crontab file for the apache user. crontab u apache e Add the following line to the crontab replacing the URL with the name of your website. This will cause Drupal s cron to run at 15 and 45 minutes past each hour. 15,45 * * * * /usr/bin/wget -O - -q http://www.yoursite.org/cron.php Page 15 of 19

Assigning a Static IP to Your EC2 Instance Static IP s in EC2 are called Elastic IP s. You can request Elastic IP s for free and only pay if you do not use it. In the EC2 Console, select Elastic IPs from the Navigation section. Click the Allocate New Address button. Select the new Elastic IP you allocated and then click the Associate button. Select the EC2 instance that you would like to assign the static IP to. That s it. Usually within a few seconds your EC2 instance will have the new IP address assigned. You can now configure your DNS settings to point a domain name to the IP address. Page 16 of 19

Snapshotting Your EBS As one backup measure, you should periodically snapshot your EBS volumes. You can create new EBS volumes using these snapshots. In a terminal, run the following commands to lock MySQL and freeze the volume s file system. mysql -uroot pmcn09demo flush tables with read lock; system xfs_freeze -f /ebsvol In the EC2 Console, select Volumes in the Navigation section. Select the volume that you would like to snapshot, and then click the Create Snapshot button. Select Snapshots in the Navigation section. You must wait for the status of your new snapshot to say, completed. Once complete return to the terminal, unfreeze the file system, and unlock the MySQL table. system xfs_freeze -u /ebsvol unlock tables; exit Note: Your website will be unresponsive during this process. Page 17 of 19

Bundling Your EC2 Instance into an AMI All this work would be lost if your instance crashed for some reason. As a way to save your work, you can bundle your instance into a new AMI. You can then launch this AMI as many times as you would like to have an EC2 instance that is in the exact same configuration as when you created the AMI. Your AWS account has X.509 certificate files available. You must copy these to your EC2 instance in order to bundle the instance. You can find these files from the Amazon Web Services web site, under Home Your Account Security Credentials. The file names should be something like, cert-xxxxxx.pem and pk-xxxxxx.pem. Copy these files to the /mnt/ directory your EC2 instance. For example, run this command on your local machine. scp -i mcn09.pem cert-xxxxxx.pem pk-xxxxxx.pem \ root@ec2-174-129-38-75.compute-1.amazonaws.com:/mnt/ You now need to run the AWS bundle command in a terminal on your EC2 instance. ec2-bundle-vol -e /ebsvol \ -p mcn09-ami.img \ -d /mnt \ -r i386 \ -s 10240 \ -c /mnt/cert-xxxxxx.pem \ -k /mnt/pk-xxxxxx.pem \ -u <Your AWS User ID> Parameters explained: -e Exclude the /ebsvol from your AMI. This was saved in the EBS snapshot. -p The name of the output AMI file. -d The directory to output the bundled AMI. -r This is a 32-bit AMI. (x86_64 for 64-bit AMI s) -s The size of the AMI in KB. (10240 == 10GB) -c The path to the public X.509 cert file. -k The path to the private X.509 cert file. -u Your 12 digit AWS user ID/account #. Include dashes (xxxx-xxxx-xxxx). Once the bundling process is complete, run the following command to upload your AMI image. ec2-upload-bundle -m /mnt/mcn09-ami.img.manifest.xml \ -b <Bucket Name> \ -a <AWS Access Key> \ -s <AWS Secret Key> Parameters explained: -m The name of the manifest file outputted from the bundle process. -b The S3 bucket in which to upload the files. -a Your AWS access key. -s Your AWS secret key. Page 18 of 19

Finally you must register your AMI file. In the EC2 Console, select AMIs from the Navigation section. Click the Register New AMI button. Enter the S3 path to your manifest file in the form of <s3bucket>/<manifest>. For example, if you uploaded your AMI files to a bucket, mcn09-ami-bucket, and the manifest file was named, mcn09- ami.img.manifest.xml, you would enter, mcn09-ami-bucket/mcn09-ami.img.manifest.xml. You should now see this new AMI in the list of available AMI s that you can launch. Page 19 of 19