Drupal in the Cloud. by Azhan Founder/Director S & A Solutions

Transcription

1 by Azhan Founder/Director S & A Solutions

2 > Drupal and S & A Solutions S & A Solutions who? doing it with Drupal since 2007 Over 70 projects in 5 years More than 20 clients 99% Drupal projects We love Drupal (obviously!)

3 > Case Study 1 Content publishing of stories from Asia Launched in 2010 using Amazon EC2+RDS Drupal 6, on shared hosting during development Despite performance tuning efforts, still slow pages "Ok S & A, so what do we do now?"

4 > Case Study 1 S & A commissioned to do a site performance evaluation We need to Find out what is causing the slow pages Find out what server size is suitable Decide between single tier or multi-tier Provide a solution for short term/future scalability Prove with benchmarking test results

5 > Case Study 1 Drupal and other optimizations done: Turned on Boost module, CSS/Javascript aggregation Optimized images, custom module SQL queries Removed references to missing files/images Results: Reduced page loading times from 8 secs to 5 secs average ~ 30-40% improvement Apparent network latency stays at 3-4 secs > Hints on bottleneck in the network, web server, database

6 > Case Study 1 Network test: wget download test >> 132kb/sec (good) ping latency hovering ms (normal for SG to USA) Network no longer the suspect Apache ab k -n100 -c1 >> 2-5 secs per page load (SLOW!) 70% content errors when ramped up to n=100, c=10 Final suspects: Web server, MySQL needs more CPU and/or memory

7 > Case Study 1 Other opportunities in improving performance Serve static files using a CDN (> 1GB of static files) Load balanced web servers Memcached / Varnish Reverse proxy eg. Nginx Tune & recompile Apache/MySQL for performance Allocate more resources to Apache and MySQL (easy)

8 > Case Study 1 Other opportunities in improving performance Serve static files using a CDN (> 1GB of static files) Load balanced web servers Memcached / Varnish Reverse proxy eg. Nginx Tune & recompile Apache/MySQL for performance Allocate more resources to Apache and MySQL (easy)

9 > Case Study 1 TheAsiaMag asked for Amazon web services US West Virginia availability zone mostly worldwide audience 4 types of server configurations were tried involves a Load balancer, EC2 with EBS, RDS

10 > Case Study st run: Load balancer, 2x EC2 micro instances, small RDS 2 nd run: Load balancer, 1x LARGE EC2 instance, small RDS 3 rd run: Load balancer, 1x small EC2 instance, small RDS 4 th run: Load balancer, 1x medium EC2 instance, small RDS Benchmarking tools used: Apache ab Drupal Devel module, MySQL slow logs Firebug

11 > Case Study 1 RESULTS of 1 st run: Apache ab -n c10 Load balancer, 2x EC2 micro instances, small RDS EC2 CPU 100% EC2 RAM 100% (no swap) RDS CPU 100% Apache server dies at a random time within a day and could not revive itself

12 > Case Study 1 RESULTS of 2 nd run: Apache ab -n c10 Single LARGE EC2 instance, small RDS 10,000 requests completed in under 20mins EC2 CPU 5-10% EC2 RAM 30-50% RDS CPU <5% Impressive, but overkill. And expensive!

13 > Case Study 1 RESULTS of 3 rd run: Apache ab -n c10 Single small EC2 instance, small RDS 10,000 requests completed in 1hr EC2 CPU 15-20% EC2 RAM 85% RDS CPU 50% Performance is just OK, but cost is within budget! No need for 4 th run.

14 > Case Study 2 Property listing portal, launched in 2010 Drupal 5, dedicated server hosting In slow pages, occasional server deaths S & A, what do we do?

15 > Case Study 2 The problem Just properties 10k page views per month Very complex search+filters using native Drupal search Real time statistics killing the site Watchdog?!!! First aid Identified slow queries, MySQL always using too much RAM Had to kill watchdog Plenty of opportunity for code optimization, but have to wait Hardware upgrade to rescue the site first

16 > Case Study 2 Current hosting at a leading US-based reputable company Amazon web services suggested as an alternative

17 > Scalability challenges Code, database, OS optimizations Single, monster server vs multi-tier architecture How to handle spikes in traffic/load Security, reliability Cost for all the fancy setup Time to do all that?! without breaking the bank! > I know we could do all that with Amazon web services

18 > Case Study 2 Proposed architecture on Amazon web services Load balancer 2x small EC2 for web server 1x small EC2 for hosting static files Shared by all web servers via NFS 1x LARGE RDS for MySQL database

19 > Case Study 2 Final architecture on Amazon web services Load balancer (later, perhaps) Single LARGE EC2 for web server + MySQL Keep it simple! Building the business case for the move to AWS: Benchmarking the AWS setup vs the current site AWS setup has identical CPU, double RAM

20 > Case Study 2 Amazon setup vs dedicated server Current Amazon web services CPU Dual core AMD 64bit CPU 4 ECUs (Intel Xeon dual core equivalent) OS 64bit RHEL 4 64bit Ubuntu RAM 4GB 7.5GB Location USA USA (West Virginia) Average Cost USD 300 per month USD 200 per month Upgrading current to 8GB, RHEL will cost USD700!

21 > Case Study 2 Benchmarking test results comparison No of requests per second (more is better) Current AWS 0 Apache ab requests per sec Benchmarking Tests

22 > Case Study 2 Benchmarking test results comparison Time in secs Current AWS 2 0 Apache ab timing per request Benchmarking Tests

23 > Case Study 2 Benchmarking test results comparison Time in ms Current AWS Devel PHP timings on Homepage Benchmarking Tests

24 > Case Study 2 Benchmarking test results comparison Time in secs Current AWS Devel PHP timings on Query page Benchmarking Tests

25 > Case Study 2 Benchmarking test results comparison Time in secs Current AWS 1 0 Firebug network latency Benchmarking Tests

26 > Case Study 2 Benchmarking test results comparison Time in secs Current AWS 10 0 Firebug full page loading time Benchmarking Tests

27 > Case Study 2 Client is convinced! (Yeay!!)

28 > Why Amazon web services? No to AWS Could easily have been other hosting providers Other cloud vendors can address scalability issues Getting over the Amazonian barriers AWS may be viewed as difficult by some

29 > Why Amazon web services? Yes, AWS! A platform for benchmarking, server sizing Easy and quick to create/kill servers, run tests Actually a safe, cheap and controlled means of hosting your site or application Easily spawn additional servers to cater to high periods eg. A product launch lasting 4 weeks terminate the additional servers after site traffic has died down You can get really creative designing your architecture

30 > Drupal web load balancing Amazon Load Balancer Normally a complex setup with traditional hosting, the AWS Console makes it so easy to set this up in minutes

31 > Drupal and memcached Amazon ElastiCache Normally a complex setup with traditional hosting the AWS Console makes it so easy to set this up in minutes if paired with the Drupal memcache module

32 > Drupal and CDN Amazon Cloudfront or S3

33 > Drupal and high availability Amazon multi-availability Zones for EC2 and RDS

34 > Drupal and MySQL Running MySQL can be done using one of the following options: MySQL on Amazon EC2 MySQL on Amazon RDS Multi-AZ RDS

35 > Drupal & Amazon web services Amazon CloudFormation Templates to quickly deploy a Drupal stack into the cloud

36 > Drupal & Amazon web services It works!

37 > Questions? Website: Thank you!