Tipping The Scale Tips, Tools, and Techniques For Building Scalable. Steve French Senior Software Engineer digg.com

Size: px

Start display at page:

Download "Tipping The Scale Tips, Tools, and Techniques For Building Scalable. Steve French Senior Software Engineer digg.com"

Jeremy Lewis
8 years ago
Views:

1 Tipping The Scale Tips, Tools, and Techniques For Building Scalable Steve French Senior Software Engineer digg.com

2 First Thing s First... The Stack Server OS Linux, MacOS X, UNIX, Windows Web Server apache, iis, nginx, lighttpd, tornado Programming Language php, python, ruby, java, asp, jsp Datastore mysql, postgresql, oracle, db2, mssql cassandra, mongodb, couchdb What is Digg.com Linux + Apache PHP, Python, Java MySQL, Cassandra

3 The Stack What Else? Load Balancer Barracuda, Cisco, Coyote, Caching memcache $userid = 211; //build key and set timeout for memcache use $key = user. $userid; $timeout = 60 * 60 * 24; //try to find in memcache $user = $memcache->get($key); //if not found, fetch and store in cache if($user === null) { $user = fetch_user( select * from users where userid =?, $userid); $memcache->store($key, $user, $timeout); } //use the object echo Hello. $user->first_name. \n ;

4 Challenges? Request/Connection Bound Your site can t respond to all connections being requested. Webserver Load Your webservers are doing too much work for active processes. Datastore Load You can t read/write data fast enough Unable to Bend Space/Time Self explanatory!

5 Request/Connection Bound Client Side Request Caching HTTP/ OK Date: Thu, 27 Sep :19:41 GMT Server: Apache/1.3.3 (Unix) Cache-Control: max-age=3600, must-revalidate Expires: Fri, 27 Sep :19:41 GMT Last-Modified: Mon, 24 Sep :28:12 GMT Server Side Request Caching Loadbalancer / Caching Proxy Server

6 Request/Connection Bound Minimize Requests Minimize the number of CSS/JS/Images/AJAX that you call on a page. Has the additional benefit that your pages will render much faster. MXHR - Multipart XMLHttpRequest Multiple resources can be Multipart encoded into a single request -- Content-Type: image/gif <snip> R0lGODlhIAAgAOYAAJGRkWlpaf///+vr66+vr/r6+vb29vz8/ Pn5+fPz8+7u7vHx8W1tbe3t7f39/ejo6Orq6ufn59fX1/v7+/Ly8u/v7/ X19ff399zc3PT09OXl5f7+/ ubm5tpt08lcwsrkynr0dlu7u4cagozs7onp6acgonls0tnz2bw1twxsbpj 4+OTk5NTU1IyMjNXV1aGhoWpqatbW1piYmKOjo7q6uqysrLGxsc3NzeDg </snip> -- Content-Type: text/plain This is some text to paste into your application -- Content-Type: text/css.foo { border: 1px solid black; } --

7 Request/Connection Bound Content Delivery Networks (CDN) You can push static content (images/css/javascript/video) to external delivery networks which take the request load off of your servers Add More Webservers Technically this is true, but isn t generally going to be the best option. Tune Existing Webservers If your server has enough resources you can experiment with increasing the maxclients that your webserver will allow.

8 Challenges? Request/Connection Bound Your site can t respond to all connections being requested. Webserver Load Your webservers are doing too much work for active processes. Datastore Load You can t read/write data fast enough Unable to Bend Space/Time Self explanatory!

9 Webserver Load Optimize Code and Code Use Cache Code In PHP - Alternative PHP Cache (APC) Cache Templates Cache Data Outputs Profile Code In PHP - xdebug, XHProf

10 Webserver Load Scale the System Architecture Scale Up This is really easy to do, but it really only really makes sense if hardware advances outpace your growth. Scale Out Many servers behind load balancer(s). Some care is needed to make sure that your code does not require machine affinity. You have to do this to run out of multiple data centers.

11 Challenges? Request/Connection Bound Your site can t respond to all connections being requested. Webserver Load Your webservers are doing too much work for active processes. Datastore Load You can t read/write data fast enough Unable to Bend Space/Time Self explanatory!

12 Data Access Load Code Changes Cache Data! Memcache - Test: Fetch users from mysql and memcached Caching Strategy Cache On Access Optimistic Cache Priming

13 Data Access Load Code Changes Optimize Queries Explain Queries Add Indices Sometimes many simple queries are better than a single complex one. select * from users where userid in (1000, 3002, 53, , 999,...); VS for($i=1, $i <= 500; $i++) select * from users where userid = $i

14 Data Access Load Scale the System Architecture - Read Traffic Scaling Out Read Slaves Vertical Partitioning Horizontal Partitioning Data Denormalization

15 Data Access Load Scaling Out Read Slaves

16 Data Access Load Vertical Partitions create table users( id bigint not null auto_increment, username char(20), password char(50), secretquestion char(100), secretanswer char(100), favoritecolor char(25), website text, active enum( Y, N ) ); create table users( id bigint not null autoincrement, username char(20), password char(50), active enum( Y, N ) ); create table userdetails( userid bigint not null, secretquestion char(100), secretanswer char(100), favoritecolor char(25), website text );

17 Data Access Load Horizontal Partitions

18 Data Access Load Data Denormalization Benefits Data is replicated to tables that need it. Less JOIN operations Drawbacks Updating values takes additional operations, so this technique is best used on rarely-changing data. create table users( id bigint not null autoincrement, username varchar(20), password varchar(50), secretquestion varchar(100), secretanswer varchar(100), active enum( Y, N ) ); create table stories( id bigint not null autoincrement, title varchar(100), description varchar(255), username varchar(20) ); create table comments( userid bigint not null, comment_body varchar(255), username varchar(20) );

19 Data Access Load Scale the System Architecture - Write Traffic Query-Specific Server Pools Scale Out Writable Servers Horizontal Partitioning

20 Data Access Load Query-Specific Server Pools

21 Data Access Load Scale Out Writable Servers

22 Data Access Load Horizontal Partitions

23 OK, Scaling Writes Sucks... What Else?

24 Data Access Load Key/Value Stores Benefits: Faster Suited to horizontal scaling Redundancy Read/Write Tradeoffs Drawbacks: Ad-hoc queries are almost impossible.

25 Data Access Load Cassandra Data Model - Example AddressBook = { // ColumnFamily of type Super phatduckk: { // all the address book entries (supercolumns) for phatduckk John: {street: "Howard street", zip: "94404", city: "San Francisco", state: "CA"}, Kim: {street: "X street", zip: "87876", city: "Sticks", state: "VA"}, Tod: {street: "Jerry street", zip: "54556", city: "Cartoon", state: "CO"}, Bob: {street: "Q Blvd", zip: "24252", city: "Nowhere", state: "MN"}, } }, ieure: { // all the address book entries for ieure Joey: {street: "A ave", zip: "55485", city: "Elko", state: "NV"}, William: {street: "Armpit Dr", zip: "93301", city: "Bakersfield", state: "CA"}, Bob: {street: "Q Blvd", zip: "24252", city: "Nowhere", state: "MN"}, },

26 Challenges? Request/Connection Bound Your site can t respond to all connections being requested. Webserver Load Your webservers are doing too much work for active processes. Datastore Load You can t read/write data fast enough Unable to Bend Space/Time Self explanatory!

27 Bending Space and Time Real Time vs Near Time Asynchronous Job Queues Gearman - RabbitMQ - JMS - Offline Processing Cron - man cron Hadoop -

28 Bending Space and Time Gearman - Example: $users = fetch_all_users(); foreach($users as $user) { //do an expensive process } $items = fetch_all_items(); foreach($items as $item) { //do an expensive process } gearman_exec(fetch_all_users, $userkey); gearman_exec(fetch_all_items, $itemskey); while(!gearman_fetch($userskey)) sleep(1); foreach($users as $user) { //do an expensive process } while(!$items = gearman_fetch($itemskey)) sleep(1); foreach($items as $item) { //do an expensive process }

29 Questions, Comments, Flames?

30 Thank You! twitter: twitter.com/frenchs web: sfrench.com

Cache All The Things

Cache All The Things About Me Mike Bell Drupal Developer @mikebell_ http://drupal.org/user/189605 Exactly what things? erm... everything! No really... Frontend: - HTML - CSS - Images - Javascript Backend: