Growing in web environment at XING

Transcription

1 Growing in web environment at XING Jens Muecke Nuernberg, 06/24/2010

2 Content 1. Introduction into XING 2. Why does performance matter? 3. Solving growing pains 4. Why Open Source? Jens Muecke, Nuernberg, 06/24/2010 2

3 Introduction into XING

4 01 Introduction into XING Leading Business Social Network in Europe > 9 million members > premium members Highly interactive community > groups > events per year 50 releases per year ~100 employes at engineering Jens Muecke, Nuernberg, 06/24/2010 4

5 02 Introduction into XING infrastructure: scale out not scale up > 300 servers > 30% MySQL servers two data centers full redundancy Jens Muecke, Nuernberg, 06/24/2010 5

6 03 Introduction into XING In peek times, >50 thoused logged in users Users logged in Germany vs. Serbia Jens Muecke, Nuernberg, 06/24/2010 6

7 04 Introduction into XING LAMP Linux Apache MySQL Perl Jens Muecke, Nuernberg, 06/24/2010 7

8 05 Introduction into XING The true is more like: Linux memcached Apache HTTPD Gearman MySQL nagios Perl Apache Shindig Ruby on Rails prototype.js Lighttpd Git FAI munin CPAN Modules... Jens Muecke, Nuernberg, 06/24/2010 8

9 Why does performance matter?

10 01 Why does performance matter? Delays under half a second impact business metrics The cost of delay increases over time and persists Jens Muecke, Nuernberg, 06/24/

11 02 Why does performance matter? Jens Muecke, Nuernberg, 06/24/

12 03 Why does performance matter? Jens Muecke, Nuernberg, 06/24/

13 04 Why does performance matter? Full Story at Conclusion High performance brings more then user satisfaction Jens Muecke, Nuernberg, 06/24/

14 Solving growing pains

15 01 Solving growing pains Cheap standard hardware Identify bottleneck by constantly monitoring all systems and solve them itteratively How to scale out? Jens Muecke, Nuernberg, 06/24/

16 02 Solving growing pains At the beginning internet application server Jens Muecke, Nuernberg, 06/24/

17 03 Solving growing pains Next steps... SSL/LoadBalancer internet database application servers Jens Muecke, Nuernberg, 06/24/

18 04 Solving growing pains step by step add new servers for one job memcached static content cron job Jens Muecke, Nuernberg, 06/24/

19 05 Solving growing pains vertical scaling merge different Tables on User ID First name Last name 1 Michael Otto 2 Norbert Schuler different servers Message ID User ID Subject 1 2 Hi 2 1 Re: Hi Jens Muecke, Nuernberg, 06/24/

20 06 Solving growing pains vertical scaling our $CONTEXT_MAIN = 1; our $CONTEXT_MSG = 10; # query user my $sql = 'select first_name from user limit 20'; = db::query_hashrefs($context_main, $sql); # query message my $sql = 'select subject from message limit 20'; = db::query_hashrefs($context_msg, $sql); Jens Muecke, Nuernberg, 06/24/

21 07 Solving growing pains horizontal scaling seperate data User ID First name Last name 1 Michael Otto 2 Norbert Schuler in different clusters User ID First name Last name 3 Hanno Zulla 4 Heiner Juergensen Jens Muecke, Nuernberg, 06/24/

22 08 Solving growing pains vertical scaling our $CONTEXT_USR1 = 1; our $CONTEXT_USR2 = 2; # query user my $sql = 'select first_name from user where id=1'; = db::query_hashrefs($context_usr1, $sql); one table required to store clusterid Jens Muecke, Nuernberg, 06/24/

23 09 Solving growing pains separate application read write slave/lvs master replication Jens Muecke, Nuernberg, 06/24/

24 10 Solving growing pains separate read and write our $CONTEXT_MAIN_READ = 1; our $CONTEXT_MAIN_WRITE = 2; # query user my $sql = 'select first_name from user limit 20'; = db::query_hashrefs($context_main_read, $sql); # update my $sql = 'update user set =...'; db::query($context_main_write, $sql); Jens Muecke, Nuernberg, 06/24/

25 11 Solving growing pains Delivering of user images <4kb of size often per page Usage of SSD instead of spinning disks decrease seeking time from 3.5 msec to 0.2 msec Jens Muecke, Nuernberg, 06/24/

26 12 Solving growing pains Jens Muecke, Nuernberg, 06/24/

27 13 Solving growing pains Jens Muecke, Nuernberg, 06/24/

28 14 Solving growing pains Jens Muecke, Nuernberg, 06/24/

29 15 Solving growing pains Viewport load content, when user is scrolling down Jens Muecke, Nuernberg, 06/24/

30 16 Solving growing pains - 50%!!! Jens Muecke, Nuernberg, 06/24/

31 17 Solving growing pains Best Practices for Speeding Up Your Web Site Cache Control Header GZip Components CSS at top script at bottom... Jens Muecke, Nuernberg, 06/24/

32 18 Solving growing pains There are so many more but it's all about monitoring and fixing it (devide & conquer) FireBug and Speed Tracer munin Jens Muecke, Nuernberg, 06/24/

33 19 Solving growing pains Jens Muecke, Nuernberg, 06/24/

34 Why Open Source?

35 01 Why Open Source? Trust and security Possibility to understand Trace bottlenecks Community watch the code Jens Muecke, Nuernberg, 06/24/

36 02 Why Open Source? Development and feedback Active community People to hire Conferences to share knowledge More motivation Jens Muecke, Nuernberg, 06/24/

37 02 Why Open Source? Save, and return control No license cost You can buy support Use open standards Jens Muecke, Nuernberg, 06/24/

38 Thank you for your kind attention! Presentation Titel Author City, 24/06/10 38