Handling Flash Crowds From Your Garage

Transcription

1 Handling Flash Crowds From Your Garage Jeremy Elson and Jon Howell Microsoft Research USENIX ATC 2008

2 Scaling For Many Users

3 Scaling For Many Users... quickly

4 Scaling For Many Users... quickly... on a budget

5 Enabled by Utility Computing Pay proportional to use Minimal Starting Cost (often free!) Quickly scalable

6 Examples of Utility Computing

7 Storage Data Networks

8 Storage Data Networks SDNs store static content in large clusters Expose HTTP interface

9 Storage Data Networks SDNs store static content in large clusters Expose HTTP interface Different from CDNs: No startup or recurring cost Fee based on usage

10 Storage Data Networks SDNs store static content in large clusters Expose HTTP interface Different from CDNs: No startup or recurring cost Fee based on usage Examples: S3, Nirvanix

11 Storage Data Networks SDNs store static content in large clusters Expose HTTP interface Different from CDNs: No startup or recurring cost Fee based on usage Examples: S3, Nirvanix

12 Compute Clouds Allow developers to configure and save a VM image Instantiate images as needed Fee based on compute units Examples: EC2, Flexiscale

13 DNS Outsourcing Used to host authoritative DNS servers Many replicas provide reliability and performance Examples: UltraDNS and DynDNS

14 Outline Introduction Examples of Utility Computing Scaling Architectures Analyzing the Design Space Application Experiences

15 Scaling Architectures

16 Using the bare SDN Works only if content is predominantly static

17 HTTP Redirection Uses a dedicated redirection server Redirects to server S on first access Redirector All future interactions with server S

18 HTTP Redirection Uses a dedicated redirection server Redirects to server S on first access Redirector All future interactions with server S

19 L4/L7 Load Balancing Spread incoming traffic across servers Spreading based on source TCP port or HTTP session Free/utility load balancers available: HAProxy, Linux Virtual Server, Flexiscale, EC2 Load balancing

20 DNS Load Balancing Advertise many IP addresses for single hostname Use DNS to spread load across servers Example: RightScale, or build your own using UltraDNS Relies on good DNS resolver behavior

21 DNS Load Balancing

22 DNS Load Balancing Advertise many IP addresses for single hostname Use DNS to spread load across servers

23 DNS Load Balancing Advertise many IP addresses for single hostname Use DNS to spread load across servers Example: RightScale, or build your own using UltraDNS

24 DNS Load Balancing Advertise many IP addresses for single hostname Use DNS to spread load across servers Example: RightScale, or build your own using UltraDNS Relies on good DNS resolver behavior

25 Outline Introduction Examples of Utility Computing Scaling Architectures Analyzing the Design Space Application Experiences

26 Analyzing the Design Space Application Scope Scale Limitation Scale-up Time Fault Tolerance

27 Application Scope SDN: Only HTTP

28 Application Scope SDN: Only HTTP HTTP Redirection: Only HTTP

29 Application Scope SDN: Only HTTP HTTP Redirection: Only HTTP L7 Load Balancing: All apps with the same L7 protocol

30 Application Scope SDN: Only HTTP HTTP Redirection: Only HTTP L7 Load Balancing: All apps with the same L7 protocol L4/DNS Load Balancing: All apps

31 Scale Limitations SDNs: Limited only by provider s SLA

32 Scale Limitations SDNs: Limited only by provider s SLA HTTP Redirection: Very scalable if client sessions long

33 Scale Limitations SDNs: Limited only by provider s SLA HTTP Redirection: Very scalable if client sessions long L4/L7 Load Balancing: Scales if app is not communication-bound

34 Scale Limitations SDNs: Limited only by provider s SLA HTTP Redirection: Very scalable if client sessions long L4/L7 Load Balancing: Scales if app is not communication-bound DNS Load Balancing: Scales to thousands of servers

35 Scale Limitations SDNs: Limited only by provider s SLA HTTP Redirection: Very scalable if client sessions long L4/L7 Load Balancing: Scales if app is not communication-bound DNS Load Balancing: Scales to thousands of servers

36 Scale-Up Times

37 Scale-Up Times SDN: Scales up very quickly

38 Scale-Up Times SDN: Scales up very quickly L4/L7 Load Balancing+HTTP Redirection ~1-minute scale-up time

39 Scale-Up Times SDN: Scales up very quickly L4/L7 Load Balancing+HTTP Redirection ~1-minute scale-up time DNS Load Balancing Scale-up time hampered by DNS caching

40 Front-End Failures SDN: Failure unlikely

41 Front-End Failures SDN: Failure unlikely L4/L7 Load Balancers Single load-balancer fails miserably Need hot spares or DNS load-balanced front-ends

42 Front-End Failures SDN: Failure unlikely L4/L7 Load Balancers Single load-balancer fails miserably Need hot spares or DNS load-balanced front-ends HTTP Redirectors Existing sessions don t fail

43 Front-End Failures SDN: Failure unlikely L4/L7 Load Balancers Single load-balancer fails miserably Need hot spares or DNS load-balanced front-ends HTTP Redirectors Existing sessions don t fail DNS Load Balancers With replicated DNS, failover is very quick

44 Latency when DNS Servers Fail

45 Back-end Failures SDN: Not our problem!

46 Back-end Failures SDN: Not our problem! HTTP Redirector+L4/L7 Load Balancer Transient failures at worst

47 Back-end Failures SDN: Not our problem! HTTP Redirector+L4/L7 Load Balancer Transient failures at worst DNS Load Balancer Bad! Several minutes of delay...

48 Application Experiences

49 MapCruncher Tool to author AJAX-style interactive maps Server only retrieved from a 25 GB set of map images S3 solved the flash crowd problem ~$200 total cost

50 Asirra CAPTCHA service Images stored in S3 Based on EC2 and Python DNS-based load balancing Handled 75,000 requests on first day, including a DOS attack Starting more servers solved the DOS problem

51 InkblotPassword.com Implemented in Python and EC2 No code optimization Depend on starting new servers for scale ~10,000 users registered in a day ~$150 spent to handle Slashdotting

52 Conclusion Utility computing enables cheap handling of flash crowds No best architecture Depends on application workload Enable scaling to many instances instead of optimizing one instance

53