Benchmarking Broadband Internet with Bismark

Transcription

1 Benchmarking Broadband Internet with Bismark Srikanth Sundaresan, Nick Feamster Georgia Tech, Atlanta, USA Walter de Donato, Antonio Pescapé University of Napoli Federico II, Napoli, Italy Renata Teixeira University Pierre et Marie Curie, Paris, France

2 Broadband Connectivity is exploding 200 million Americans have broadband National Broadband plan aims to increase that But, what is current experience with broadband?

3 Talk Outline Questions about broadband performance Bismark: our measurement platform Some initial results Future work

4 Are users really getting what they pay for?

5 Study in France confirms this view Over 20,000 users across major ISPs, geographical regions Latency and throughput study from end hosts

6 Significant performance variation ISP, service plan, geographical location have effect. Gap between plan rate and achieved rates al u mu C Fewer than half of the users achieve 80% of advertised SLA. Why? S. Sundaresan, L. Di Cioccio, N. Feamster, R. Teixeira. Which Factors Affect Home Network Performance? 6

7 How do we study access link performance? Study from outside Dischinger et al (IMC 2008), Netalyzr (IMC 2010) Not continuous, not many per user, no view into home Study from inside Grenouille project Hard to account for device diversity Hard to account for home network

8 We don't really understand home networks How much of user performance is affected by factors outside ISP's control? Home networks have a big impact Network setup multiple diverse devices WiFi effect Competing traffic User behavior

9 Talk Outline Questions about broadband performance Bismark: our measurement platform Some initial results Future work

10 Bismark Deploy programmable gateways in homes Noxbox deployment: about 35 around Atlanta SamKnows deployment: about around the US NoxBox Netgear

11 The design Architecture Overview DB Measurement Server Management Server Internet Router Router Measurement Server

12 Vantage point between the home and the ISP Observes all traffic passing through network Isolate individual factors affecting network performance Effect of home network Wireless Access link

13 Measurements - Active Latency, Jitter and Packet Loss Last mile and end-to-end latency (ping) Upstream and downstream jitter, packet loss (DITG) Important for gaming, VoIP Throughput Upstream and downstream (curl, shaperprobe) Capacity, available bandwidth of access link DNS lookup time and faults Affects user performance

14 Measurements Passive ARP and DHCP logs Nature of home network Aggregate flow statistics Incoming and outgoing (TIE) Understand usage patterns, available bandwidth Wireless stations Access point configurations (airodump-ng) Understand home wireless networks

15 Current Deployment 10 boxes deployed 7 in ATT, 2 in Comcast, 1 ClearWire Most of the deployments within Atlanta All measurements done to server in GaTech

16 Talk Outline Questions about broadband performance Bismark: our measurement platform Some initial results Future work

17 Overview of results Initial look at latency from gateway to: First IP hop in ISP network (last mile) Server in GaTech High variation across homes, ISPs Initial throughput analysis Single threaded HTTP throughput using curl Path capacity using shaperprobe

18 RTT(ms) RTT(ms) Latencies vary within same ISP Baselines Different for 2 ATT customers

19 Access link technology has impact High variation in WiMax and Cable ADSL latencies are more tightly bound RTT(ms) RTT(ms) Comcast Clear

20 Last mile latencies dominate All but 2ms comes from last mile ADSL last mile 8 to 25ms, WiMAX ~ 75ms! End-to-end latency Last mile latency Very high correlation (0.95) to end-to-end latency

21 Dowstream throughput doesn't quite reach maximum limit CDF Plan rate Kbps Single ATT customer Single threaded TCP downloads consistently see about 80% of promised capacity

22 Why not? Single threaded throughput error? Multiple parallel TCP sessions see higher throughput (> 95% for ATT) However, highly disruptive to user Capacity measurements also close to limit We are interested in realistic usage scenarios What does a normal user with normal usage patterns see?

23 Upstream throughput exceeds advertised limits CDF Plan rate Kbps Single ATT customer Throughput Consistently greater for ATT

24 Key findings Latency is important Baseline varies Highly dependent on access link technology Last mile latency dominates, highly correlated to end-to-end latency Downloads are consistently lower than promised rate Uploads are consistently higher

25 Talk Outline Questions about broadband performance Bismark: our measurement platform Some initial results Future work

26 Future Deployment and Measurements Larger deployment for critical amount of data Variety in ISP, service plans, geographical locations Working with SamKnows User study to understand What users expect from their Internet How home networks are set up How can we improve user experience?

27 More questions about performance Does application affect performance? How do different factors affect performance? Latency, ISP, service plan Better statistical tools to analyze current data What would you like to know/measure?

28 Outsourcing Home Network Configuration and Management What prevents untrusted tools from: Sniffing my traffic? Consuming all my bandwidth? Using my printer? Solution: Hold tools accountable to a management policy PlanetLab-like testbed for home networks?

29 Thanks!