ALLION USA PUBLIC WI-FI HOTSPOT COMPETITIVE ANALYSIS WHITE PAPER

Transcription

1 ALLION USA PUBLIC WI-FI HOTSPOT COMPETITIVE ANALYSIS WHITE PAPER Date: 6/21/2013 Rev 1.0 Visit our Web Site at:

2 Introduction Public Wi-Fi hotspots are becoming increasingly common in the U.S. Today, hotspots can be found in many areas including cafes, train stations, airports, restaurants, parks, shopping centers etc. The convenience of having hotspots while on-the-go provides users access to the Internet on their mobile devices via Wi-Fi (laptops, smartphones, tablets) allowing them to browse the web, check their s, and stream audio / video without encountering data usage issues. However, the Internet service provider behind the hotspots can have a large impact to the overall user experience. Allion s competitive analysis performed several tests across 3 major Wi-Fi hotspot service providers in the U.S. (AT&T, Boingo Wireless, Comcast) to evaluate the overall performance of each vendor. The service providers were primarily determined by the amount of hotspots offered + geographic distributions within the country. Select hotspots were sampled for testing in cities that offered hotspots from all service providers within close proximity. Baltimore, MD Boston, MA Philadelphia, PA San Francisco, CA The primary focus of the test was to determine which provider offered the fastest Internet speed. Downlink and uplink throughput measurements were taken across 6 different Wi-Fi client devices via Speedtest.net. NOTE: A table of summary for all datasets is available in the Appendix 2

3 Test Areas / Locations Hotspot Service Providers: As described earlier, data was collected from hotspots distributed by the following major service providers: AT&T Boingo Wireless* Comcast Locations: A mix of cafes, restaurants, retail / small business, and outdoor locations were randomly selected in the downtown region where hotspots were heavily deployed. Tests were repeated in the following cities: Baltimore, MD Boston, MA Philadelphia, PA San Francisco, CA NOTE: It was discovered during testing that the majority of hotspots listed on Boingo Wireless hotspot distribution map included 3 rd party hotspots operated by other vendors. This issue significantly lowered the number of Boingo Wireless hotspots to test with. Hence, data samples from Boingo Wireless were limited for this study. T-Mobile and Verizon hotspots were excluded from this study as the number of available hotspots were significantly less than the providers mentioned above. 3

4 Wi-Fi Client Devices Used As the Wi-Fi performance can greatly vary depending on the client platform, chipset used, OS etc., Allion has selected a mix of devices to use during the test while taking into consideration popular devices currently on the market. Laptops o Acer Aspire P5WE0 (Windows 7 64bit) o Macbook Air (Mac OSX) Mobile Phones o iphone 5 (ios) o Samsung Galaxy SIII (Android) Tablets o ipad 4 (ios) o Google Nexus 10 (Android) 4

5 Data Collected / Test Methodology Data Collected: The following data was collected at each hotspot location: Speedtest.net throughput (Mbps) o Speedtest.net is a widely used network tool that can quickly determine the broadband connection speed. Tests can be performed through web pages or apps depending on the client platform. o Downlink measures the amount of traffic that can be received by the client, and uplink measures the amount of traffic that can be transmitted by the client. Received-Signal-Strength-Indicator or RSSI (dbm) o Indicates the Wi-Fi signal strength from the client perspective. Test Methodology: 1. Connect 1 st device to AP 2. Record RSSI reported by the client device 3. Navigate to Speedtest.net and run test (or open Speedtest app on mobile device) 4. Record downlink throughput 5. Record uplink throughput 6. Repeat steps 1-5 with next device 7. Repeat all of the above 3 times 5

6 Conclusion This study shows performance comparisons of the Wi-Fi hotspots operated by the 3 major service providers in the U.S. Tests were conducted across 6 different Wi-Fi client devices in 4 cities for a total of 2,800+ data samples to determine the provider with the best performance. Below is the total average throughput of all data points collected from each city. As shown in the graph, Wi-Fi hotspots operated by Comcast offered a significantly faster downlink throughput than AT&T or Boingo Wireless. Average Throughput by Provider NOTE: Boingo Wireless hotspots were inaccessible in San Francisco, CA at the time of testing As described in more detail in the following sections, AT&T and Boingo Wireless hotspots appeared to be limited in downlink throughput showing a range of 1-4Mbps and rarely exceeding 5Mbps. On the other hand, Comcast hotspots exhibited a much higher throughput 6

7 frequently exhibiting 10+ Mbps and 20+ Mbps on occasion. Although the uplink throughput was comparable between the providers, Comcast still exhibited up to 3x faster throughput than others. This trend of Comcast outperforming its competitors were observed across all 6 devices in all 4 cities visited. Additional analysis of RSSI vs. Throughput also revealed that Comcast hotspots exhibited a linear trend of increased throughput with higher RSSI, while AT&T & Boingo Wireless hotspots showed limited throughput regardless of the signal strength. This suggests that AT&T & Boingo Wireless hotspots are rate limited to a low throughput (as shown with the flat line) indicating a potential limitation in the service level. Below is the dataset from Philadelphia (similar behavior was also observed in other cities; shown in later sections). Based on the results and analysis performed, Comcast hotspots offered significantly faster throughput speeds than all other providers. The end result of higher throughput will most likely lead to a more pleasant Internet streaming / browsing experience for the users. With the many hotspots offered around the country today, users should look for the fastest service provider to get the best experience on the web. 7

8 Throughput Performance Box plots: Box plots can provide graphical summaries of large datasets in a convenient form. The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of a dataset. The whiskers above and below the box each represent 25% of the data (other 50%), and data points outside of the whiskers are considered outliers. These graphs are useful in interpreting the results from each device to understand how the throughput data was distributed during the study. Summary: The graphs below represent all data points collected for downlink throughput. As shown in the graph, Comcast hotspots significantly outperformed the other 2 providers in throughput speeds for all 6 devices. Although the data showed a wider spread (as indicated with the longer box), Comcast hotspots frequently exhibited 10+ Mbps while the majority of AT&T and Boingo Wireless data samples were limited at 1-4Mbps. This trend in the data was observed across all 4 cities visited during the test suggesting the following: AT&T and Boingo Wireless hotspots appear to be limited in the max achievable throughput (frequently exhibiting 1-4Mbps and rarely exceeding 5Mbps) Comcast hotspots can achieve approximately 10Mbps regularly and 20+ Mbps on occasion Comcast hotspot users will most likely experience a smoother Internet streaming / browsing experience than AT&T or Boingo Wireless hotspots While the throughput values were at a reduced scale, a similar behavior was also observed for uplink throughput (available in Appendix) where Comcast hotspots exhibited up to 3x faster throughput speeds compared to other providers. 8

9 Baltimore, MD (Downlink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 9

10 Philadelphia, PA (Downlink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 10

11 Boston, MA (Downlink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 11

12 San Francisco, CA (Downlink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 12

13 RSSI vs. Throughput In wireless systems, the signal strength of the communication link can greatly impact the throughput performance of a client device. While no Over-The-Air (OTA) environment is the same due to the complexities of the building structure, materials used, and neighbor RF interference, analyzing the throughput performance relative to the Wi-Fi signal condition can provide some insights to the performance of each provider. The graphs above represent all downlink throughput samples from the 6 client devices used during the test (uplink data available in Appendix). As shown above, Comcast hotspots exhibited a linear trend of increased throughput with higher RSSI, while AT&T & Boingo Wireless hotspots showed limited throughput regardless of the signal condition in most cities. This data is in-line with the data presented in the previous section where AT&T and Boingo Wireless hotspots appeared to be limited in the max achievable throughput while Comcast hotspots exhibited a greater range in throughput speeds. 13

14 Other (Google Hotspots) Google has deployed several hotspots for public use in Mountain View, CA. According to the company, the hotspots were primarily deployed to better serve its employees, but has made it accessible to the general public. They are currently limited to the Mountain View area and the company has not yet announced an expansion of coverage. However, Allion has tested several of their hotspots to evaluate their performance. As shown above, Google hotspots exhibited limited throughput speeds comparable to AT&T. While the test was performed in a different geographic area with limited data samples, Comcast hotspots continued to show superior performance outperforming both current and potential future competitors in throughput speeds. 14

15 Appendix Baltimore, MD (Uplink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 15

16 Philadelphia, PA (Uplink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 16

17 Boston, MA (Uplink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 17

18 San Francisco, CA (Uplink) NOTE: The length of the box indicates the variation / spread (50% of data), and the horizontal line crossing the box indicates the median of the dataset Whiskers above and below the box each represent 25% of the data (total of 50%) Data points outside of the whisker are considered outliers 18

19 RSSI vs. Throughput (Uplink) 19

20 Baltimore, MD (Downlink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Boingo Galaxy S3 Comcast Laptop AT&T Laptop Boingo Laptop Comcast MacBook AT&T MacBook Boingo MacBook Comcast Nexus10 AT&T Nexus10 Boingo Nexus10 Comcast ipad4 AT&T ipad4 Boingo ipad4 Comcast iphone5 AT&T iphone5 Boingo iphone5 Comcast Baltimore, MD (Uplink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Boingo Galaxy S3 Comcast Laptop AT&T Laptop Boingo Laptop Comcast MacBook AT&T MacBook Boingo MacBook Comcast Nexus10 AT&T Nexus10 Boingo Nexus10 Comcast ipad4 AT&T ipad4 Boingo ipad4 Comcast iphone5 AT&T iphone5 Boingo iphone5 Comcast

21 Philadelphia, PA (Downlink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Boingo nan Galaxy S3 Comcast Laptop AT&T Laptop Boingo nan Laptop Comcast MacBook AT&T MacBook Boingo nan MacBook Comcast Nexus10 AT&T Nexus10 Boingo nan Nexus10 Comcast ipad4 AT&T ipad4 Boingo nan ipad4 Comcast iphone5 AT&T iphone5 Boingo nan iphone5 Comcast Philadelphia, PA (Uplink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Boingo nan Galaxy S3 Comcast Laptop AT&T Laptop Boingo nan Laptop Comcast MacBook AT&T MacBook Boingo nan MacBook Comcast Nexus10 AT&T Nexus10 Boingo nan Nexus10 Comcast ipad4 AT&T ipad4 Boingo nan ipad4 Comcast iphone5 AT&T iphone5 Boingo nan iphone5 Comcast

22 Boston, MA (Downlink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Boingo Galaxy S3 Comcast Laptop AT&T Laptop Boingo Laptop Comcast MacBook AT&T MacBook Boingo MacBook Comcast Nexus10 AT&T Nexus10 Boingo Nexus10 Comcast ipad4 AT&T ipad4 Boingo ipad4 Comcast iphone5 AT&T iphone5 Boingo iphone5 Comcast Boston, MA (Uplink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Boingo Galaxy S3 Comcast Laptop AT&T Laptop Boingo Laptop Comcast MacBook AT&T MacBook Boingo MacBook Comcast Nexus10 AT&T Nexus10 Boingo Nexus10 Comcast ipad4 AT&T ipad4 Boingo ipad4 Comcast iphone5 AT&T iphone5 Boingo iphone5 Comcast

23 San Francisco, CA (Downlink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Comcast Laptop AT&T Laptop Comcast MacBook AT&T MacBook Comcast Nexus10 AT&T Nexus10 Comcast ipad4 AT&T ipad4 Comcast iphone5 AT&T iphone5 Comcast San Francisco, CA (Uplink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 AT&T Galaxy S3 Comcast Laptop AT&T Laptop Comcast MacBook AT&T MacBook Comcast Nexus10 AT&T Nexus10 Comcast ipad4 AT&T ipad4 Comcast iphone5 AT&T iphone5 Comcast

24 Mountain View, CA (Downlink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 Google Laptop Google MacBook Google Nexus10 Google ipad4 Google iphone5 Google Mountain View, CA (Uplink) Device Provider # of Hotspots Min QI Median QIII Max Mean Std Galaxy S3 Google Laptop Google MacBook Google Nexus10 Google ipad4 Google iphone5 Google

25 Contact Information North America Contact TEL: Takafumi Watari (Primary Author) Engineer takawatari@allionusa.com Russell Nakamura - Engineer russellnakamura@allionusa.com Chad Meyer Vice President chadmeyer@allionusa.com Ryan Hoppes President Allion USA, LLC ryanhoppes@allionusa.com 25