G R O U P. The Impact of Atheros Super G Proprietary Performance Mode on g Devices WHITE PAPER. August 2004

Transcription

1 The authoritative, unbiased source for IT certification, research and testing WHITE PAPER August 2004 A white paper commissioned by Broadcom Corp. Document # High-Speed Wireless LANs: The Impact of Atheros Super G Proprietary Performance Mode on g Devices 2004 The Tolly Group

2 Statement of Licensing Info and Acceptable Usage Entire contents 2004 The Tolly Group, Inc. All rights reserved. For additional information on acceptable usage of this document (Tolly Group Document #204120) contact The Tolly Group at (561) or via Reproduction of this publication in any form without prior written permission is forbidden. The information contained herein is believed to be accurate and reliable. The Tolly Group shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. All excerpts from this report must be approved by The Tolly Group in advance of publication or use in any public materials. Tolly Group Services With more than 15 years experience validating leading-edge Information Technology products and services; The Tolly Group has built a global reputation for producing accurate and unbiased evaluations and analysis. We employ time-proven test methodologies and fair testing principles to benchmark products and services with the highest degree of accuracy. Launched in 2003, The Tolly Group's "Tolly Verified" service provides in-depth, vendorneutral certification of an array of features, functions and performance characteristics in technology disciplines as diverse as WLAN Switching and Anti-spam. See our "Tolly Verified" Home Page. Our "Up-to-Spec" service provides the custom testing complement to the "standard", granular tests offered in "Tolly Verified". See our "Up-to-Spec" Home Page. The "First & Foremost" designation is awarded to products that are the "First" to demonstrate specific performance or functionality in The Tolly Group's labs. Products also may earn a "Foremost" designation by demonstrating record performance or improved functionality that exceeds other products previously tested and certified by The Tolly Group. Such products are deemed the "Foremost" in their category by performance or by functionality. See our "First & Foremost" Home Page. Plus, unlike narrowly focused testing labs, The Tolly Group combines its vast technology knowledge with focused marketing services to help clients better position product benchmarks for maximum exposure. This document was authored by: Charles Bruno, Executive Editor The Tolly Group

3 Table of Contents 4 Executive Summary 5 The Impact of Proprietary Performance Modes on g Devices 7 Good Neighbor/Bad Neighbor Performance 7 Baseline Netgear Super G Mode, USR Turbo Mode 7 Mixed-Brand g + PPM Scenario 8 Multi-AP Performance g Baseline Performance 9 Proprietary Performance Mode 10 Big Picture Findings List of Figures 6 Figure 1: WLAN Devices Tested 7 Figure 2: Impact of Netgear Proprietary Super G Mode on a Belkin g Wireless Router 8 Figure 3: Average TCP Throughput in Mbps for Mixed Brand g Devices 9 Figure 4: Average TCP Aggregate Throughput in Mbps for Multi-AP Scenario

4 : Executive Summary Tolly Group testing, conducted in late 2003 and early 2004, shows that a non-standard WLAN performance-enhancement technology offered by Atheros Communications, Inc. (the Atheros Super G at the heart of the Netgear WGT624) effectively robs precious bandwidth from standardsbased access points (APs)/broadband routers. In fact, when a standards-based Belkin AP/router (based on Broadcom technology) operated 30' away from a Netgear AP implementing Atheros Super G, its system throughput dropped over 90% from 23 Mbps to just 1.3 Mbps. This deleterious effect of the Atheros Super G extended well beyond Broadcom equipment as tests show that g gear using silicon from Broadcom, Texas Instruments and, ironically, even Atheros, all suffered performance degradation with Super G mode active. One of the reasons for this 'bad neighbor' behavior lies at the very base architecture in the Atheros chipset that the Netgear WGT624 uses. The Atheros Super G technology relies upon multiple performance enhancement features including radio enhancements and 'double-channel bonding.' Double-channel bonding operates by using two radio channels, bonding them so they appear as a single unified channel to the transmitter and the receiver. On the downside, double-channel bonding reduces the number of available channels on the network, in effect, starving out g devices that contend for the same radio spectrum. The Netgear WGT624 also employs dynamic packet bursting. In standard WLANs, there is a pause after each data packet is transmitted to allow other devices to contend for bandwidth. But Atheros Super G technology does not respect the pause. In a Netgear white paper, the company says, "Increasing the number of packets sent without pausing to see if another node wants to use the wireless LAN can increase throughput." The presence of Atheros Super G technology severely degrades the performance of adjacent networks based upon Wi-Fi g standards. The effects can be observed even when the networks are separated by distances of 30' to 150'. One important point to note: Standards-based g products do not cause harmful interference with other g devices The Tolly Group 4 August 2004

5 The Impact of Proprietary Performance Modes on g Devices The ancient Greek poet Hesiod may never have known the benefits of high-speed wireless LANs but he spoke prophetically about them when he said: "A bad neighbor is a misfortune, as much as a good one is a great blessing." In wireless LANs (WLANs), the key to a successful wireless infrastructure is having access points (APs), wireless routers and wireless clients that all get along and respect each others' space just as good neighbors do. In any community, one noisy neighbor cranking up the music makes it impossible for the good neighbors to listen to their own musical favorites because the bad neighbor drowns out surrounding sites. In a WLAN, a bad neighbor who doesn't respect other neighbors RF space gets in the way so the good neighbors cannot accomplish routine tasks. In fact, the advent of proprietary WLAN implementations raised user concerns that such technologies would interfere with other standards-compliant vendor products. Anecdotal evidence began to mount that emerging proprietary WLAN technologies that promise to surpass the speed of a/g products caused interference problems, leading Broadcom Corp. to commission The Tolly Group to conduct an in-depth evaluation of this and related issues. The Tolly Group took a long, hard look at the Atheros Super G technology to determine the impact of this proprietary performance enhancing technology 1 on standards-based g products. Some vendors who use these Proprietary Performance Mode (PPM) technologies, such as Netgear with its 108 Mbps Wireless Super G technology are offering proprietary extensions to g that promise to push maximum bandwidth close to traditional wired Fast Ethernet rates. Netgear and other vendors who use PPM technologies are looking to differentiate themselves from the WLAN pack with so-called proprietary speed enhancers. The question that needed to be addressed was: Are PPM technologies like Super G 'good' neighbors, or are they bad neighbors that hoard bandwidth by plundering it from other standards-based devices? 1 Collectively, these are referred to as "Proprietary Performance Mode" (PPM in charts) throughput the remainder of this document. 5

6 Figure 1: WLAN Devices Tested Vendor Product name Chipset source Belkin Corp. Netgear Corp. U.S. Robotics WLAN Devices Tested g Wireless DSL/Cable Gateway Router (F5D7230-4) WGT Mbps Wireless Firewall Router USR g Wireless Turbo Router Broadcom Atheros Tolly Group tests show that the presence of APs communicating with clients via the Atheros Super G proprietary mode has a dramatic adverse impact on the throughput of nearby devices communicating via standards-compliant g protocols. In effect, g devices that use the Atheros proprietary Super G mode make for 'bad neighbors.' At just 30-feet away from the Atheros-based solution (a Netgear WGT Mbps-Wireless Router), the performance of an g solution based upon Broadcom technology, drops from 22.9 Mbps (achieved when the Atheros solution was powered off) to only 1.27 Mbps on average across tests of all 11 channels. Texas Instruments Engineers observed the same type of adverse affect when products using the Atheros were configured to provide g-compliant communications versus the Atheros Super G-based system. Depending upon the systems involved, performance degradation remained significant even when the APs were 50' to 150' apart. A set of "control" tests, using all products operating in g mode, showed that there was some level of performance degradation though it was not nearly as dramatic as when the Atheros Super G mode was active. Even when tests devices were "deoptimized" to run on the same channels, average throughput of the lower-performing pair typically exceeded 6 Mbps much higher than the throughput attained when Super G was active. Moreover, when test systems were run over non-overlapping channels (i.e. one AP on channel 6, the other on channel 1 or 11) both systems delivered 24 Mbps of throughput which is the maximum throughput attainable for standards-compliant g devices. Finally, area aggregation tests, where three separate systems were set up near one another, illustrated that the three g devices that are configured to non-overlapping channels (one AP each on channels 1, 6 and 11) can deliver aggregate throughput of more than 75 Mbps. In contrast, three devices based upon the Atheros Super G technology achieved aggregate throughput of only 48.6 Mbps due to interference among the devices. 6

7 Good Neighbor/Bad Neighbor Performance Baseline Netgear Super G Mode, USR Turbo Mode Figure 2: Impact of Netgear Proprietary Super G Mode on a Belkin g Wireless Router Impact of Netgear Proprietary Super G Mode on a Belkin g Wireless Router Average Application Throughput in Mbps Belkin F5D g Wireless Router Engineers measured the effect that some PPM-enabled AP/routers have on g standards-compliant devices across all 11 channels. Netgear WGT624 Wireless Firewall Router (Super G mode) 30' apart Baseline single router In the scenario, a "bad neighbor" with a PPM-enabled AP/router communicates with a proprietary WLAN client while a "good neighbor" that has an g standardscompliant AP/router communicates with its standards-based WLAN client. In the test scenario, a Belkin g Wireless DSL/Cable Gateway Router (good neighbor) was paired with a Netgear WGT624 (bad neighbor) running in proprietary Super G mode. Tests show that the proprietary Netgear WGT624 Super G-based AP/router thrives at the expense of the g Belkin AP. In a test with the devices 30' apart, the Netgear WGT624 delivered 42.3 Mbps, not far off its baseline pace. What is startling though is the throughput of the Belkin g Wireless DSL/Cable Gateway Router plummets from a baseline of 23 Mbps to just 1.3 Mbps when it shares radio frequency space with the Netgear AP operating in Super G mode. That means throughput dropped more than 94% on the g standards compliant AP due to interference from the Netgear WGT624. For the test, the Netgear WGT624 remained camped on Channel 6, with the Belkin product hunting from channel to channel to find safety from the proprietary Super G technology. Regardless of which channel it tried, the g Belkin AP experienced low throughput while the Netgear AP tracked close to its baseline. Mixed-Brand g + PPM Scenario The Good Neighbor/Bad Neighbor test, as described earlier, demonstrates that PPM-enabled APs/routers can help PPM clients gain substantial bandwidth. Some of these devices can, at the same time, communicate with standards-based clients. But does the presence 7

8 Figure 3: Average TCP Throughput in Mbps for Mixed Brand g Devices Average TCP Throughput in Mbps for Mixed Brand g Devices Belkin client (F5D7010 Wireless CardBus) Netgear WGT624 AP (Super G mode) of PPM-enabled APs/routers have any impact on standards-based clients that are in the same network using the same APs? The Tolly Group sought an answer to this question in the mixed-brand g scenario test. This test allowed engineers to determine the impact (if any) and throughput capabilities that a PPM-enabled AP/router has with g standards-complaint wireless cards planted in client PCs. Engineers conducted two tests, one to determine that the g clients could associate with the PPM-enabled AP/router and another to determine the throughput as measured in Mbps. In the association test, engineers sought to determine how long it takes for an g client to associate with an AP running in PPM. In the AP/client association test with the Netgear WGT624, the Belkin F5D g Wireless Notebook Network Card took an average of five seconds to reassociate/reconnect to the Netgear WGT624 AP in the g-only mode. The Belkin wireless card did not detect the network when Netgear WGT624 was in proprietary Super G mode. Netgear client (WG511T Wireless CardBus) Average N/A 1 N/A 1 1 The client did not yield throughput because the Belkin client did not detect the network with the Netgear WGT624 AP in proprietary Super G mode. Next, engineers measured the throughput of the two wireless clients associated with an AP/router running in the PPM. One client was configured to run in PPM and the other to run in the g-compliant mode. To measure the throughput for each of the devices, engineers used Ixia IxChariot to generate bidirectional (twoway) traffic between a wireless and wired client pair. Each device was subjected to three runs of each test, transferring a file of 1 MB in size. Engineers recorded the maximum throughput that can be sustained for each run and averaged the results. In one g scenario with the clients communicating with a Netgear WGT624 AP in PPM, the Belkin client could not associate with the AP even though it was supposed to support both Super G and g simultaneously. With the client pair operating with a mix of Belkin and US Robotics APs, the Belkin client averaged throughput of 10.6 Mbps and the USR8054 averaged throughput of 12.4 Mbps. Multi-AP Performance Often times users, whether at home or in the office, have two or three APs "co-located" in the same "RF space" - that is, close enough for their radio frequency signals to reach one another. 8

9 Engineers set out to determine the maximum aggregate throughput of three such systems three each based on the g standard, the Atheros Super G and the TI Turbo Mode technologies. Results show that the nature of PPM technologies makes them unscalable, even with other APs supporting the same technology in the same RF space. When dealing with IEEE standards-compliant devices in a multi-ap environment, the devices as a whole are additive meaning that the total throughput for each AP was added together to obtain a total for all three APs together. The three APs were run on non-overlapping channels. To measure the throughput for each of the devices, engineers used Ixia IxChariot to generate bidirectional traffic between three wireless and wired client pairs. Each AP/router then underwent three test runs of each test, transferring a file of 1MB in size. Engineers recorded the maximum throughput that was sustained for each run and averaged the three. The test bed consisted of three mini-lans, where each LAN connected to a Dell PowerConnect 5212 switch via a wired connection from each LAN's AP/router. The IxChariot console had a wired connection to the switch so it could communicate to each LAN. Each LAN was made up of the following: one AP/router, one wired client and one wireless client g Baseline Performance For the test, engineers first established an g performance benchmark for each of the three AP/routers operating independent of one another across Channels 1, 6 and 11. Figure 4: Average TCP Aggregate Throughput in Mbps for Multi-AP Scenario Average TCP Aggregate Throughput in Mbps for Multi-AP Scenario (802.11g Mode vs. Proprietary Mode) Belkin F5D g Wireless Router g mode Netgear WGT624 Wireless Firewall Router (802.11g mode) U.S. Robotics 8054 (802.11g mode) PPM Netgear WGT624 Wireless Firewall Router (Super G mode) Average The Belkin g Wireless DSL/Cable Gateway Router delivered an average throughput of 76.5 Mbps across the three channels. The Netgear WGT624 AP averaged aggregate throughput of 64.7 Mbps and the US Robotics USR8054 delivered aggregate throughput of 72 Mbps. Proprietary Performance Mode Here, the cumulative throughput for the Netgear AP, with Super G running, was 48.6 Mbps, almost 25% less than measured in pure g mode. 9

10 Big Picture Findings For users who employ high-speed wireless in their homes, or small offices, the Atheros Super G technology cannot be used in situations where there are many APs/routers. Even in a home setting, where users may deploy multiple APs to support media servers or to extend the network's range, APs/routers based upon the Atheros Super G technology likely will not be the best choice. Tests show that when users employ more than one of the Super G devices in a network, they begin to interfere with one another and throughput suffers. Moreover, such devices, quite literally, may be bad neighbors since their RF footprint may overlap with neighboring homes that employ g APs. In such a case, proprietary Super G and other PPM technologies likely would have a deleterious bandwidth impact on neighbors. Contrast Super G's impact with the fact that testing shows standardsbased g devices result in zero harmful interference to other g devices. Considering that wireless is a shared-bandwidth service, PPM technologies like Super G break the rules of consideration that come into account when sharing bandwidth. PPM technologies such as the Atheros Super G exhibit "selfish" tendencies when it comes to radio spectrum usage, so any user who deploys such technologies in a home of office does so with the knowledge they may be creating bandwidth turbulence in their neighbors' networks. Moreover, from a practical standpoint, if users are contemplating mixing installed g AP/routers alongside Super G models, test data shows that the PPM offerings, like the Super G-based Netgear WGT624, will interfere with, and rob bandwidth from, standards-based g devices, thus rendering useless an installed investment. By contrast, g devices do not cause harmful interference with other g devices. In the end, Super G-based offerings make for bad neighbors, while standards-based g equipment turns out to be a good neighbor that, as Heriod said, is a "great blessing." # # # 10

11 Information technology is an area of rapid growth and constant change. The Tolly Group conducts engineering-caliber testing in an effort to provide the internetworking industry with valuable information on current products and technology. While great care is taken to assure utmost accuracy, mistakes can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential damages which may result from the use of information contained in this document. All trademarks are the property of their respective owners. The Tolly Group, Inc FAU Blvd. Suite 100 Boca Raton, FL Phone: Fax: