TOLLY No November 1999

Transcription

1 Premise: Internet Service Providers (ISPs), portals and ebusinesses are investigating higher-layer (Layer 4 to 7) Web switches to provide advanced traffic management services such as load balancing and bandwidth management to servers. In addition, some vendors look to less expensive PC-based load balancing devices that emulate Layer 4 switching performance. Consequently, it has become critically important to understand the session processing performance and scalability of these switches and devices. In order to work effectively, they must have the capability to quickly set up and tear down sessions. In addition, Web switches should show linear scalability of this process as the number of connections increases. Alteon WebSystems, Inc. commissioned The Tolly Group to evaluate the TCP session-processing rate of its Alteon 180e Web switch against Foundry Networks ServerIron switch and the F5 Networks BIG/ip HA load-balancing device. While the Alteon 180e and the Foundry ServerIron are Layer 4 switches, the F5 BIG/ip HA is a PC-based device. The Tolly Group benchmarked the average number of sessions each product could set up and tear down each second using one-, two- and three-port Fast Ethernet connections. Testing was performed in August Test results show that the Alteon 180e established connections at a rate that equaled the limits of the test tool hard- T H E TOLLY G R O U P No November 1999 Alteon WebSystems, Inc. Alteon 180e Web Switch versus Foundry Networks' ServerIron and F5 Networks BIG/ip HA Load-Balancing Device TCP Session Processing Performance Evaluation via Layer 4 Switching Total sessions processed per second 80,000 70,000 60,000 50,000 40,000 30,000 20,000 10, ,116 19,201 10,224 Test Highlights 47,940 Test Summary! Utilizes a distributed architecture that enables linear scalability of TCP session processing! Performs session set up and tear down at a rate of 24,116 sessions per second with no session loss in a single-port pair environment! Executes session set up and tear down at a rate of 71,597 sessions per second with no session loss in a three-port pair environment! Exhibits high session-management performance while maintaining 10,000 to 30,000 open sessions Session-Processing Rate Scalability 11,998 71,597 5, Alteon Foundry F5 Number of 100 Mbit/s client and server ports (each) Source: The Tolly Group, November 1999 Figure The Tolly Group Page 1

2 ware and Smart TCP test application 6,500 sessions per second per SmartBits card, or 26,000 sessions total across all port scenarios tested with no session loss. The Foundry ServerIron experienced severe session processing performance degradation. The ServerIron demonstrated a linear drop in session processing performance from 80% of the performance of the Alteon 180e on the single-port test to 10% of the Alteon 180e on the three-port test as the aggregate number of sessions and the offered rate was increased. This is attributed to the central processing architecture, which handles session administration. For the single-port test, the F5 BIG/ip processed less than half of the sessions that the Alteon 180e was able to process. Further, the F5 BIG/ip was unable to process any measurable number of sessions for the other tests. According to F5, an architectural limitation on its adapters prevents more than 160 simultaneous connection attempts to be supported at any time. Any sessions beyond that ceiling are dropped due to a limited amount of transmit ring buffer capacity. 1 1 The F5 BIG/ip HA was tested in singleport tests only because the product was unable to attain a zero-loss point when engineers attempted to configure it using two- and three-port connections. F5 Networks offered the following explanation: "Apparently the Netcom Systems Inc. SmartBits 2000 attempts as many connections as it has been configured for; just as one would expect from a tool like this. This works well for switch-based products that are inherently (in most cases) designed to handle a punch like that - with ASIC(s) and processors per switched port. There is a difference, of course, between switched ports and ports on a NIC. The Intel NICs that we use have a receive ring buffer configuration that can handle a maximum of 32 connection attempts at any given time (32 buffers), we've enabled an additional 128 buffers in the BIG/ip OS for a total overall transmit TX ring buffer capacity of 160. This means that anything over 160 simultaneous connection attempts is going to result in dropped attempts. Please note that this does not limit the overall number of open connections, it simply affects the rate at which new connections can be established. And this, we believe, is responsible for what The Tolly Group saw during testing." Results Alteon 180e Web Switch Distributed Processing Architecture Source: The Tolly Group,October 1999 Figure 2 Single-Client Port and Single-Server Port Results showed that the Alteon 180e processed an average of 24,116 sessions per second in a single-port pair configuration when testing its capability to set up and tear down 40,000 sessions while 10,000 base sessions were active. In the same scenario, results demonstrated that the Foundry ServerIron processed an average of 19,201 sessions per second, or 20% less than the Alteon 180e. When the F5 BIG/ip HA was tested in the same situation, results demonstrated that it processed an average of 10,224 sessions per second in a single-port pair configuration, representing less than half the performance of the Alteon 180e. See figure 1. Dual-Client Ports and Dual-Server Ports In a second set of tests, The Tolly Group measured the set up and tear down performance of the Alteon 180e and the Foundry ServerIron in a dualport pair, dual-server scenario, when processing 80,000 sessions after an initial 20,000 base sessions were activated. Results showed that the Alteon 180e is capable of processing an average of 47,940 sessions per second. The Foundry ServerIron demonstrated an average set up and tear down rate of 11,998 sessions per second, or 75% less than the Alteon 180e. See figure 1. Due to architectural limitations, the F5 BIG/ip did not participate in this test. See footnote 1. Three Client Ports and Three Server Ports Engineers next conducted tests utilizing three client ports and three server 1999 The Tolly Group Page 2

3 ports to process 120,000 sessions after an initial 30,000 base sessions were activated. Results demonstrated that the Alteon 180e is capable of processing an average of 71,597 sessions per second. In the same configuration, the Foundry ServerIron showed that it can set up and tear down sessions at an average rate of 5,993 sessions per second, only 10% of total performance offered by the Alteon 180e. See figure 1. Due to architectural limitations, the F5 BIG/ip could not participate in this test. See footnote 1on page 2. Analysis ISPs, portals and hosters interested in improving and balancing the traffic flow to servers are purchasing Layer 4 switches that can balance the data from additional servers added to the network. The benefit of adding Layer 4-7 functionality is that it can provide access to a back-up processing unit in the event that a server fails, as well as provide ultra-granular control of directing traffic on session information such as TCP/UDP ports, URLs, cookies, etc., found deep in each packet. When searching for such a device, ebusinesses need to be assured that their load-balancing purchase can provide them with optimal set up and tear down performance, even when presented with a range of operating scenarios. So-called flash crowds one-time events driven by an overwhelming number of simultaneous users trying to access the same resources are becoming a common occurrence in the on-line world. This Layer 4 Web switch should be able to quickly set up and tear down TCP sessions, as well as successfully handle linear scalability when additional client and server connections are enabled. The Alteon 180e Web switch shows that when adding a second link, it can double its set up/tear down session performance. Furthermore, when adding a third link, it can triple its performance. Alteon achieves its linear scalability due to a distributed processing approach that outfits each switch port with dual processors and 2 Mbytes of memory. All Alteon switches utilize the same distributed processing architecture. See figure 2. This test was conducted using the Fast Ethernet ports on the Alteon 180e. According to Alteon, identical results can be expected on the ACEdirector products, which feature Fast Ethernet only. Foundry, by contrast, relies upon a central processing architecture to handle session administration. When a session set up request comes in on an Alteon 180e port, the processors and memory reside at the port so the request is handled in a distributed, port-by-port basis. When a session set up/tear-down request enters a Foundry ServerIron port, it must be passed internally to the central location where the memory and processor set up all requested sessions. Consequently, the Server- Iron s central processing design resulted in session set-up degradation. While consumers may look to a PCbased load-balancing device for cost savings, such as the F5 BIG/ip HA, customers should realize that such products may be limited in certain environments because of the inherent constrictions of network adapters. Customers can overcome such limitations by moving to an ASIC (application specific integrated circuit)-based solution. ASIC-based designs are better able to handle large traffic bursts, such as those offered in these tests because of integrated processing resources. The Alteon 180e is first and foremost a switch. As a switch, it is built to handle input from multiple ASIC-based ports and transfer traffic across these ports. Additionally, Alteon has built the 180e with dedicated memory and processors on each of these ASICs, allowing for distributed processing of session requests. Alteon WebSystems, Inc. Alteon 180e Web Switch TCP Session Processing Performance Evaluation Alteon WebSystems, Inc. Alteon 180e Web Switch Product Specifications*! Eight Gbit/s aggregate switch capacity! Eight selectable 10/100/1000 Mbit/s Ethernet ports and one 1000 Mbit/s uplink! Physical redundancy on 10/100/1000 Mbit/s ports! Simultaneous Layer 2, 3, 4 and 7 switching! Support for URL-based redirection and load balancing! VRRP support for active/active redundancy at Layer 3 and Layer 4! 192,000 session set ups and tear downs per second per switch! Up to 2,048 services per virtual IP address! Application redirection of any type! Per-port packet filtering of up to 224 packet filtering rules per switch for flexibility and control of all IP traffic! Supports local and global server load balancing, firewall load balancing, packet filtering, IP routing and TCP/IP redirection services! Support for TCP, UDP and IP server load balancing including http (persistent and non-persistent), FTP and passive FTP, SSL, DNS, Radius, Telnet and NNTP! Support for 802.1Q VLAN tagging with 256 network-wide VLANs per port For more information contact: Alteon WebSystems, Inc. 50 Great Oaks Parkway San Jose, CA Phone: Fax: URL: *Vendor-supplied information not verified by The Tolly Group 1999 The Tolly Group Page 3

4 Physical Test Bed W&G Domino Analyzer Links between the SmartBits chassis and the Layer 2 devices connect four ML-7710 cards to each Layer 2 device. One half of the 7710 cards act as servers while the remaining half are configured as clients. SmartBits 2000 and 10 chassis with 40 total ML /100 cards Layer 4 switch device under test There is a single link between each Layer 2 device and the device under test. SmartBits Controller Note: In this test, es acted as traffic aggregators for the Layer 4 switch under test. Source: The Tolly Group, November 1999 Figure 3 The F5 BIG/ip HA is not a switch. It is a PC that uses multiple Intel processors in this case, Pentium III processors, running at 500 MHz. By default, the F5 BIG/ip HA ships with two network adapters installed one for incoming traffic, such as that from a router connected to the Internet, and one for outgoing traffic bound for local resources, such as Web servers. In these tests, The Tolly Group added network adapters in order to simulate multiple inbound and outbound links. Test Configuration and Methodology The following Layer 4 switches were used for testing: an Alteon Web- Systems, Inc. Alteon 180e Web Switch, an eight-port 10/100/1000 Mbit/s Ethernet Layer 4 switch with a single Gigabit Ethernet uplink, software code ; and a Foundry Networks ServerIron Switch model number FBS8, an eight-port 10/100 Mbit/s Ethernet Layer 4 switch with a single Gigabit Ethernet uplink, software version T12. In addition, an F5 Networks BIG/ip HA Layer 4 loadbalancing device Version 2.0.4PTF- 03 was tested in single-port tests. All devices were connected via full-duplex Fast Ethernet. The Logical Traffic Flow Session traffic SmartBits 2000 and 10 chassis with 40 total ML /100 cards providing traffic aggregation Session traffic Layer 4 switch device under test Source: The Tolly Group, November 1999 Figure The Tolly Group Page 4

5 systems under test were connected to six Alteon ACEswitch 110 Layer 2 switches, model number In addition, each linked to four ML /100 Ethernet cards loaded on a Netcom Systems, Inc. SmartBits 2000 Advanced Multiport Performance Chassis, model number SMB The SmartBits generated traffic that ran across simulated Internet connections. There were a total of 40 SmartBits cards loaded on the chassis; however, engineers only utilized 24 for this series of tests. Half of the SmartBits cards were configured as clients and half were configured as servers. Each Layer 2 switch was connected to either four client or four server cards because a single SmartBits card (6,500 sessions per second) is not fast enough to stress the switches under test. In this test scenario, the Layer 2 switches acted as traffic aggregation points, collecting traffic from the various SmartBits ports and forwarding it to a specified port on the device under test. In order to observe network traffic patterns, The Tolly Group connected a Wavetek Wandel Goltermann DominoFastEthernet Inline Analyzer model number DA-350, between the device under test and one of the Layer 2 devices, which provided access to server ports on the Smart- Bits. The Domino Core software version 2.3 was running on a 200- MHz Intel Pentium 200 MMX with 32 Mbytes of RAM. The analyzer host was equipped with a Compaq Netflex 2 10/100 PCI adapter, running Microsoft Corp. Windows NT Workstation Operating System version 4.0 Build 1381 with Service Pack 3 installed. Each of the links between the Layer 2 devices and the devices under test were meant to simulate links that would be used by multiple clients or servers. In a real-world scenario, the ingress links coming into the devices under test would likely emanate CLIENT Client sends session request to server Client acknowledges server and completes connection set up Client initiates close of session Client accepts and acks back Session Set Up and Tear Down for SmartTCP Tests SYN-ACK DATA FIN-ACK from Internet routers, whereas the device s egress ports would provide connections to Layer 2 devices offering access to downstream servers. The SmartBits chassis was connected to a SmartBits Controller running on a 300-MHz Intel Corp. Pentium II with 64 Mbytes of RAM running Microsoft Windows NT Workstation version 4.0 Service Pack 4. The SmartBits Controller hosted all of the SmartBits-related applications including SmartTCP version 1.0. The controller also was equipped with a 3Com PCI Ethernet Controller adapter model number 3C905B-TX. See figure 3. For a more logical look at the test bed environment. See figure 4. Engineers ran the SmartTCP script to set up the base sessions and left them open on the device under test for the duration of each test. For the SYN ACK FIN ACK SERVER Server acknowledges client request and offers to open session Data flows between server and client Server acknowledges and offers to close session Session closes Source: The Tolly Group, November 1999 Figure 5 single-port tests, 10,000 base sessions were activated and for twoport tests, 20,000 base sessions were activated. When engineers conducted three-port tests, 30,000 base sessions were established. Engineers then ran the Session Rate Test, which is also part of the SmartTCP application software version 1.0. The software processed 40,000 sessions per server port. Engineers then executed the SmartTCP test for three iterations. Once engineers verified that there was no session loss, connection set up rate was recorded in a Microsoft Excel spreadsheet created by the SmartTCP application. For the purpose of session set up and tear down, six steps come into play. First, three steps are used to initiate a session between the client and the server. The client sends a SYN request to the server, which replies 1999 The Tolly Group Page 5

6 with a SYN-ACK, and then the client acknowledges with an ACK. That sets up the session and enables data to flow. To close the session, the client sends a FIN back to the server, which replies with a FIN-ACK. The client responds with a final ACK and the session terminates. See figure 5. Equipment Acquisition and Support The Alteon 180e and the Foundry ServerIron were supplied by Alteon WebSystems, Inc. F5 Networks, Inc. supplied the F5 BIG/ip HA used for testing. The Tolly Group contacted executives at Foundry Networks and F5 Networks and invited them to provide a higher level of support than available through normal channels. Foundry accepted the invitation. Alteon acquired a current version of the Foundry software under test. Foundry was notified of the configuration used by engineers and provided technical support to configure/tune the device for the test suites executed by The Tolly Group. F5 Networks accepted the invitation to provide a higher level of support and provided on-site support for a portion of testing and thereafter support by phone and . The Tolly Group verified product release levels and shared test configurations with Foundry and F5 in order to give both opportunities to optimize their devices for the testing. The Tolly Group shared test results with Foundry, but as publication of this report neared, Foundry was unable to acknowledge the validity of the results due to technical issues. When The Tolly Group shared test results with F5, it sent the explanation found in Footnote 1 above. For a more complete understanding of the interaction between The Tolly Group and Foundry Networks, check out the Technical Support Diary for Competitive Products Tested posted on The Tolly Group s World Wide Web site at See document The Tolly Group gratefully acknowledges the providers of test equipment used in this project. Vendor Product Web address Netcom Systems SmartBits Wavetek Wandel Goltermann DominoFastEthernet Since its inception, The Tolly Group has produced highquality tests that meet three overarching criteria: All tests are objective, fully documented and repeatable. We endeavor to provide complete disclosure of information concerning individual product tests, and multiparty competitive product evaluations. As an independent organization, The Tolly Group does not accept retainer contracts from vendors, nor does it endorse products or suppliers. This open and honest environment assures vendors they are treated fairly, and with the necessary care to guarantee all parties that the results of these tests are accurate and valid. The Tolly Group has codified this into the Fair Testing Charter, which may be viewed at Project Profile Sponsor: Alteon WebSystems, Inc. Document number: Product class: Layer 4 switch Products under test: " Alteon 180e Web Switch " Foundry ServerIron " F5 BIG/ip HA Testing window: August 1999 Software status: " All Readily available Additional information available: " Technical Support Diary " Configuration Files For more information on this document, or other services offered by The Tolly Group, visit our World Wide Web site at send to [email protected], call (800) or (732) Internetworking technology is an area of rapid growth and constant change. The Tolly Group conducts engineering-caliber testing in an effort to provide the internetworking industry with valuable information on current products and technology. While great care is taken to assure utmost accuracy, mistakes can occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential damages which may result from the use of information contained in this document. All trademarks are the property of their respective owners. The Tolly Group doc rev. clk 17 Nov The Tolly Group Page 6