Network Security Equipment The Ever Changing Curveball breakingpointsystems.com
This document contains information that is the property of BreakingPoint Systems, Inc. This information may not be copied, reproduced, or transferred in any form for purposes other than its intended use without prior written consent of BreakingPoint Systems, Inc. The information found within this document is subject to change without notice. All information provided is believed to be accurate and is presented without warranty of any kind, expressed or implied. Notwithstanding any other warranties, all files are provided as is with all faults. BreakingPoint Systems, Inc. disclaims all warranties, expressed or implied, including, without limitation, those of merchantability, fitness for a particular purpose and noninfringement. In no event shall BreakingPoint Systems, Inc. be liable for any claim, damages, or other liability arising out of the use or inability to use this document. BreakingPoint and the BreakingPoint System logo are registered trademarks or service marks of BreakingPoint Systems, Inc. The Ever Changing Curveball, white paper Copyright 2005-2007 BreakingPoint Systems, Inc. All rights reserved.
Overview The prevalence of network security equipment in enterprise networks has reached an all-time high. Businesses rely on network security equipment to protect their network infrastructures from today s hostile network environment. Unfortunately, there are few enterprises that properly test their network security equipment. When it comes to selecting network security devices, an enterprise network will either take a security-first posture or a performance-first posture. Most enterprise networks will take the performance-first posture, so they will test for any negative performance effects on their network. Usually, they will test a security device by trialing the box in their networks for a few weeks. During this time, they are monitoring for any adverse effects to their network. However, when it comes to network security equipment, trialing the box is not enough. The trial results do not account for the future growth or changing dynamics of the network, or the dynamics of the network security equipment itself. Network security devices are regularly updated with new security packs, which provide them with the ability to detect and act upon the latest threats. Without properly testing these devices, enterprises cannot determine how these updates will affect their network or the device. This is the unforeseen challenge faced by today s enterprises: they lack test equipment that can provide an accurate representation of their network s performance and security coverage. With this challenge in mind, BreakingPoint Systems has created the BPS- 1000, a comprehensive test solution that meets the testing demands of enterprise networks. This paper focuses on how the BPS-1000 can address the security testing needs of enterprise networks. It will illustrate why it is necessary to test security and performance concurrently, and it will demonstrate how having the proper test equipment can help identify a security device s vulnerabilities. The BPS-1000 Test Solution At BreakingPoint Systems, we know that to effectively test network security equipment, you need to concurrently send live attacks while running high-speed application traffic through the device. We believe that this is the only methodology that will provide an accurate summarization of a device s effectiveness. Based on this knowledge and belief, we have created the BPS-1000. Today, it is the only test equipment that effectively tests security coverage and performance by interleaving the three baselines of security testing: TCP sessions, application traffic, and live security attacks. It is the BPS-1000 s ability to fully integrate these performance and security aspects of testing that will prepare you for every curveball that comes your way. Curveballs in the Security Landscape It isn t everyday that a comparison is made between baseball and network security, but an analogy between the two isn t that farfetched. In baseball, a curveball is a pitch that breaks sharply at the last second. The unexpected break at the end of the curveball offsets the hitter s timing and causes him to miss-swing. If you re familiar with baseball, you know that there are two things that can affect when the curveball reaches its breaking point: its velocity and its spin. The pitcher can completely change the curveball by altering either the spin or speed of the pitch. 1
To tie this analogy in with testing, let s look at how the curveball relates to network security devices. For network security equipment, two aspects create the curveball: performance and security updates. Both these aspects create unique curveballs that can vary from network to network, and device to device. A change to either of these aspects can cause a security device to miss attacks, drop packets, or block valid traffic. Performance The first aspect of the curveball, performance, refers to a device s ability to effectively provide security coverage under changing network conditions. Any variance in the network or its performance can drastically affect a device s ability to detect and block attacks. These variances can range from a change in traffic rate to an increase in application protocol usage. To illustrate this case, consider the following example: A security device effectively blocks attacks in a network with 50 users and 1 Mbps of traffic; however, when relocated to a network with 5,000 users and 1 Gbps of traffic, the device begins dropping packets and missing attacks. Suddenly, the users are experiencing slow connectivity, and even worse, they are now vulnerable to attacks. In this example, the enterprise could have uncovered this limitation with some simple testing. If they had tested the device under various loads, they would have discovered that speeds higher than 1 Mbps would expose their network to security threats and slow down their network. To further emphasize the importance of performance testing, we have created the following test case. Device Under Load Test Case This test case has been set up for two well-known Intrusion Prevention Systems 1. Using the BPS-1000, we will show how an increase in traffic can affect a security device s ability to block attacks. The BPS-1000 offers five levels of network-based attacks; each subsequent level increases the difficulty of the test. For our portion of testing, we are going to use Security Level 2, which consists of about 450 Strikes and uses no evasion techniques. The test case consists of the following two scenarios: In the first scenario, the BPS-1000 will send Level 2 attacks and 1 Mbps of background traffic to each IPS. In the second scenario, the BPS-1000 will send Level 2 attacks and 1 Gbps of background traffic to each IPS. 1 The Intrusion Prevention Systems used in our test cases will not be identified. Both device s list their throughput at 1 Gbps. 2
Test Results Figure 1: Missed Attacks Figure 1 shows the test results for the number of missed attacks by both devices at 1 Mbps and 1 Gbps. 444 attacks were sent to Device A and Device B. Our results indicate that at 1 Mbps of traffic, Device A missed 331 attacks and Device B missed 229 attacks. At higher speeds, Device A missed 329 attacks, and Device B missed 242 attacks. Figure 2: Blocked Attacks Figure 2 shows the test results for the number of blocked attacks by both devices at 1 Mbps and 1 Gbps. 444 attacks were sent to Device A and Device B. Our results indicate that at 1 Mbps of traffic, Device A blocked 113 attacks and Device B blocked 215 attacks. At higher speeds, Device A blocked 115 attacks, and Device B blocked 202 attacks. 3
As we have mentioned, the BPS-1000 offers five levels of security testing; Level 1 is the easiest security test and Level 5 is the most difficult. For this test, we used Security Level 2, which is one of the easier security tests. At this level, most commercial-grade IPS s should be able to block a majority of the attacks; however, Device A only blocked an average of 25% of the attacks, and Device B blocked an average of 47% of the attacks. Imagine the catastrophic effects Level 5 2 would have on Device A and B. Although both IPS s can perform at 1 Gbps, this test reveals that their security coverage is affected by the speed at which traffic is sent to them. At a higher load, Device A was better at blocking attacks than at a lower load. Device B, on the other hand, was better at blocking attacks at a lower load than at a higher load. Typically, we expect that a device will perform better at lower traffic loads; however, Device A clearly proves that this is not always the case. Device A provided more coverage at a higher speed. These test results did not provide the outcome we expected, but they proved our claim that security devices are unpredictable under changing network conditions. Security Updates Security updates, the second aspect of the curveball, can completely change a security device in terms of performance and security coverage. These security packs may seem innocuous, but they can impact a device s performance. Typically, a vendor will immediately release a security pack when they discover a vulnerability. Each security pack contains new or updated signatures that allow you to filter the malicious traffic. Vendors will continue to release a series of security packs until they have completely resolved the issue. A vendor will normally do this to provide an immediate fix for the problem, but the quick fix may not be the optimal solution for the problem, or for your network. This process provides a false sense of security because it can be months before the device has the capabilities to block those threats. Until then, these network devices will remain vulnerable to certain security attacks and experience a decline in performance. To show how a security pack can impact a device s performance, we have set up the following test case. Latency Test Case This test case will use the following scenario: A device receives a security pack that updates its HTTP signatures. We want to know how the update will affect the device in terms of latency. To do this, we are going to set up three test cases for Device A that will only send HTTP traffic to the device. Each test will be used to measure the percentage of change in latency that occurs between the tests. The first test will measure the device s latency with no signatures enabled. The second test will measure that device s latency with the original signatures enabled. The third test will measure the device s latency with the updated HTTP signatures enabled. The BPS-1000 will measure the minimum, average, and maximum latency values. For our test purposes, we will only look at the maximum and average latency values. 2 We will cover Level 5 security testing in another whitepaper. Keep an eye out for our next whitepaper release. 4
Test Results Figure 3: Maximum Latency Results Figure 4: Average Latency Results Figure 3 and Figure 4 show the test results for the device s maximum and average latency. In the first test, where no signatures were enabled, the device s average latency was 0.03 milliseconds, while the maximum latency was 0.11 milliseconds. We used these values as the baselines for the subsequent tests. In the second test, where the original shipping signatures were installed, the device s average latency was still 0.03 milliseconds, but its maximum latency increased to 0.14 milliseconds. The device s performance experienced a slight increase in maximum latency between the first two tests, but this slight increase caused the latency to increase by 1.2 times its baseline maximum latency. 5
The third test reveals that the device s average latency increases to 0.30 milliseconds, while the device s maximum latency increased to a whopping 69.74 milliseconds. This test was performed with the latest updated signatures available from the vendor s automatic download service. This security update causes the maximum latency to drastically increase by 634 times its baseline maximum latency, with an average latency increase of 10 times the baseline latency. This huge increase in latency will most likely impact a network by slowing down any Web-based traffic and causing HTTP connections to be dropped, which will ultimately result in reduced productivity for users across the network. These test results illustrate why it is so important to test a security pack before applying it. By measuring the device s latency before the security update, we were able to determine how the security pack would affect the device s performance after it has been applied. Conclusion You have read two test cases that demonstrate the importance interleaving the crucial aspects of network security testing. The first test case showed that a security device s effectiveness can be affected by a change in traffic load. The second test case illustrated how security packs can have a drastic and unpredictable effect on a device s performance. Together, these cases validate the testing methodology that security testing can only be done by concurrently sending application traffic and attacks to the device. In the past, enterprise networks had to choose either the performance-first or the security-first posture. Today, the BPS-1000 makes it possible to choose both. The BPS-1000 is the only test equipment that truly supports concurrent performance and security testing. Enterprise networks no longer have to depend on the trial period or ad-hoc testing methods. With the flexibility of the BPS-1000, they can create tests for every curveball and every network condition imaginable. Now, enterprises can truly determine how security equipment affects their networks today and their networks in the future. Contact Information For sales information or general inquiries, please contact BreakingPoint Systems: BreakingPoint Systems 10535 Boyer Blvd., Suite 300 Austin, TX 78758 (512)821-6000 info@bpointsys.com http://www.breakingpointsystems.com/ 6