Firewall Testing Methodology W H I T E P A P E R

Transcription

1 Firewall ing W H I T E P A P E R

2 Introduction With the deployment of application-aware firewalls, UTMs, and DPI engines, the network is becoming more intelligent at the application level With this awareness the network has the ability to implement intelligent security and traffic management policies that are tied to specific application and user characteristics However, with hundreds of thousands of applications and devices on the network and new security threats being discovered on them every day, test teams are struggling to quickly and effectively test their systems - Key requirements for an application aware security testing tool o Unified The test team needs a unified solution that models both the various types of security threats (such as Distributed Denial of Service, protocol fuzzing, anti-virus, DLP, url filtering and published vulnerability detection), and also real world applications o Exhaustive The resulting tests need to be exhaustive and cover the entire attack surface The solution needs to cover thousands of tests that can be run in an automated fashion and yet allow debug and analysis when issues are found o Simplicity Since test teams are pressed for time, they need a solution that does not require an expert user The ability to make the workflow of the test tool integrate with development and issue resolution is very important This is critical to make the issues found actionable and to get them to resolution rapidly Application Identification and Performance Application Identification & Control (white-listing and black-listing) Requirements This test will verify if the target is able to successfully identify applications when they flow through it sequentially o one by one The target has application signatures that are used by the system to examine the traffic pattern flowing through it and to match the observed pattern with the signature The application list must reflect what is seen on customer networks (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks Run the apps through the target one by one using standard ports and verify if they are detected Run the apps over non-standard ports and see if they were detected Run the apps over SSL/TLS and see if they were detected Run apps that are in the whitelist and verify if they get through successfully Run apps that are blacklisted and see if they are blocked On Standard Ports App List Detection (Yes/No), Whitelist (Pass/), Blacklist (Pass/) Copyright 0, Mu Dynamics, Inc

3 With Port evasion/non standard ports With SSL encryption App List Detection (Yes/No), Whitelist (Pass/), Blacklist (Pass/) App List Detection (Yes/No), Whitelist (Pass/), Blacklist (Pass/) Concurrent Application Identification & Control This test will verify if the target is able to successfully identify multiple applications when they flow through it concurrently or in parallel The target has application signatures that are used by the system to examine the traffic pattern flowing through it and to match the observed pattern with the signature The application list must reflect what is seen on customer networks (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks Run the apps through the target concurrently using standard ports and verify if they are detected Run the apps over non-standard ports and see if they were detected Run the apps over SSL/TLS and see if they were detected Run apps that are in the whitelist and verify if they get through successfully Run apps that are blacklisted and see if they are blocked On Standard Ports With Port evasion/non standard ports With SSL encryption App List Detection (Yes/No), Whitelist (Pass/), Blacklist (Pass/) App List Detection (Yes/No), Whitelist (Pass/), Blacklist (Pass/) App List Detection (Yes/No), Whitelist (Pass/), Blacklist (Pass/) App performance baseline with real production application mix Single Dimensional for This test will verify what the maximum user concurrency is for a mix of apps that mirrors the production network The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks The user can choose a single application or a groups of applications that are similar like video apps, voice apps, file transfer apps etc Run the apps through the target concurrently at increasing levels of user concurrency Measure max user concurrency with no more than % failures for the entire test-run as well as for each group Copyright 0, Mu Dynamics, Inc

4 Single Dimensional for Average Throughput This test will verify what the average throughput is for a mix of apps that mirrors the production network The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks The user can choose a single application or a groups of applications that are similar like video apps, voice apps, file transfer apps etc Run the apps through the target concurrently at increasing levels of user concurrency Measure average throughput for the entire test-run as well as for each group Average Throughput Single Dimensional for Maximum Active TCP connections This test will verify what the maximum active TCP sessions is for a mix of apps that mirrors the production network The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks The user can choose a single application or a groups of applications that are similar like video apps, voice apps, file transfer apps etc Run the apps through the target concurrently at increasing levels of user concurrency Measure max active TCP sessions with no more than % failures for the entire test run as well as for each group Maximum active TCP connections Single Dimensional for Maximum Active UDP sockets This test will verify what the maximum active UDP sockets used is for a mix of apps that mirrors the production network The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Copyright 0, Mu Dynamics, Inc

5 Recreate the list of applications that appear on customer networks The user can choose a single application or a groups of applications that are similar like video apps, voice apps, file transfer apps etc Run the apps through the target concurrently at increasing levels of user concurrency Measure max active UDP sockets with no more than % failures for the entire test run as well as for each group Maximum active UDP sockets Single Dimensional for Maximum Connection Rate This test will verify what the maximum connection rate is for a mix of apps that mirrors the production network The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks The user can choose a single application or a groups of applications that are similar like video apps, voice apps, file transfer apps etc Run the apps through the target concurrently at increasing levels of user concurrency Measure max user concurrency with no more than % failures for the entire test run as well as for each group Maximum connection rate Single Dimensional for bytes sent and received This test will verify what the bytes sent and received is for a mix of apps that mirrors the production network The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks The user can choose a single application or a groups of applications that are similar like video apps, voice apps, file transfer apps etc Run the apps through the target concurrently at increasing levels of user concurrency Measure bytes sent and received on client and server sides for the entire test run as well as for each group Bytes Sent and Received Copyright 0, Mu Dynamics, Inc

6 Single Dimensional for Average Response Time This test will verify what the average response time is for a mix of apps that mirrors the production network The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) Recreate the list of applications that appear on customer networks The user can choose a single application or a groups of applications that are similar like video apps, voice apps, file transfer apps etc Run the apps through the target concurrently at increasing levels of user concurrency Measure average response time per app and per group Average Response Time Multi-dimensional tests - App Performance with virus This test will verify how application performance is affected when virus detection and control is also performed at the same time The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) A separate malicious set of flows consisting of applications with viruses in the payload is also sent at the same time Recreate the list of applications that appear on customer networks Add a separate track of application flows with viruses in the payload Run the apps through the target concurrently at increasing levels of user concurrency Measure key application performance metrics in this multi-dimensional test and compare against the baseline Average Throughput Maximum active TCP connections Maximum active UDP sockets Maximum connection rate Bytes Sent and Received Average Response Time Copyright 0, Mu Dynamics, Inc

7 App Performance with fuzz This test will verify how application performance is affected when fuzz or malformed traffic is also sent at the same time The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) A separate fuzz or malformed set of flows is also sent at the same time Recreate the list of applications that appear on customer networks Add a separate track of application flows with viruses in the payload Run the apps through the target concurrently at increasing levels of user concurrency Measure key application performance metrics in this multi-dimensional test and compare against the baseline Average Throughput Maximum active TCP connections Maximum active UDP sockets Maximum connection rate Bytes Sent and Received Average Response Time App Performance with known This test will verify how application performance is affected when known attack profiles for which signatures are written is also sent at the same time The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) A separate set of known attack triggers are also sent at the same time Recreate the list of applications that appear on customer networks Add a separate track of application flows with viruses in the payload Run the apps through the target concurrently at increasing levels of user concurrency Measure key application performance metrics in this multi-dimensional test and compare against the baseline Average Throughput Copyright 0, Mu Dynamics, Inc

8 Maximum active TCP connections Maximum active UDP sockets Maximum connection rate Bytes Sent and Received Average Response Time App Performance with DDOS This test will verify how application performance is affected when Application level and network level DDoS is sent at the same time The application mix must reflect what is seen on customer networks The user can use the mixes provided by the test vendor or create their own (Business apps like Oracle, Communication apps like and IM, Consumer apps like PP and Gaming etc) A separate set of DDoS are also sent at the same time Recreate the list of applications that appear on customer networks Add a separate track of application flows with viruses in the payload Run the apps through the target concurrently at increasing levels of user concurrency Measure key application performance metrics in this multi-dimensional test and compare against the baseline Average Throughput Maximum active TCP connections Maximum active UDP sockets Maximum connection rate Bytes Sent and Received Average Response Time Virus Detection & Prevention This test will verify how effective the Firewall is in detecting and preventing malware such as viruses The virus and other types of undesirable malicious content are sent within files, videos or other media over various transports including http, ftp, SMTP and others Copyright 0, Mu Dynamics, Inc

9 Create or use a library of virus and malware content Send the malicious content along with valid app traffic through multiple transports and protocols, with and without compression Measure effectiveness of detection Measure effectiveness of Prevention Measure integrity of valid content Sequential Concurrent app traffic fuzz DDoS With background known published vulnerability _ Description N Published Vulnerability Attack Detection and Prevention This test will verify how effective the Firewall is in detecting and preventing known Since new published vulnerabilities are discovered almost every day the user needs to have a steady flow of the latest published vulnerability templates The application signatures for these need to be tested on a continuous basis Create or use a library of known published vulnerabilities Send known vulnerability triggers through the target Turn on multiple evasion types such as fragmentation to evade detection Measure effectiveness of detection Measure effectiveness of Prevention Sequential Concurrent With Evasion app traffic fuzz With background DDoS _ Description Copyright 0, Mu Dynamics, Inc 9

10 N DDOS detection and prevention This test will verify how effective the Firewall is in detecting and preventing Distributed Denial of Service DDoS need to be sent from multiple IP and MAC ids Embedded ids in the payload can also be randomized so as to prevent detection Create or use a library of DDoS Send multiple types of DDoS through the target and in some cases depending on the test to the target Monitor the health of the target as well as whether it comes back after the DDoS are removed Measure effectiveness of detection Measure effectiveness of Prevention Measure the availability and resilience of the target (% of time it was accessible for new users) Resilience (percentage of time the target was down and un reachable) _ N Description Sequential Concurrent Availability Resilience app traffic URL filtering This test will verify how effective the Firewall is in detecting and preventing unwanted url accesses Url filtering is a way to restrict access to unwanted urls for reasons of security, work-place productivity, ethics and privacy Create or use a library of known good and known bad urls Send traffic that access these known urls Turn on evasion types such as http pipelining Measure effectiveness of detection Measure effectiveness of Prevention Check for false negatives and positives Copyright 0, Mu Dynamics, Inc 0

11 Sequential Concurrent With Evasion app traffic fuzz With background DDoS _ Description N DLP or data ex-filtration testing This test will verify how effective the Firewall or Security device is in detecting and preventing important and confidential data from leaving the secure network Content filtering and DLP are ways to prevent leakage of confidential information to unauthorized entities outside the secure network Create or use a library of known good and known bad message flows Send traffic that access these sets of flows Turn on evasion types such as http pipelining Measure effectiveness of detection Measure effectiveness of Prevention Check for false negatives and positives Sequential Concurrent With Evasion app traffic fuzz With background DDoS _ Description N Fuzz Attack (Negative ing) This test will verify how effective the Firewall is in detecting and preventing fuzz or malformed traffic Copyright 0, Mu Dynamics, Inc

12 Fuzz need to be sent top the firewall or network security device as well as through it to test the ability to discard the malformed traffic and remain resilient in the face of unexpected negative flows Create or use a library of Fuzz Send multiple types of Fuzz through the target and in some cases depending on the test to the target Monitor the health of the target as well as whether it comes back during the test run and after the fuzz are removed Measure effectiveness of detection Measure effectiveness of Prevention Measure the availability and resilience of the target (% of time it was accessible for new users) Resilience (percentage of time the target was down and un reachable) _ N Description Sequential Concurrent Availability Resilience app traffic Copyright 0, Mu Dynamics, Inc

13 Web: wwwmudynamicscom Address: W Maude Avenue, Suite 0, Sunnyvale, CA 90, USA Phone: --0 or Fax: 0-9- Copyright 0 Mu Dynamics All rights reserved Mu Dynamics, Mu Suite, Mu-000, Mu-000, Mu Dynamics logo, and Innovate with Confidence are trademarks of Mu Dynamics