Next Generation Firewall Evaluation Report. E-Class NSA Series

Transcription

1 Next Generation Firewall Evaluation Report E-Class NSA Series 17 July 2012 Prepared by ICSA Labs 1000 Bent Creek Blvd., Suite 200 Mechanicsburg, PA

2 Table of Contents Executive Summary... 1 System Components... 3 Test Topology... 4 Product Deployment... 5 Basic Firewall Functionality High Availability Functionality User-Based Authentication Application Identification and Control User Protection Server Protection Miscellaneous Notes Summary Partners and Resources Testing Information Page i of i 1000 Bent Creek Blvd., Suite 200 Mechanicsburg, PA 17050

3 Executive Summary About Next Generation Firewalls Traditional firewalls inspect network communications at the network layer making decisions based on IP addresses, protocols and port numbers. As computer networks have become more sophisticated and applications are being pushed into the cloud, more controlled / granular access policies are needed to monitor and secure today s corporate and enterprise networks. Next Generation Firewalls provide this extended level of access control by monitoring network communications at the application layer including granular application identification and control and user-based authentication / identification as well as protecting users from modern day threats like botnets and sophisticated malware. These additional features must function properly while still providing the tried and true protection known in traditional firewall products. Product Overview The Dell SonicWALL E-Class Network Security Appliance (NSA) Series is an industry first using patented* Reassembly-Free Deep Packet Inspection (RFDPI) technology in combination with multi-core specialized security microprocessors to deliver gateway anti-virus, anti-spyware, intrusion prevention and Application Intelligence at high speed without sacrificing network performance. *U.S. Patents 7,310,815; 7,600,257; 7,738,380; 7,835,361; 7,991,723 Areas of Evaluation Dell SonicWALL contracted ICSA Labs to evaluate the E-Class NSA Series ability to: Provide Basic Network Firewall Functionality Provide High-Availability Functionality User-Based Authentication o o Maintain own user authentication information or integrate with a third-party solution Make access control policy decisions based on user s identity Application Identification and Control o o o o o Identify a breadth of applications such as popular websites, P2P applications, chat tools, VoIP applications, etc. Identify a depth of applications such as specific components / apps of popular social media websites Maintain metadata about applications (categories, ratings, locations, etc.) Make access control policy decisions for identified applications using a combination of user identity and application metadata Make bandwidth management decisions based off of application metadata and user identification User Protection o Mitigate attacks targeting users (such as malware, botnets, application flaws / vulnerabilities, etc.) Page 1 of 31

4 o o Restrict user access to undesired locations (such as by country, network, reputational metric, etc.) Make an override / whitelist for known good IP addresses in undesired location subset ranges Server Protection o Mitigate inbound attacks targeting hosted services (SMTP, HTTP, HTTPS, SIP, etc.) False Positives o o o Mitigate attacks without negatively impacting legitimate communications Prevent user access to undesired locations without misidentification Make application-based policy decisions without misidentifying applications Summary of Findings During the course of this evaluation, Dell SonicWALL submitted signature updates which provided additional security and application identification coverage for the items listed above. The E-Class NSA Series completed this evaluation with the following functional and security effectiveness: Area of Evaluation Effectiveness User-Based Authentication % Application Identification and Control % User Protection 98.34% Server Protection 94.60% Dell SonicWALL was able to provide the above effectiveness while not impacting normal / legitimate traffic thus causing false positives. About ICSA Labs The goal of ICSA Labs is to significantly increase user and enterprise trust in information security products and solutions. For more than 20 years, ICSA Labs, an independent division of Verizon, has been providing credible, independent, 3rd party security product testing and certification for many of the world s top security product developers and service providers. Enterprises worldwide rely on ICSA Labs to set and apply objective testing and certification criteria for measuring product compliance and performance. For more information, visit. About Dell SonicWALL Guided by its vision of Dynamic Security for the Global Network, Dell SonicWALL develops advanced intelligent network security and data protection solutions that adapt as organizations evolve and as threats evolve. Trusted by small and large enterprises worldwide, Dell SonicWALL solutions are designed to detect and control applications and protect networks from intrusions and malware attacks through award-winning hardware, software and virtual appliance-based solutions. For more information, visit Page 2 of 31

5 System Components Introduction ICSA Labs requires that vendors submit for evaluation at ICSA Labs all hardware, software, and documentation that comprise the product under test. For the purposes of this document, the term product refers to the complete system submitted by the vendor to ICSA Labs to be evaluated during testing. This includes any and all documentation, hardware, firmware, software, host operating systems, management stations, etc. used during testing. Servers providing common management services such as syslog and NTP are provided by ICSA Labs and are not considered part of the product under test. This section details the components of the product (or product family) submitted by Dell SonicWALL for evaluation. All items not listed in this section, as well as any relevant components, were provided by ICSA Labs. Hardware Dell SonicWALL submitted the following hardware to ICSA Labs for this evaluation: NSA E5500 The NSA E5500 had an 8 core 550Mhz Octeon CPU from Cavium Networks with 1GB RAM and 512MB Flash. The NSA E5500 had eight 10/100/1000Base-T ports, one RJ-45 Serial port, 2 USB ports, and a dedicated HA link port. NSA E7500 The NSA E7500 had a 16 core 600Mhz Octeon CPU from Cavium Networks with 2GB RAM and 512MB Flash. The NSA E7500 had four 10/100/1000Base-T ports, four mini-gbic ports, one RJ-45 Serial port, 2 USB ports, and a dedicated HA link port. Software Dell SonicWALL submitted the following software and/or firmware to ICSA Labs for this evaluation: SonicOS Enhanced Base operating system of the product. Testing began with SonicOS Enhanced version o and concluded with version o Dell SonicWALL Security Services Dynamic security updates for Intrusion Prevention, Gateway Anti-Virus, Content Filter, Anti-Spyware. Throughout testing the Security Services were updated before each test round. Testing completed with signatures dated on or before 20 April Documentation Dell SonicWALL submitted the following documentation to ICSA Labs for this evaluation: SonicOS Administrator s Guide Rev. D December 21, 2011 SonicOS 5.8 Application Control Feature Module January 10, 2011 SonicOS Log Events Reference Guide June 28, 2011 Product Family Description As this testing is not a certification engagement, there is no certified family. However, Dell SonicWALL chose to submit the E-Class NSA Series to the same rigorous selection process as performed for their ICSA Labs Network Firewall Enterprise Certification. A representative set of models was submitted for Page 3 of 31

6 testing and listed in the Hardware section above. In order to submit a family of products for certification, the vendor must attest that: The relevant components of the vendor software, including the functional software and the operating system software, are consistent across all members of the product family The components of the management interface(s) are consistent across all members of the product family Each member in the product family has an equivalent set of functionality Differences within the product family will in no way impact the testing differently among the product family members Any changes made to any members of the Product Family must be communicated to ICSA Labs in writing via or hardcopy when the said changes become applicable to the generally available product Family Members NSA E5500 NSA E6500 NSA E7500 NSA E8500 NSA E8510 Test Topology Introduction ICSA Labs designs individual test plans for each custom test in order to simulate a realistic deployment of products in a typical end user environment. Since products submitted for testing can often be configured many different ways, ICSA Labs frequently confronts many configuration-related decisions both before and after installing products under test. For the purposes of this engagement, ICSA Labs installed and configured the products as a typical end user would and according to their intended use. The provided documentation was used to assist with all configuration decisions. The final configuration used for testing is detailed within the Product Deployment section of this evaluation report. Test Description ICSA Labs deployed both products in the test infrastructure. This test infrastructure was designed to simulate an enterprise network deployment with an untrusted segment with potentially unknown users connected to services hosted on a trusted or protected segment as well as trusted users connecting outbound through another protected segment to services / applications on an untrusted segment (i.e. the Internet). Page 4 of 31

7 Test Bed Diagram Product Deployment Introduction Products can often be configured many different ways. Therefore, ICSA Labs frequently confronts many configuration-related decisions before ever adding a single security policy rule on a product in the lab. Decisions that ICSA Labs must make often include whether or not to use: Bridge versus router mode Proxied versus filtered network services NAT versus straight-thru (non-nat) mode for outbound services Straight-thru, port forwarding, or 1-to-1 public-to-private IP mapping for inbound services DNS servers on the product itself rather than at a separate host or ISP Additional network interfaces for server protection and network segregation Detailed Findings The E-Class NSA Series was previously configured for ICSA Labs Enterprise testing. The interface IP address configuration, DNS, WINS and syslog settings were updated to conform to changes made to the test bed to help with next generation firewall Page 5 of 31

8 testing. The diagnostics configuration was accessed by browsing to a specified URL and selecting Internal Settings. Under DHCP Settings the Enable DHCP Server Network Pre-Discovery setting was enabled. Under the Firewall Settings section, the Enable Tracking Bandwidth Usage for default traffic and Protect against TCP State Manipulation DoS settings were enabled. Under the Security Services Settings section the Apply IPS Signatures Bidirectionally setting was enabled. Under the Management Settings section the Allow SonicWALL Global Management System (SGMS) to preempt a logged in administrator setting was enabled. Under the Flow Reporting section the Enable Flow Reporting and Visualization, Enable Visualization UI for Non-Admin/Config users and Allow launching of AppFlow Monitor in a stand-alone browser frame settings were enabled. After these settings were updated they were saved by clicking Apply. The Internal Settings page was then closed by clicking the Close button and the admin user was returned to the main web UI. The next Page 6 of 31

9 step was to configure the device for logging under the Log -> Categories section. The Logging Level was set to Informational and the Alert Level was set to Error. The three logging methods were enabled for all categories by clicking the topmost checkboxes in each of the Log, Alerts and Syslog columns. Two syslog servers were then added by clicking on the Syslog section and clicking Add The IP addresses were added and the syslog port was left at its default setting of UDP port 514. Page 7 of 31

10 Flow reporting was then configured by clicking on the Flow Reporting section. The Send AppFlow and Real-Time Data To EXTERNAL Collector setting was enabled and the External Flow Reporting Format dropdown box was set to IPFIX with extensions. The IP address for the external collector was entered and the rest of the settings were left at their defaults. The settings were saved by clicking Accept on the top of the page. The product was then configured so all management functions needed would be available. Under System -> Administration -> Advanced Management, the Enable SNMP setting was enabled and then the Configure button was clicked. The desired SNMP settings were then entered and the OK button was clicked. Page 8 of 31

11 The Advanced Firewall features Enable Stealth Mode, Randomize IP ID, and Decrement IP TTL were enabled under Firewall Settings -> Advanced. The Accept button was then clicked to save the settings. The Dell SonicWALL E-Class NSA Series provided three methods to authenticate users which were a Local User database, a RADIUS server or an LDAP server. For this engagement, LDAP authentication was chosen so users could be authenticated using an existing Microsoft Active Directory configuration. When a user browsed to an Internet website, the connection was intercepted by the E-Class NSA device and the user was required to supply their Microsoft Windows credentials. This method would work well, however would require users to enter their credentials more than once. The E-Class NSA device supported a method that requires the user to enter their credentials only once and it was called Single Sign On (SSO). The Dell SonicWALL SSO solution consisted of two separate components, a software agent installed on a Microsoft Windows Domain Controller (DC) and an E-Class NSA device. The software agent was called the SonicWALL Directory Connector and version was installed on the DC. Detailed installation instructions were found in the vendor supplied documentation. Configuration continued after launching the SonicWALL Directory Connector as a user in the Administrator user group. The tree in the left windows pane was expanded and the SonicWALL Appliances section was clicked. To add a new appliance the green plus image was clicked. The appliance IP address, port and friendly name were entered followed by clicking on the Generate Key button to create a random shared key. This key must be entered into the appliance to complete the configuration. The settings were saved by clicking the OK button. The next step was to configure the Dell SonicWALL E-Class NSA appliance to query the Microsoft Windows Active Directory server (AD) under the Users -> Settings section. The Authentication method for login dropdown was set to LDAP and the Configure button was clicked to open the LDAP Page 9 of 31

12 Configuration dialog. The IP address was set to the DC, the port number was set to 389 and LDAP version 3 was selected. The Give login name/location in tree radio box was clicked and a username and password of a schema administrator was entered in the appropriate text boxes. The Test tab was clicked, the credentials listed above were entered and the Test button was clicked. The test was successful so the Test Status status field indicated LDAP authentication succeeded. The final step was to configure the Dell SonicWALL appliance to communicate with the SSO Agent installed on the DC. After returning to Users -> Settings, the drop-down labeled Single-sign-on method was clicked and SSO Agent was selected. The Configure button next to this drop-down was clicked. The Add button in the SonicWALL SSO Authentication Configuration dialog was clicked to add an authentication agent. The IP address and the port number that was previously configured when installing the SSO agent on the DC were entered. Note that this is not the same as the LDAP (389) port. Page 10 of 31

13 The shared key generated when adding the Dell SonicWALL appliance to the SSO agent on the DC was entered and the OK button was clicked. The SonicWALL SSO Authentication Configuration dialog was reopened to test the configuration. The Test tab was clicked and the Test button was clicked to check the connectivity between the Dell SonicWALL appliance and the SSO agent on the DC. The test was successful so the Test Status field indicated Agent is ready. Page 11 of 31

14 After these steps were completed, security policies could be applied to users or groups of users according to their membership in an Active Directory server. The App Rules functionality was the enabled under the Firewall -> App Rules section by clicking the Enable App Rule checkbox and clicking the Accept button. The App Control setting was also verified to be enabled under the Firewall -> App Control Advanced section. After the App Control feature was enabled, an App Rule Policy was created to control what applications users in the Marketing user group could access. A policy consists of a Match Object, an Action Object and Users/Groups. The Match Object was created under the Firewall -> Match Objects by clicking Add New Match Object. Applications that the policy was configured to match were added to the list in addition to a subset of individual functions of some applications. Page 12 of 31

15 When an application defined in the Match Object is detected the device will perform some action on the traffic. Which action is performed is determined by applying an Action Object, several of which are preconfigured and can be viewed under the Firewall -> Action Objects. The device was also configured to block websites with IP addresses registered in certain countries using the Geo-IP Filter security service. Under the Security Services -> Geo-IP Filter section the Block connections to/from following countries setting was enabled. The Enable Logging checkbox was clicked and the Russian Federation IP address space was selected for blocking. Page 13 of 31

16 There was an exception added for the IP address of St. Petersburg University s website by first creating the object under the Firewall -> Address Objects by clicking Add and selecting it as a Geo-IP Exclusion Object. The Accept button was then clicked to apply the newly configured Geo-IP Filter. The Content Filtering Service was configured next under the Security Services -> Content Filter section. The Configure button was then clicked to access the configuration dialog. The Policy tab was clicked followed by clicking the Edit icon next to the default policy. In the Edit CFS Policy dialog the URL List tab was clicked. A selection of categories was chosen by clicking their corresponding checkboxes and the settings were saved by clicking the OK button in the Edit CFS Policy dialog and the configuration dialog. Page 14 of 31

17 The Gateway Anti-Virus service was configured next under the Security Services -> Gateway Anti- Virus section. The Enable Gateway Anti-Virus setting was checked as well as all the checkboxes in the protocol inspection table. The settings were saved by clicking the Accept button. The Intrusion Prevention service was configured under the Security Services -> Intrusion Prevention section. The Enable IPS setting was checked as well as all the checkboxes in the signature group table. The settings were saved by clicking the Accept button. The next step was to configure the Intrusion Prevention settings and apply the service to a zone under the Security Services -> Intrusion Prevention section. The Enable IPS checkbox was clicked as well as the Prevent All and Detect All checkboxes for the High, Medium and Low categories of attacks. The settings were then applied under the Network -> Zones section by clicking the Configure button for the LAN. In the Edit Zone dialog the Enable IPS, Enable Gateway Anti-Virus Service, Enable Anti-Spyware Service and Enable App Control Service checkboxes were clicked and the settings were saved by clicking the OK button. After all configuration settings were applied, testing commenced. Page 15 of 31

18 Basic Firewall Functionality Introduction Basic or traditional firewall functionality is the first line of defense for next generation firewalls. Blocking network layer denial-of-service attacks and properly enforcing TCP state are just some of the many functions a traditional firewall can offer. Basic Firewall Functionality is covered by ICSA Labs Network Firewall Enterprise Module Certification. This certification program is the first of its kind combining the ICSA Labs Network Firewall Corporate Module Certification requirements with additional requirements such as High Availability functionality, Voice-over-IP functional security and IPv6 requirements. The Administrative and Logging requirements are also expanded upon along with making Time and Date Acquisition requirements mandatory to provide a full feature deployment scenario allowing for a higher level of real world testing. All requirements must be met with a single configuration making it one of the hardest certifications to achieve at the writing of this evaluation report. Detailed Findings This section of the evaluation was covered by the Dell SonicWALL E-Class NSA Series Network Firewall Enterprise Module Certification. For more information on this certification, please visit: /product/sonicwall-e-class-network-security-appliance-nsa-series High Availability Functionality Introduction Network firewalls have become a critical part of the infrastructure of almost every business, small/medium business to enterprise, today. A common deployment is at the network edge making firewalls the first line of defense as well as an integral part of Internet connectivity. To provide maximum uptime, network firewalls vendors offer high availability functionality. With high availability functionality, two (or more) network firewalls work together to allow the maximum uptime possible. ICSA Labs offers High Availability Functionality testing as an optional add-on module for Network Firewall Certification, as part of the Network Firewall Enterprise Module Certification and through custom testing engagements like this one. Detailed Findings This section of the evaluation was covered by the Dell SonicWALL E-Class NSA Series Network Firewall Enterprise Module Certification. For more information on this certification, please visit: /sites/default/files/sw_enterprise_ha.pdf Page 16 of 31

19 User-Based Authentication Introduction Granular access policies have existed since the birth of network firewalls. However, as more services have moved to the Internet, it has become important to improve upon granular access control from just source IP addresses / ports, destination IP addresses / ports and services / protocols. Also, to simplify network deployment strategies, enterprise networks have employed thin client environments leveraging technologies like Microsoft Terminal Services, Citrix Xen Desktop or VMware View allowing multiple concurrent users originating from a single host in a server room or data center. To apply a granular access policy in today s enterprise environment a new solution was needed. User-Based Authentication allows for this level of granularity and can be implemented in many different ways. All access requests are first authenticated to confirm the user requesting access. Once the identity of the user is validated, a custom access policy for this user (or a security group this user is a member of) is applied. With User-Based Authentication, an employee in the marketing department can have a different access policy than an employee in accounting or human resources allowing the enterprise to have better control over Internet access and a better overall view of network activity. Detailed Findings To test User-Based Authentication, the test infrastructure was expanded to include a Microsoft Windows Active Directory environment including multiple domain controllers for the E-Class NSA Series to connect to. As every enterprise network is different and can contain more than just one client operating system, multiple operating systems were used. For a full list of operating systems tested, please see below. To continue to simulate the enterprise, users were divided into three unique groups each having their own access policies. These users made access requests from multiple operating systems. Also, multiple users made access requests originating from a single operating system to confirm the E7500 could differentiate access requests originating from a single IP address. As outlined in the Product Deployment section of this evaluation report, the E-Class NSA Series was configured to use Dell SonicWALL s SSO Agent. This agent, at the time of this evaluation, worked primarily with Microsoft Windows operating systems. This allowed users on Windows 7 or Window XP making access requests to have a seamless authentication experience. To allow users to authenticate where the SSO Agent could not be installed, the E-Class NSA Series was configured to provide a captive portal. At this captive portal, users were prompted to enter their credentials so that the correct access policy could be applied. Below is a screenshot of the captive portal a user on Ubuntu was greeted with: Page 17 of 31

20 Once the user was authenticated, an additional popup window appeared outlining the time remaining in their authenticated session as well as offering the user the ability to extend their time remaining as well as logout of their authenticated session early. The E-Class NSA Series offered an interface inside the web UI where an administrative user could view the currently authenticated users. This also allowed for the administrative user to de-authenticate any currently authenticated user. Below is a screenshot showing the web UI outlining the currently authenticated users. Note the Logout column on the right. Page 18 of 31

21 Conclusion Dell SonicWALL E-Class NSA Series The E-Class NSA initially met and continued to meet all of the requirements outlined for this evaluation with respect to User-Based Authentication without improperly categorizing legitimate requests or causing false positives. Application Identification and Control Introduction Traditional firewall functionality inspected network communications at the network layer making decisions based on IP addresses, protocols and port numbers. As computer networks have become more sophisticated and more applications have been pushed into the cloud, more controlled / granular access policies are needed to monitor and secure today s corporate and enterprise networks than ever before. Traditionally, network communications over TCP port 80 and TCP port 443 were strictly for HTTP and HTTPS respectively however this is no longer true. Today s Internet uses these ports for everything from streaming media to instant messaging to file transfers as well as Web 2.0 applications making traditional firewalling unable to apply the granular access needed to secure enterprise networks. Using User-Based Authentication to identify users, granular access policy decisions can be made at the application layer to identify and control access to online applications. Restricting user access by identity to online applications allows enterprises to allow their marketing teams to social media without needing to allow the whole enterprise. Combining bandwidth management mechanisms, enterprises identify applications consuming large amounts of Internet bandwidth and limit their impact on business critical applications. Application Identification and Control puts the power back in the hands of the enterprise and allows a complete picture of network usage while applying granular access control to allow applications that are needed while limiting, or preventing, the ones that aren t. Detailed Findings To test Application Identification and Control, a test set was created to simulate enterprise user activity. This test set included hundreds of Internet based applications / websites that were accessed to measure the E-Class NSA Series ability to identify the application or website and control access to it based on the configured access policy. As outlined in the Product Deployment section of this report, the E-Class NSA Series was configured to authenticate users and apply a security policy, based on their identity, to limit access to categories of online applications as well as provide granular control functionality of allowed applications. Based on the user s identity, they were configured to be in one of three user groups (Finance, Marketing or IT). The Finance user group was only allowed access to business related websites and applications. The Marketing user group was allowed to everything Finance was allowed to as well as to social media websites and applications. However, the Marketing group was not allowed to use all features of popular social media websites and applications. As an example, the Marketing users were allowed to access Facebook to post status updates and use messaging to communicate with customers. However these users were not allowed to play the online games Facebook offers. The IT user group was allowed access to everything Finance and Marketing could access and there was no restriction on their Internet access. Page 19 of 31

22 As such, they were allowed to access online gaming, streaming media such as Pandora and Grooveshark, instant messaging such as Google Talk and web mail such as Yahoo and Gmail. IT users application utilization was still identified however no access, including items like Facebook games and Twitter posts, was prevented. A common way to help prove, or confirm, an online identity to a user is to employ SSL certificates. SSL certificates provide a way to communicate securely between a client and server as well as when signed by a trusted Certificate Authority, or CA, a user can validate the identity of the website or application attempting to communicate securely. However, a common practice to evade security access policies is to use encrypted communications to avoid inspection. Without the security product having an ability to decrypt this traffic and compare it to the configured security policy, a user who is to be protected or restricted to a location could use this technique to bypass the configured protection. To avoid detection and use Facebook, a Finance user could go to instead of To prevent this evasion, the E-Class NSA Series was configured to act as a man-in-the-middle for SSL connections. This meant that the client issuing the SSL connection would initiate a connection with the E-Class NSA Series, instead of the actual application it is trying to connect to, and the E-Class NSA Series would initate its own connection with the destination application. Using the DPI-SSL configuration, the E-Class NSA Series was configured to inspect all SSL client connections. Page 20 of 31

23 This allowed the E-Class NSA Series to decrypt all SSL connections, inspect the content, and compare it to the enforced security access policy. With the E-Class NSA Series in the middle of the SSL connections, users would receive SSL certificate errors. To adddress this, the E-Class NSA Series included a CA certificate that could be used and was added to the browsers and operating systems of the protected users. With the E-Class NSA configured as a trusted CA on the user computers, users could access applications using SSL without SSL certificate errors. Once configured, users protected by the E-Class NSA began to access the Internet from the operating systems identified in the User-Based Authentication section of this report. Users in all groups attempted to access applications and websites that they were allowed to as well as were denied to by the configured security policy. For example, Finance users attempted to access Facebook while Marketing users attempted multiple ways to access Mafia Wars and Farmville. While the Finance and Marketing users attempted to bypass the enforced policy, IT user access was confirmed to be identified however not improperly blocked. Also, bandwidth management was applied to certain categories of applications, such as peer-to-peer applications, limiting the amount of bandwidth they were allowed to consume. The E-Class NSA Series also included metadata about applications tested during this evaluation. Below is a screenshot of the metadata about YouTube. The E-Class NSA contained several different options for viewing application utilization other than viewing log events. One of these was the built-in AppFlow Monitor. This was located inside of the E-Class NSA Series web UI and allowed the administrative user to view current application usage by application or by user. An administrative user could also continue to drill down using the menu to select utilization of a specific user. Page 21 of 31

24 Conclusion The initial round of testing, noted by a 1 in the graph below, found the E-Class NSA Series initially 83.02% effective at identifying and controlling access to applications. This was accomplished without improper categorization of any applications tested. In general, the initial round of testing showed that the E- Class NSA Series was able to detect the majority of the applications tested however issues were discovered around identifying and controlling components of social media applications / websites thus impacting the overall effectiveness score. Dell SonicWALL was provided with a list of application identification and control misses and allowed to submit signature updates to improve their overall effectiveness. The E-Class NSA Series completed this part of the evaluation identifying and controlling 100% of the applications tested, noted by a 3 in the graph to the right, with no improper categorization. This included components of social media applications / websites. Furthermore, identifying and properly controlling bandwidth utilization was also successful. Page 22 of 31

25 User Protection Introduction The threat landscape has changed over the past several years putting enterprise users in the crosshairs of everyone from novice script kiddies to large organized crime syndicates. From the computers being used to the applications and services enterprises rely on each day, it is easy to understand why this has become such a large attack surface. Attacks enter the enterprise network in various forms. They can start as a downloaded piece of malware, a phishing with malicious attachments or URLs, or can be unsafe websites or locations on the Internet. Regardless of the entry point, these types of attacks can be combined together into rootkits giving the attacker, whether novice or expert, remote access to the protected network. This leads to everything from additional protected machines being compromised to sensitive data being stolen. To protect users, next generation firewalls need to be able to detect and prevent malware, exploits targeting application vulnerabilities and restrict access to undesirable locations. A combined approach, such as this, can provide overall security coverage for the enterprise. Detailed Findings To test User Protection, a collection of security concerns was tested. First, a test set of attacks targeting application vulnerabilities was developed. This set included remotely exploitable, high-severity vulnerabilities found in common user applications and operating systems spanning the last three years with all having a Common Vulnerability Scoring System, or CVSS, score of 7 (out of 10) or greater. The test set was weighted most heavily with vulnerabilities in software developed by popular developers including Microsoft, Adobe, Mozilla, and Oracle. To develop test cases for the vulnerabilities selected, a variety of sources was researched. The pie chart to the right outlines the distribution of the vulnerability set across software vendors. Before building any test cases, all vulnerabilities to be used as part of this, or any engagement, are researched and the vulnerable applications, services and operating systems are acquired. This allows for a better understanding of what is being tested as well as allows all test cases to be validated on the actual vulnerable target (i.e. application, service or operating system) and not rely on a third party tool to confirm exploitation. Once this research is completed and the actual target application, service or operating system is running in the lab, popular sources of information available online are used to retrieve proof-of-concept, or PoC, examples of how to exploit a target susceptible to the vulnerability in question. Sometimes, these PoC examples, as provided, do not actually exploit the vulnerable target. The ICSA Labs Vulnerability Research Team uses its knowledge of the vulnerability to correct the PoC example so that it works as expected. Next, commercial testing tools, such as Core Impact, are used to capture a baseline of commercial attacks available that exploit the vulnerable target. This includes open source projects such as Metasploit which have commercial support available. As the previous test cases are developed using publicly available, community based or commercially based attack test cases, a vendor could do their due diligence and download or buy any or all of the sources used prior to this stage of the test case development and provide exploit based coverage protection. To confirm the product under test is providing the fullest protection possible for the vulnerabilities being tested, ICSA Labs develops its own attacks targeting these vulnerabilities. As with all Page 23 of 31

26 of the previous test cases, these attacks are confirmed to exploit the vulnerability they are designed for by using the actual vulnerable target. Once this process is completed, the test set generation is complete. As outlined in the Product Deployment section of this report, the E-Class NSA Series was configured to detect and block for low, medium and high priority attacks. Once configured, a baseline of normal / legitimate traffic was run to confirm connectivity was functioning as expected. To confirm the configuration was correct, a baseline set of attacks targeting a small set of vulnerabilities (not included in the actual test set outlined above) was executed to confirm the E-Class NSA Series was detecting and preventing attacks as expected. With the configuration confirmed, testing commenced. A mixture of normal / legitimate traffic along with attack traffic (targeting the collection of vulnerabilities selected) was introduced. This normal traffic accessed similar functionality of the vulnerable applications and operating systems that the exploits targeted however did not include any attack vectors. Initially, the E-Class NSA Series was found to be 86.10% effective, as noted by the 1 in the graph to the left, at detecting and preventing attacks targeting the vulnerabilities selected. Dell SonicWALL was provided a list of vulnerability misses and allowed to provide signature updates to improve the overall security protection effectiveness. This evaluation completed with the E-Class NSA Series providing 91.70% security effectiveness, as noted by the 3 on the graph to the left, targeting user application and operating system vulnerabilities. Next, the ability to detect and prevent malware related attacks was tested. A collection of malware that was considered in-the-wild was used to create the test set. The test set was initially based off of the current traditional WildList and the current Extended WildList as well as other malware samples considered relevant by ICSA Labs. Starting with hundreds of samples as a base, the samples selected were replicated, where applicable, as well as verified to confirm that they were malicious in nature. If samples were polymorphic, they were replicated through multiple generations to confirm that the E-Class NSA could detect a variety of samples of polymorphic malware. When the test set generation process was completed, the test set included thousands of malware samples. As outlined in the Product Deployment section of this report, the E-Class NSA Series was configured to scan and block all malware related traffic for all protocols both inbound and outbound using the Gateway Anti-Virus settings. To test the malware effectiveness, a collection of files including both normal / legitimate files (i.e. known, not malicious files) as well as samples from the test set were downloaded through the E-Class NSA Series. All connections were confirmed to be terminated or, in the cases where the connection completed, any remnants downloaded were confirmed to not match the original sample and be modified in some sort of fashion. The bar graph below outlines how the E-Class NSA Series performed with respect to the original sample list (in blue) compared to the test set (in red) defined above. Page 24 of 31

27 During the initial round of testing, the E-Class NSA Series provided 83.22% coverage protection, as noted by the 1 in the chart to the below, for the malware test set and 99.59% against the original samples prior to the test set generation process. The reason for this discrepancy in the coverage protection revolves around polymorphic samples. The majority of the misses from the test set were polymorphic samples thus only accounting for a small number prior to replication in the original sample list. At the end of the initial round of testing, Dell SonicWALL was provided with a list of misses based on the original sample name on the original sample list as reported In-The-Wild. Once Dell SonicWALL confirmed the reported issues were addressed, the E-Class NSA Series was retested. This evaluation completed with the E-Class NSA Series providing % security effectiveness for malware protection. Lastly, the ability to restrict access to undesirable locations was tested. As outlined in the Product Deployment section of this report, the E-Class NSA Series was configured to block categories of websites based on their content as well as was block connections based on their geographical location. To build the undesirable locations test set, categories of websites were selected and URLs meeting the requirements were selected. Also, to add in additional undesirable location data, research was performed to find and validate services running in locations blocked via geographical location. Connection attempts were made to locations that should be allowed and should be denied based on the configured policy. Connection attempts included HTTP, DNS, SMTP and Telnet to ensure proper blocking across various service types. To the right is an example of what a user received when accessing a blocked website via the security access policy. This included connection attempts to locations that were in the geographically blocked location however were overridden, or whitelisted, via the configured policy. During the initial round of testing, the E-Class NSA Series provided 63.86% coverage protection, as noted by the 1 in the graph to the left, for undesirable locations. The majority of the coverage protection misses related to content filtering with the geographical location providing 100% coverage during the initial and subsequent testing rounds. Dell SonicWALL was provided with a list of misses along with a description of the configuration to help with their investigation. Once Dell SonicWALL confirmed the reported issues were addressed, the E-Class NSA Series was retested. This evaluation completed with the E-Class NSA Series providing % security effectiveness, as noted by the 3 in the graph to the left, for blocking undesirable locations. Page 25 of 31

28 Conclusion The areas discussed above, when combined, provide an overall view of User Protection. Application vulnerabilities, malware and undesirable locations are combined together to create exploit root kits as well as botnets. Failure to provide adequate coverage in one area may compromise the effectiveness of the others thus leaving users without proper protection. To conclude the overall view of the effectiveness of the E-Class NSA Series, all areas were combined into one single security effectiveness percentage. The initial round of testing showed the E-Class NSA Series provided 83.96% effectiveness, as noted by the 1 in the chart to the right, in providing User Protection. During the course of testing, Dell SonicWALL was able to provide updates to improve the overall effectivess at detecting and blocking attacks related to User Protection. This evaluation completed with the E-Class NSA Series providing 98.34% security effectiveness, as noted by the 3 in the chart to the right, for blocking attacks targeting users. This security effectiveness was accomplished without negatively impacting normal / legitimate traffic causing false positives. Server Protection Introduction Enterprises rely on network based communications to perform essential day to day operations of their businesses. These services are available on more than just the local area network, or LAN. Almost every business today uses technology to communicate with their customers, business partners or resellers as well as perform ecommerce transactions or manage their finances. To make this all possible, network servers are deployed offering enterprise users the ability to communicate with each other and the world at large. To allow the enterprise network to participate in the larger Internet community, enterprise servers are deployed in an Internet facing de-militarized zone, or DMZ, and accessed by potentially unknown Internet users. These public facing services can be anything from the web server hosting the enterprise s website to the enterprise s mail server and even application servers hosting ecommerce applications and Voice-over-IP, or VoIP, phone systems. Furthermore, as client operating systems and network deployments have become more sophisticated, any host on the network can operate as a server. This includes endpoint protection client software deployed on an employee laptop opening a TCP socket to listen for commands from the central administrative software to the client operating system, by default, listening for Windows SMB connections. Enterprises need to secure and monitor their networks. These organizations are faced with protecting themselves at several, often dissimilar, points in their networks against a myriad of threats. They need a means to not only block malicious attackers entering through perimeter Internet and wide area networks, or WAN, connections but also to prevent the exploitation of network resources by valid users who either unknowingly or purposefully introduce compromised equipment or exploits into the core of an organization s network. Page 26 of 31

29 Detailed Findings Dell SonicWALL E-Class NSA Series To test Server Protection, a collection of relevant vulnerabilities was selected. This set included remotely exploitable, high-severity vulnerabilities found in enterprise software spanning the past three years with all having a Common Vulnerability Scoring System, or CVSS, score of 7 or greater. The test set was weighted most heavily with vulnerabilities in software developed by popular developers including Microsoft, HP, Oracle, Symantec, CA, IBM, McAfee, Novell, etc. as well as important industry verticals like power (i.e. supervisory control and data acquisition or SCADA). To develop test cases for the vulnerabilities selected, a variety of sources was researched. The pie chart to the right outlines the distribution of the vulnerability set across software vendors. Following the methodology outlined in the User Protection for test set generation, a Server Protection test set was created targeting services and server operating systems. As outlined in the Product Deployment section of this report, the E-Class NSA Series was configured to detect and block for low, medium and high priority attacks. Once configured, a baseline of normal / legitimate traffic was run to confirm connectivity was functioning as expected. To confirm the configuration was correct, a baseline set of attacks targeting a small set of vulnerabilities (not included in the actual test set outlined above) were executed to confirm the E- Class NSA Series was detecting and preventing attacks as expected. With the configuration confirmed, testing commenced. A mixture of normal / legitimate traffic along with attack traffic (targeting the collection of vulnerabilities selected) was introduced. This normal traffic accessed similar components of the vulnerable systems that the exploits targeted however did not include any attack vectors. This mixture of traffic was repeated in both directions to confirm, regardless of direction, the E-Class NSA Series would provide protection. Conclusion During the initial round of testing, the E-Class NSA Series provided 82.40% security coverage protection, as noted by the 1 in the graph to the right, for the vulnerabilities selected. Dell SonicWALL was provided with a list of vulnerability misses, by Common Vulnerabilities and Exposures Identifier, or CVE ID, and allowed the opportunity to provide signature updates to improve on their Server Protection security coverage protection effectiveness. Once Dell SonicWALL confirmed the reported issues were addressed, the E-Class NSA Series was retested. This evaluation completed with the E-Class NSA Series providing 94.60% security coverage protection effectiveness, as noted by the 3 in the graph to the right, for the vulnerabilities selected. This high effectiveness was accomplished without negatively impacting normal / legitimate traffic causing false positives. Page 27 of 31

30 Miscellaneous Notes Introduction Factual observations, general notes, and/or specific comments collected during testing that did not fall neatly into one of the preceding sections are included below. Note that all observations and comments that follow may have no direct bearing on the evaluation results. The information is presented as it may be useful to enterprise end users. Detailed Findings The E-Class NSA Series performed well while blocking traffic to / from geographical locations deemed unsuitable, for the purposes of this engagement, for enterprise users. There was one observation made that created a race condition and could potentially allow content that was to be blocked to be allowed. When a new request was made to a geographical location that has not yet been accessed or has not been accessed for some time, the E-Class NSA Series would perform its lookup in parallel to the new connection. If the new connection completed before the E-Class NSA Series received a response from the Dell SonicWALL Security Services, the user making the request would not know this connection was to be blocked. This was possible to do if the connection was small like a DNS request. Also, if an HTTP request was made directly to a website that should have been blocked using its IP address instead of the fully qualified domain name, or FQDN, part of the content of the website would be displayed before the connection was terminated by the E-Class NSA Series. The E-Class NSA Series had several different methods of logging various categories of events. During this evaluation, three methods of logging were used. These were the local logging in the web UI, exporting log events using syslog and exporting the log events using external flow reporting via IPFIX. Syslog and external flow reporting relied on separate hosts that received events over a network connection. The local logging method stored events in some form of transient memory which could be viewed via the web UI. This memory could only hold a few hundred log events before it exhausted its resources and started overwriting data. For this reason, the majority of this evaluation relied on the external logging options as it was beneficial to have a larger logging storage. A CentOS workstation, provided by ICSA Labs, was used for receiving and storing logged events sent via syslog. To capture IPFIX data and report on it, Dell SonicWALL provided licensing for the Dell SonicWALL Scrutinizer with Flow Analytics which was installed onto a Windows VM provided by Dell SonicWALL. This product offered advanced reporting functionality not covered directly as part of this evaluation. For more information on it and its capabilities, please visit Dell SonicWALL s website. Below is a screenshot of Scrutinizer web UI. Page 28 of 31

31 To provide complete Application Identification and Control, a combination of App Rules and Content Filter Policies were needed. At the end of this evaluation, Application Identification and Control as well as content filtering testing completed with % security effectiveness. Summary Dell SonicWALL completed all tests included in this next generation firewall evaluation. During the course of this evaluation, Dell SonicWALL submitted signature updates which provided additional security and application identification coverage to improve their overall detection / prevention coverage effectiveness. The E-Class NSA Series completed this evaluation with the following functional and security effectiveness: Area of Evaluation Effectiveness User-Based Authentication % Application Identification and Control % User Protection 98.34% Server Protection 94.60% Dell SonicWALL was able to provide the above effectiveness while not impacting normal / legitimate, traffic thus causing false positives. Page 29 of 31

32 Partners and Resources Introduction This evaluation was made possible through the use of ICSA Labs partnerships, commercial tools, open source tools and resources available on the Internet. The following is a list of partnerships, tools and resources used during this evaluation. Commercial Partnerships Open Source Projects / Other Commercial Tools Other Research Sources Page 30 of 31

33 Testing Information This report is issued by the authority of the Managing Director, ICSA Labs. Tests are done under normal operating conditions. Please visit for the most current information about this and other products. Lab Report Date 17 July 2012 Test Location ICSA Labs 1000 Bent Creek Blvd., Suite 200 Mechanicsburg, PA Product Developer s Headquarters Dell SonicWALL 2001 Logic Drive San Jose, CA The test methods used to produce this report are accredited and meet the requirements of ISO/IEC as verified by the ANSI-ASQ National Accreditation Board/ACLASS. Refer to certificate and scope of accreditation number AT Testing reports shall not be reproduced except in full, without prior written approval of ICSA Labs. All other product, brand and company names in this document are trademarks or registered trademarks of their respective companies. Page 31 of 31