2013 Thomas Skybakmoen, Francisco Artes, Bob Walder, Ryan Liles



Similar documents
2013 Thomas Skybakmoen, Francisco Artes, Bob Walder, Ryan Liles

How To Create A Firewall Security Value Map (Svm) 2013 Nss Labs, Inc.

NEXT GENERATION FIREWALL COMPARATIVE ANALYSIS

DATA CENTER IPS COMPARATIVE ANALYSIS

WEB APPLICATION FIREWALL COMPARATIVE ANALYSIS

DATA CENTER IPS COMPARATIVE ANALYSIS

SSL Performance Problems

How To Get A Fortinet Security System For Free

ENTERPRISE EPP COMPARATIVE REPORT

Internet Advertising: Is Your Browser Putting You at Risk?

WEB APPLICATION FIREWALL PRODUCT ANALYSIS

NEXT GENERATION FIREWALL COMPARATIVE ANALYSIS

Can Consumer AV Products Protect Against Critical Microsoft Vulnerabilities?

Breach Found. Did It Hurt?

Evolutions in Browser Security

Multiple Drivers For Cyber Security Insurance

TEST METHODOLOGY. Hypervisors For x86 Virtualization. v1.0

An Old Dog Had Better Learn Some New Tricks

Mobile App Containers: Product Or Feature?

BROWSER SECURITY COMPARATIVE ANALYSIS

Internet Explorer Exploit Protection ENTERPRISE BRIEFING REPORT

The CISO s Guide to the Importance of Testing Security Devices

CORPORATE AV / EPP COMPARATIVE ANALYSIS

TEST METHODOLOGY. Distributed Denial-of-Service (DDoS) Prevention. v2.0

Customer Advantage Program

TEST METHODOLOGY. Network Firewall Data Center. v1.0

Next-Generation Firewalls: CEO, Miercom

TEST METHODOLOGY. Endpoint Protection Evasion and Exploit. v4.0

Secure Upgrade Plus. Customer Advantage Program. Secure Upgrade Plus. Customer Loyalty Bundles. Address other business needs

Is the Security Industry Ready for SSL Decryption?

TEST METHODOLOGY. Web Application Firewall. v6.2

How To Sell Security Products To A Network Security Company

Compliance in the Age of Cloud

TEST METHODOLOGY. Data Center Firewall. v2.0

Why Is DDoS Prevention a Challenge?

Full version is >>> HERE <<<

Remote Firewall Deployment

FortiGuard Security Services

Achieve Deeper Network Security

Global Network Security Appliance Market

NETWORK INTRUSION PREVENTION SYSTEM PRODUCT ANALYSIS

Windows Hard Disk Encryption

Dell One Identity Cloud Access Manager How to Configure vworkspace Integration

Commonwealth of Virginia Security and Search Network Appliances. IFB Exhibit E - Pricing. Product Price List. phone: (301)

Dell One Identity Cloud Access Manager SonicWALL Integration Overview

GENERAL TARIFF. 2) Internet Service is provided at C&W's discretion, depending on the availability of suitable facilities and equipment.

How To Test A Ddos Prevention Solution

Radius Integration Guide Version 9

NEXT GENERATION FIREWALL PRODUCT ANALYSIS

Adaptec: Snap Server NAS Performance Study

Achieve Deeper Network Security and Application Control

jetnexus Accelerating Load Balancer Extreme (ALB-X) 2.0 Features Update Report

Securing Amazon It s a Jungle Out There

Monitor free disc space on a server. AdRem NetCrunch 6.x Tutorial

Symantec Endpoint Protection 11.0 Architecture, Sizing, and Performance Recommendations

Using NetCrunch for compliance and security audits

What to Look for When Evaluating Next-Generation Firewalls

Web Security Firewall Setup. Administrator Guide

How To Create A Report For Bandwidth Utilization On An Adrem Netcrone 6.X.X (Netcrone) On A Network With A Network (Netcon) On An Ipad Or Ipad (Netcra) On Your

Dell Advanced Network Monitoring Services Service Description

Analysis of the Global Unified Threat Management (UTM) Market Enterprise Features and Product Value Propel Market Growth

NETWORK FIREWALL TEST METHODOLOGY 3.0. To receive a licensed copy or report misuse, Please contact NSS Labs at: or advisor@nsslabs.

WiFiSurvey Using AirPort Utility for WiFi Scanning Guide

Dell InTrust Preparing for Auditing Cisco PIX Firewall

AN INSIDE LOOK AT S&P MILA 40

Web Application Firewalls: The TCO Question

Main Findings. 1. Microsoft s Windows Server 2003 enterprise license and support costs are competitive with Red Hat Enterprise Linux.

E-Sign Disclosure & E-Statements Terms and Conditions

Global Enterprise Network Firewall Market

NEXT GENERATION FIREWALL TEST REPORT

ADP Ambassador /Referral Rewards Program. Terms and Conditions of Use

Applaud Solutions Technical Support Policies

The Nuts and Bolts of Fixed Indexed Annuities

NEXT GENERATION FIREWALL PRODUCT ANALYSIS

A Closer Look at Interest Rate Floors

Single Stock Trade Idea Generation and Strategy. Greg Harmon, CMT, President and Founder Dragonfly Capital Management LLC Dragonflycap.

TestFlight FAQ Apple Inc.

Spotlight Management Pack for SCOM

2013 North America Auto Insurance Pricing Benchmark Survey Published by

Symantec Critical System Protection Agent Event Viewer Guide

GENOA, a QOL HEALTHCARE COMPANY WEBSITE TERMS OF USE

2016 Firewall Management Trends Report

PocketSuite Terms of Service. Last modified: November 2015

NEXT GENERATION INTRUSION PREVENTION SYSTEM (NGIPS) TEST REPORT

Compatibility Matrix. VPN Authentication by BlackBerry. Version 1.7.1

END USER LICENSE AGREEMENT

Is Hyperconverged Cost-Competitive with the Cloud?

Dell One Identity Cloud Access Manager How To Deploy Cloud Access Manager in a Virtual Private Cloud

ASYMMETRIC DIGITAL SUBSCRIBER LINE INTERNET ACCESS TERMS

Transcription:

FIREWALL COMPARATIVE ANALYSIS Total Cost of Ownership (TCO) 2013 Thomas Skybakmoen, Francisco Artes, Bob Walder, Ryan Liles Tested s Barracuda F800, Check Point 12600, Cyberoam CR2500iNG, Dell SonicWALL NSA 4500, Fortinet FortiGate- 800c, Juniper SRX550, NETASQ NG1000- A, NETGEAR ProSecure UTM9S, Palo Alto Networks PA- 5020, Sophos UTM 425, Stonesoft FW- 1301, WatchGuard XTM 1050

Overview The implementation of firewall solutions can be complex, with multiple factors affecting the overall cost of deployment, maintenance and upkeep. All of these should be considered over the course of the useful life of the solution, including: 1. Acquisition costs for firewall devices and central management system 2. Fees paid to the vendor for annual maintenance and support 3. Labor costs for installation, maintenance and upkeep By using total cost of ownership (TCO) instead of purchase price, it is possible to factor in management of the device via labor costs associated with product installation, maintenance and upkeep. In addition, no two network security products deliver the same security effectiveness or throughput, making apples to apples comparisons extremely difficult. In order to capture the relative value of devices on the market and facilitate such comparisons, NSS Labs has developed a unique metric to enable value- based comparisons: TCO per protected megabit/sec. NSS Labs defines this metric as: Total Cost / (security effectiveness * throughput). Since management capabilities are also critical in the successful deployment of security devices, a second metric is also used that multiplies the security effectiveness score by the score for enterprise management capabilities as tested. Security Effectiveness = Firewall Policy Protection x Stability and Reliability x Evasion x Leakage Weighted Security Effectiveness = Security Effectiveness x Management Capabilities Figure 1 Security Effectiveness Formulae For the purpose of this analysis, NSS Labs developed an enterprise use case with one (1) central management system and ten (10) firewall devices deployed at disparate locations. Since configuration is performed via central management, the device cost reflects initial setup and upkeep per device. Actual Tested Throughput Weighted Security Effectiveness 3 Year TCO TCO / Protected- Mbps Barracuda F800 7,827 $211,169 76% $628,616 $11 Check Point 12600 8,400 $623,300 100% $1,078,793 $13 Cyberoam CR2500iNG 8,733 $204,439 55% $887,101 $18 Dell SonicWALL NSA 4500 850 $56,945 95% $278,826 $35 Fortinet FortiGate- 800c 9,667 $105,973 100% $371,427 $4 Juniper SRX550 2,127 $141,700 100% $424,540 $20 NETASQ NG1000- A 2,540 $159,701 70% $476,447 $27 NETGEAR ProSecure UTM9S 231 $3,670 4% $578,020 $5,954 Palo Alto Networks PA- 5020 4,120 $412,500 95% $941,001 $24 Sophos UTM 425 3,000 $84,700 46% $593,800 $44 Stonesoft FW- 1301 5,147 $350,420 100% $658,090 $13 WatchGuard XTM 1050 2,200 $235,950 85% $756,360 $40 Figure 2 Total Cost of Ownership 2013 NSS Labs, Inc. All rights reserved. 2

Table of Contents Analysis... 4 Labor and Equipment Costs... 4 Labor for Central Management... 4 Labor for Device Setup and Upkeep... 5 Equipment and Software Costs... 5 Total Cost of Ownership... 6 Normalizing the Data... 6 Price (Vendor- Claimed Performance)... 7 Total Cost of Ownership (Vendor- Claimed Throughput)... 7 Factor In Security Protection... 8 Total Cost of Ownership With Actual Tested Throughput... 9 Determining Value... 10 Weighted Value... 11 Test Methodology... 12 Contact Information... 12 Table of Figures Figure 1 Security Effectiveness Formulae... 2 Figure 2 Total Cost of Ownership... 2 Figure 3 Labor Required for Central Management... 4 Figure 4 - Labor Required per Firewall Device... 5 Figure 5 Equipment and Software Costs... 5 Figure 6 Year 1 Total Cost of Ownership... 6 Figure 7 Price (Vendor- Claimed Throughput)... 7 Figure 8 TCO with Vendor- Claimed Throughput... 7 Figure 9 Price per Protected- Mbps... 8 Figure 10 TCO with Vendor- Claimed Throughput... 8 Figure 11 Claimed vs. Actual Throughput (Mbps)... 9 Figure 12 TCO/Protected- Mbps... 9 Figure 13 Weighted TCO/Protected- Mbps... 10 Figure 14 Value Based Upon Price... 10 Figure 15 Value Based Upon TCO... 11 Figure 16 Comparison of Price to Weighted Value... 11 2013 NSS Labs, Inc. All rights reserved. 3

Analysis Labor and Equipment Costs Implementation of firewalls can be a complex process with multiple factors affecting the overall cost of deployment, maintenance and upkeep. With the shortage of skilled and experienced practitioners, it is important to consider the required time and resources to properly install and maintain the solution. Failure to do so could result in products not achieving their full security potential. There are two main components to be considered: 1. Installation (capital expenditure/capex) the time required to take the device out of the box, configure it, put it into the network, apply updates and patches, perform initial tuning, and configure desired logging and reporting. 2. Upkeep (operating expenditure/opex) the time required to apply periodic updates and patches from vendors, including hardware, software, and protection (filter/rules) updates. Labor for Central Management Labor costs for central management refer to day- to- day management tasks, including device management, policy configuration and deployment, alert handling, reporting, and so on. In a typical enterprise, administrators will use some form of centralized management system to manage deployed firewall sensors. Without central management, which is only an option with some vendors, it would be necessary to extrapolate these hours (and thus increase costs) by multiplying them by the number of deployed firewall sensors. The variation in installation/setup and upkeep time reflects the efficiency of each management system. Initial Setup Time (Hours) Day- to- Day Management (Hours per Year) Barracuda F800 12 1000 Check Point 12600 8 500 Cyberoam CR2500iNG 8 2500 Dell SonicWALL NSA 4500 8 750 Fortinet FortiGate- 800c 8 750 Juniper SRX550 8 1000 NETASQ NG1000- A 8 1000 NETGEAR ProSecure UTM9S 0 0 Palo Alto Networks PA- 5020 8 1500 Sophos UTM 425 8 2000 Stonesoft FW- 1301 8 500 WatchGuard XTM 1050 12 1750 Figure 3 Labor Required for Central Management 2013 NSS Labs, Inc. All rights reserved. 4

Labor for Device Setup and Upkeep This table estimates the annual labor required to maintain each device. Even with a central management system, it is still necessary for administrators to work on the individual physical devices during the course of the year for troubleshooting and remediation purposes. NSS assumptions are based upon the time required by an experienced security engineer ($75 per hour fully loaded), allowing us to hold constant the talent cost, and measure only the difference in time required for installation and upkeep. Initial Setup Time (Hours) Upkeep (Hours per Year) Barracuda F800 12 9 Check Point 12600 8 8 Cyberoam CR2500iNG 8 12 Dell SonicWALL NSA 4500 8 8 Fortinet FortiGate- 800c 8 9 Juniper SRX550 8 8 NETASQ NG1000- A 8 9 NETGEAR ProSecure UTM9S 8 250 Palo Alto Networks PA- 5020 8 12 Sophos UTM 425 8 10 Stonesoft FW- 1301 8 8 WatchGuard XTM 1050 8 9 Equipment and Software Costs Figure 4 - Labor Required per Firewall Device All product costs are based on list prices as provided to NSS researchers by vendors. Actual costs to end- users may be lower depending on the negotiated discount. However, assuming all vendors will provide similar discounts, the cost ratios will remain constant. Initial (hardware as tested) Initial (enterprise management system) Annual Cost of Maintenance & Support (hardware/software) Annual Cost of Maintenance & Support (enterprise management) Barracuda F800 $20,597 $5,199 $5,266 $1,439 Check Point 12600 $61,200 $11,300 $10,404 $2,091 Cyberoam CR2500iNG $19,999 $4,449 $2,752 $1,334 Dell SonicWALL NSA 4500 $4,995 $6,995 $918 $327 Fortinet FortiGate- 800c $9,998 $5,993 $2,188 $1,405 Juniper SRX550 $11,000 $31,700 $627 $4,810 NETASQ NG1000- A $14,985 $9,851 $1,999 $1,642 NETGEAR ProSecure UTM9S $367 $0 $195 $0 Palo Alto Networks PA- 5020 $40,000 $12,500 $5,120 $1,267 Sophos UTM 425 $8,470 $0 $1,000 $0 Stonesoft FW- 1301 $33,592 $14,500 $5,310 $3,760 WatchGuard XTM 1050 $23,595 $0 $3,317 $0 Figure 5 Equipment and Software Costs 2013 NSS Labs, Inc. All rights reserved. 5

Total Cost of Ownership TCO incorporates both CAPEX and OPEX costs over a three- year period. This includes initial acquisition and deployment costs, plus annual maintenance and update costs (software and hardware updates) and all associated labor costs. Upkeep labor includes day- to- day management, patching/updating and troubleshooting. Calculations are as follows: Value Description of Calculation Year One Cost Initial Price + Maintenance + ((Installation + Upkeep) x Labor rate $/hr) Year Two Cost Maintenance + (Upkeep x Labor rate $/hr) Year Three Cost Maintenance + (Upkeep x Labor rate $/hr) Three Year TCO Year One Cost + Year Two Cost + Year Three Cost Calculations are based on a Labor rate of $75 per hour (fully loaded) as well as vendor- provided pricing information. Where possible, the 24/7 maintenance and support option with 24- hour replacement is utilized since this is the option typically selected by enterprise customers. Prices include an enterprise management solution to manage up to ten firewall devices. Maintenance / year Year 1 Cost Year 1 labor Cost 1 Year TCO Barracuda F800 $211,169 $54,099 $265,268 $91,650 $356,918 Check Point 12600 $623,300 $106,131 $729,431 $50,100 $779,531 Cyberoam CR2500iNG $204,439 $28,854 $233,293 $203,100 $436,393 Dell SonicWALL NSA 4500 $56,945 $9,510 $66,455 $68,850 $135,305 Fortinet FortiGate- 800c $105,973 $23,285 $129,258 $69,600 $198,858 Juniper SRX550 $141,700 $11,080 $152,780 $87,600 $240,380 NETASQ NG1000- A $159,701 $21,632 $181,333 $88,350 $269,683 NETGEAR ProSecure UTM9S $3,670 $1,950 $5,620 $193,500 $199,120 Palo Alto Networks PA- 5020 $412,500 $52,467 $464,967 $128,100 $593,067 Sophos UTM 425 $84,700 $10,000 $94,700 $164,100 $258,800 Stonesoft FW- 1301 $350,420 $56,857 $407,277 $50,100 $457,377 WatchGuard XTM 1050 $235,950 $33,170 $269,120 $144,900 $414,020 Figure 6 Year 1 Total Cost of Ownership Normalizing the Data There are multiple methods that can be used to determine value. The benefit of this analysis is that, within a given performance range, it will provide insight as to whether a product is priced above or below the majority of its competitors. A high price could indicate a premium based upon protection offered, brand recognition, level of customer service, or a price penalty for an underperforming product. 2013 NSS Labs, Inc. All rights reserved. 6

Price (Vendor- Claimed Performance) The most simplistic means of determining value, but frequently misleading, is determining the price per megabit per second based upon the initial purchase price of the product and the performance claims of the vendor. Vendor- Claimed Throughput Price/Mbps Barracuda F800 9,200 $211,169 $2.30 Check Point 12600 10,000 $623,300 $6.23 Cyberoam CR2500iNG 28,000 $204,439 $0.73 Dell SonicWALL NSA 4500 990 $56,945 $5.75 Fortinet FortiGate- 800c 20,000 $105,973 $0.53 Juniper SRX550 5,500 $141,700 $2.58 NETASQ NG1000- A 7,000 $159,701 $2.28 NETGEAR ProSecure UTM9S 850 $3,670 $0.43 Palo Alto Networks PA- 5020 5,000 $412,500 $8.25 Sophos UTM 425 6,000 $84,700 $1.41 Stonesoft FW- 1301 5,000 $350,420 $7.01 WatchGuard XTM 1050 10,000 $235,950 $2.36 Total Cost of Ownership (Vendor- Claimed Throughput) Figure 7 Price (Vendor- Claimed Throughput) A more sophisticated approach involves determining the price per megabit per second based upon the total cost of ownership of the product. This calculation is performed in many purchasing departments. Unfortunately, this approach is equally flawed, since it relies on the vendor- claimed performance, without testing, to determine the actual throughput of the device under real- world conditions. Vendor- Claimed Throughput Price 3 Year TCO TCO / Mbps Barracuda F800 9,200 $211,169 $628,616 $6.83 Check Point 12600 10,000 $623,300 $1,078,793 $10.79 Cyberoam CR2500iNG 28,000 $204,439 $887,101 $3.17 Dell SonicWALL NSA 4500 990 $56,945 $278,826 $28.16 Fortinet FortiGate- 800c 20,000 $105,973 $371,427 $1.86 Juniper SRX550 5,500 $141,700 $424,540 $7.72 NETASQ NG1000- A 7,000 $159,701 $476,447 $6.81 NETGEAR ProSecure UTM9S 850 $3,670 $578,020 $68.00 Palo Alto Networks PA- 5020 5,000 $412,500 $941,001 $18.82 Sophos UTM 425 6,000 $84,700 $593,800 $9.90 Stonesoft FW- 1301 5,000 $350,420 $658,090 $13.16 WatchGuard XTM 1050 10,000 $235,950 $756,360 $7.56 Figure 8 TCO with Vendor- Claimed Throughput 2013 NSS Labs, Inc. All rights reserved. 7

Factor In Security Protection Determining value solely upon TCO and throughput is acceptable when dealing with a pure networking device. However, for security devices, protection also needs to be factored into the equation. This table determines the protected price per megabit per second based upon purchase price, vendor- claimed performance, and protection rating (based upon test results). Vendor- Claimed Throughput Security Effectiveness Price / Protected- Mbps Barracuda F800 9,200 $211,169 80.0% $2.87 Check Point 12600 10,000 $623,300 100.0% $6.23 Cyberoam CR2500iNG 28,000 $204,439 100.0% $0.73 Dell SonicWALL NSA 4500 990 $56,945 100.0% $5.75 Fortinet FortiGate- 800c 20,000 $105,973 100.0% $0.53 Juniper SRX550 5,500 $141,700 100.0% $2.58 NETASQ NG1000- A 7,000 $159,701 70.0% $3.26 NETGEAR ProSecure UTM9S 850 $3,670 28.0% $1.54 Palo Alto Networks PA- 5020 5,000 $412,500 100.0% $8.25 Sophos UTM 425 6,000 $84,700 70.0% $2.02 Stonesoft FW- 1301 5,000 $350,420 100.0% $7.01 WatchGuard XTM 1050 10,000 $235,950 100.0% $2.36 Figure 9 Price per Protected- Mbps The following table determines the TCO per protected Mbps based upon the 3- year TCO, vendor claimed performance, and protection rating (based upon test results). Vendor- Claimed Throughput Security Effectiveness 3 Year TCO TCO / Protected- Mbps Barracuda F800 9,200 $211,169 80.0% $628,616 $8.54 Check Point 12600 10,000 $623,300 100.0% $1,078,793 $10.79 Cyberoam CR2500iNG 28,000 $204,439 100.0% $887,101 $3.17 Dell SonicWALL NSA 4500 990 $56,945 100.0% $278,826 $28.16 Fortinet FortiGate- 800c 20,000 $105,973 100.0% $371,427 $1.86 Juniper SRX550 5,500 $141,700 100.0% $424,540 $7.72 NETASQ NG1000- A 7,000 $159,701 70.0% $476,447 $9.72 NETGEAR ProSecure UTM9S 850 $3,670 28.0% $578,020 $242.87 Palo Alto Networks PA- 5020 5,000 $412,500 100.0% $941,001 $18.82 Sophos UTM 425 6,000 $84,700 70.0% $593,800 $14.14 Stonesoft FW- 1301 5,000 $350,420 100.0% $658,090 $13.16 WatchGuard XTM 1050 10,000 $235,950 100.0% $756,360 $7.56 Figure 10 TCO with Vendor- Claimed Throughput Vendor performance claims are frequently exaggerated in marketing materials, or simply fail to take into account real- world deployment conditions. Knowing that, many enterprise IT professionals will over- purchase based on performance claims to ensure adequate performance headroom. Below is a chart of vendor- claimed throughput vs. actual throughput. 2013 NSS Labs, Inc. All rights reserved. 8

Claimed Actual % Delta Barracuda F800 9,200 7,827-15% Check Point 12600 10,000 8,400-16% Cyberoam CR2500iNG 28,000 8,733-69% Dell SonicWALL NSA 4500 990 850-14% Fortinet FortiGate- 800c 20,000 9,667-52% Juniper SRX550 5,500 2,127-61% NETASQ NG1000- A 7,000 2,540-64% NETGEAR ProSecure UTM9S 850 231-73% Palo Alto Networks PA- 5020 5,000 4,120-18% Sophos UTM 425 6,000 3,000-50% Stonesoft FW- 1301 5,000 5,147 3% WatchGuard XTM 1050 10,000 2,200-78% Figure 11 Claimed vs. Actual Throughput (Mbps) Total Cost of Ownership With Actual Tested Throughput Improving on the prior tables, the formula now determines the price per megabit based upon the TCO of the product and the actual performance of the product based upon NSS test results. The following table illustrates the clear difference between measuring TCO with vendor- claimed performance and measuring TCO with protected Mbps (as a security device). Security effectiveness factors in how well the device under test (DUT) resisted evasion attempts, the stability of the DUT, and whether or not the DUT leak attacks under stress. Once again, this is important because we are dealing with in- line security devices, and each of these factors can have a serious impact on security protection or business continuity. Vendor- Claimed Throughput Actual Tested Throughput Security Effectiveness 3 Year TCO TCO / Protected- Mbps Barracuda F800 9,200 7,827 $211,169 80.0% $628,616 $10.04 Check Point 12600 10,000 8,400 $623,300 100.0% $1,078,793 $12.84 Cyberoam CR2500iNG 28,000 8,733 $204,439 100.0% $887,101 $10.16 Dell SonicWALL NSA 4500 990 850 $56,945 100.0% $278,826 $32.80 Fortinet FortiGate- 800c 20,000 9,667 $105,973 100.0% $371,427 $3.84 Juniper SRX550 5,500 2,127 $141,700 100.0% $424,540 $19.96 NETASQ NG1000- A 7,000 2,540 $159,701 70.0% $476,447 $26.80 NETGEAR ProSecure UTM9S 850 231 $3,670 28.0% $578,020 $893.15 Palo Alto Networks PA- 5020 5,000 4,120 $412,500 100.0% $941,001 $22.84 Sophos UTM 425 6,000 3,000 $84,700 70.0% $593,800 $28.28 Stonesoft FW- 1301 5,000 5,147 $350,420 100.0% $658,090 $12.79 WatchGuard XTM 1050 10,000 2,200 $235,950 100.0% $756,360 $34.38 Figure 12 TCO/Protected- Mbps 2013 NSS Labs, Inc. All rights reserved. 9

In this final step, a weighting is applied to reflect the functional completeness of an enterprise management system (see the Management Comparative Analysis Report for further details of this testing.) The management score is multiplied by the security effectiveness score to produce the final management- weighted security effectiveness score as shown in Figure 14. Actual Tested Throughput Weighted Security Effectiveness 3 Year TCO TCO / Protected- Mbps Barracuda F800 7,827 $211,169 76.0% $628,616 $10.57 Check Point 12600 8,400 $623,300 100.0% $1,078,793 $12.84 Cyberoam CR2500iNG 8,733 $204,439 55.0% $887,101 $18.47 Dell SonicWALL NSA 4500 850 $56,945 95.0% $278,826 $34.53 Fortinet FortiGate- 800c 9,667 $105,973 100.0% $371,427 $3.84 Juniper SRX550 2,127 $141,700 100.0% $424,540 $19.96 NETASQ NG1000- A 2,540 $159,701 70.0% $476,447 $26.80 NETGEAR ProSecure UTM9S 231 $3,670 4.2% $578,020 $5,954.30 Palo Alto Networks PA- 5020 4,120 $412,500 95.0% $941,001 $24.04 Sophos UTM 425 3,000 $84,700 45.5% $593,800 $43.50 Stonesoft FW- 1301 5,147 $350,420 100.0% $658,090 $12.79 WatchGuard XTM 1050 2,200 $235,950 85.0% $756,360 $40.45 Figure 13 Weighted TCO/Protected- Mbps Determining Value The following tables show how the actual value of a product can change dramatically as tested performance and security effectiveness are factored in. Claimed Performance Actual Performance Actual Performance + Weighted Protection Price/Mbps Price/Protected- Mbps Price/Protected- Mbps Price/Protected- Mbps Barracuda F800 $2.30 $2.87 $3.37 $3.55 Check Point 12600 $6.23 $6.23 $7.42 $7.42 Cyberoam CR2500iNG $0.73 $0.73 $2.34 $4.26 Dell SonicWALL NSA 4500 $5.75 $5.75 $6.70 $7.05 Fortinet FortiGate- 800c $0.53 $0.53 $1.10 $1.10 Juniper SRX550 $2.58 $2.58 $6.66 $6.66 NETASQ NG1000- A $2.28 $3.26 $8.98 $8.98 NETGEAR ProSecure UTM9S $0.43 $1.54 $5.67 $37.81 Palo Alto Networks PA- 5020 $8.25 $8.25 $10.01 $10.54 Sophos UTM 425 $1.41 $2.02 $4.03 $6.21 Stonesoft FW- 1301 $7.01 $7.01 $6.81 $6.81 WatchGuard XTM 1050 $2.36 $2.36 $10.73 $12.62 Figure 14 Value Based Upon Price 2013 NSS Labs, Inc. All rights reserved. 10

Claimed Performance Actual Performance Actual Performance + Weighted Protection TCO/Mbps TCO/Protected- Mbps TCO/Protected- Mbps TCO/Protected- Mbps Barracuda F800 $6.83 $8.54 $10.04 $10.57 Check Point 12600 $10.79 $10.79 $12.84 $12.84 Cyberoam CR2500iNG $3.17 $3.17 $10.16 $18.47 Dell SonicWALL NSA 4500 $28.16 $28.16 $32.80 $34.53 Fortinet FortiGate- 800c $1.86 $1.86 $3.84 $3.84 Juniper SRX550 $7.72 $7.72 $19.96 $19.96 NETASQ NG1000- A $6.81 $9.72 $26.80 $26.80 NETGEAR ProSecure UTM9S $68.00 $242.87 $893.15 $5,954.30 Palo Alto Networks PA- 5020 $18.82 $18.82 $22.84 $24.04 Sophos UTM 425 $9.90 $14.14 $28.28 $43.50 Stonesoft FW- 1301 $13.16 $13.16 $12.79 $12.79 WatchGuard XTM 1050 $7.56 $7.56 $34.38 $40.45 Weighted Value Figure 15 Value Based Upon TCO This compares the vendor stated value metric with that generated from the NSS test results. E.g. Price / Vendor- Claimed Performance vs. Price / Protected and Validated Performance. Claimed Performance Price/Mbps Actual Performance + Weighted Protection Price / Protected- Mbps Weighted Value Delta % Delta Barracuda F800 $2.30 $3.55 $211,169 $412,235 $201,066 95% Check Point 12600 $6.23 $7.42 $623,300 $582,149 ($41,151) - 7% Cyberoam CR2500iNG $0.73 $4.26 $204,439 $332,888 $128,449 63% Dell SonicWALL NSA 4500 $5.75 $7.05 $56,945 $55,963 ($982) - 2% Fortinet FortiGate- 800c $0.53 $1.10 $105,973 $669,933 $563,960 532% Juniper SRX550 $2.58 $6.66 $141,700 $147,385 $5,685 4% NETASQ NG1000- A $2.28 $8.98 $159,701 $123,222 ($36,479) - 23% NETGEAR ProSecure UTM9S $0.43 $37.81 $3,670 $673 ($2,997) - 82% Palo Alto Networks PA- 5020 $8.25 $10.54 $412,500 $271,254 ($141,246) - 34% Sophos UTM 425 $1.41 $6.21 $84,700 $94,599 $9,899 12% Stonesoft FW- 1301 $7.01 $6.81 $350,420 $356,682 $6,262 2% WatchGuard XTM 1050 $2.36 $12.62 $235,950 $129,597 ($106,353) - 45% Figure 16 Comparison of Price to Weighted Value The weighted value indicates whether a product is underpriced, overpriced or priced accurately depending on the measured performance and security effectiveness. 2013 NSS Labs, Inc. All rights reserved. 11

Test Methodology Methodology Version: Firewall v4 A copy of the test methodology is available on the NSS website at www.nsslabs.com Contact Information NSS Labs, Inc. 206 Wild Basin Rd, Suite 200A Austin, TX 78746 USA +1 (512) 961-5300 info@nsslabs.com www.nsslabs.com v2013.02.07 This and other related documents available at: www.nsslabs.com. To receive a licensed copy or report misuse, please contact NSS Labs at +1 (512) 961-5300 or sales@nsslabs.com. 2013 NSS Labs, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the authors. Please note that access to or use of this report is conditioned on the following: 1. The information in this report is subject to change by NSS Labs without notice. 2. The information in this report is believed by NSS Labs to be accurate and reliable at the time of publication, but is not guaranteed. All use of and reliance on this report are at the reader s sole risk. NSS Labs is not liable or responsible for any damages, losses, or expenses arising from any error or omission in this report. 3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY NSS LABS. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON- INFRINGEMENT ARE DISCLAIMED AND EXCLUDED BY NSS LABS. IN NO EVENT SHALL NSS LABS BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 4. This report does not constitute an endorsement, recommendation, or guarantee of any of the products (hardware or software) tested or the hardware and software used in testing the products. The testing does not guarantee that there are no errors or defects in the products or that the products will meet the reader s expectations, requirements, needs, or specifications, or that they will operate without interruption. 5. This report does not imply any endorsement, sponsorship, affiliation, or verification by or with any organizations mentioned in this report. 6. All trademarks, service marks, and trade names used in this report are the trademarks, service marks, and trade names of their respective owners. 2013 NSS Labs, Inc. All rights reserved. 12