Next Generation Firewall Capabilities Assessment 1. Introduction Comparison of Next Generation Firewall offerings from Cisco, Intel Security and Palo Alto Next generation firewalls, commonly abbreviated as NGFW, build on the capabilities of traditional stateful firewalls by adding application awareness and deep packet inspection capabilities to detect and block threats on the network. Traditional stateful inspection firewalls have essentially become obsolete because they do not inspect the payload of the packet and have no application awareness to distinguish between legitimate business application traffic and that of a malicious attack. In contrast, instead of allowing all traffic on typical Web ports, an NGFW can distinguish between specific applications (for instance, Netflix vs. Salesforce.com) and then apply policies based on business rules. Gartner defines an NGFW as a wirespeed integrated network platform that performs deep inspection of traffic and blocking of attacks. At minimum, Gartner states, an NGFW should provide: Application awareness, full stack visibility and granular control Nondisruptive inline bumpinthewire configuration Standard firstgeneration firewall capabilities, such as networkaddress translation (NAT), stateful protocol inspection (SPI), and virtual private networking (VPN) Integrated signaturebased Intrusion Prevention System (IPS) engine Ability to incorporate information from outside the firewall, such as directorybased policy, blacklists, and white lists Upgrade path to include future information feeds and security threats, and Secure Socket Layer (SSL) decryption to enable identifying undesirable encrypted applications
Application awareness is what makes a firewall a next generation firewall. NGFW vendors use a variety of techniques, including predefined application signatures, header inspection, and payload analysis to determine specific applications. The NGFW stores a library of approved applications and allows them to traverse the network, while examining the data packets for any anomalies. Along with predefined applications, NGFWs can also learn new applications by watching how the applications behave. The NGFW creates a baseline of normal behaviors and can alert administrators if the application deviates from normal. This study assesses the capabilities of three NGFW vendors: 1. Cisco Adaptive Security Appliance (ASA) with FirePOWER services 2. Intel Security McAfee NGFW 3. Palo Alto NGFW Our study evaluates the vendors NGFW capabilities, including strengths and weaknesses, based on the technical assessments, testing, and insights provided by Miercom, NSS Labs, Gartner, ESG Labs, and vendor published specifications. Our goal is to provide federal agencies with market intelligence to help them acquire an NGFW that most closely aligns with their requirements and mission. This white paper is not meant to be a detailed engineering report, but rather a concise summary of the capabilities provided by Cisco, Intel Security, and Palo Alto. For a deeper understanding of the NGFW capabilities, Why these vendors? SwishData selected these vendors because they tend to dominate our discussions with customers. Cisco is the major network vendor and usually the default firewall choice for many organizations simply because it is Cisco. The other two NGFW vendors, Intel Security and Palo Alto, were shown in Department of Defense (DoD) testing to be the only NGFW products on the market to successfully identify all application traffic thrown at them. evaluation instrumentation, and test methodologies, the reader is referred to the original sources from Miercom, NSS Labs, Gartner, ESG Labs, and vendor published specifications. A complete list of references is provided at the end of this white paper. 2. Next Generation Firewall Vendor Overview 2.1 Cisco ASA with FirePOWER Services Although known for its routers and switches, Cisco also has a strong security focus and provides security products that are used by enterprises and data centers. Cisco s first move into the NGFW market began with the conversion of the legacy ASA firewall into an NGFW product called ASA CX. Unfortunately, Cisco ASA CX proved to be a limited product that never gained wide acceptance in the market. To strengthen its NGFW capabilities, Cisco acquired Sourcefire. Sourcefire provided Cisco with the intellectual property to deliver a next generation firewall and IPS. However, Cisco uses a Sourcefire blade (rebranded as FirePOWER) in the same ASA chassis as its legacy ASA firewall. This means that the end product is inheriting the limitations of the chassis. Ultimately, the NGFW capability in Cisco ASA is a bolton solution. Strengths Cisco brand High effectiveness against signaturebased threats Support for high availability (HA) failover in active/ standby mode Weaknesses Weak NGFW capabilities: cannot run IPS and application control simultaneously. If you can t run IPS and application control simultaneously, then it does not really qualify as NGFW No support for clustering, active/active load balancing capability Active/standby capabilities are limited and result in feature loss, including advanced threat detection in ASA 9.x code 2 Copyright SwishData 2015 MARCH 2015
FirePOWER management console can only support up to 150 devices. Once 150 devices are exceeded, customers must purchase another console. Management of ASA legacy features requires secondary management through Cisco Security Manager (CSM). Poor performance against Advanced Evasion Techniques (AETs) 2.2 Intel Security / McAfee NGFW McAfee is a wholly owned subsidiary of Intel and has undergone rebranding from McAfee to Intel Security. However, to retain McAfee brand cachet, most products within Intel Security s portfolio retain the McAfee name (e.g., McAfee NGFW). Intel Security s NGFW offering stands out from its competitors by leading the security market in the field of AET research, which is critical for being able to detect advanced persistent threats (APTs) in an enterprise network. Gartner identifies McAfee NGFW as a visionary product in the Gartner Magic Quadrant, because it has firewall features that are not seen in competitor s offerings. The McAfee NGFW can be purchased as a hardware appliance, a virtual machine, and MILSTD810 ruggedized tactical appliances. Strengths Market leader in AET detection and remediation Highest throughput of any NGFW in the market with all security features enabled Designed to provide ASIClike performance in x86 architecture ASIClike performance of virtual appliances running in VMware environment Builtin active/active clustering that scales to 16 nodes, with dynamic load balancing. No scheduled downtime required for software upgrades within a cluster. Integrates with McAfee s ecosystem of security products, including Host Based Security System (HBSS) and Global Threat Intelligence (GTI) McAfee Security Management Center (SMC) supports up to 2000 managed devices Management center can receive logs from other platforms, allowing SMC to act as a log server Low total cost of ownership (TCO), as recognized by the NSS Lab Security Value Map (SVM) Weaknesses Poor US presence and install base. Few US customers available as reference. Unique user interface (UI) means that due to the learning curve, the end user may require more upfront training No onboard management. NGFW appliances need to be deployed together with SMC server for integrated management. Onboard management capability is currently being added for inclusion with the next firmware release. 2.3 Palo Alto NGFW Palo Alto Networks is a pureplay network security company. Gartner assesses Palo Alto as a leader, largely because of its NGFW design, consistent displacement of competitors, rapidly increasing revenue and market share, and market disruption that forces competitors in all quadrants to react. However, Gartner does not test the products and so is unable to discuss the limitations of the Palo Alto NGFW. For example, Palo Alto struggles with performance when additional features are turned on and requires thirdparty software to support clustering, which limits its scalability. Palo Alto has achieved market success because it was the first vendor to offer a firewall with true NGFW capabilities: firewall, IPS, DPI, application control, user ID visibility, and antimalware. Strengths Robust application control and DPI capabilities Strong IPS solution with the NGFW ASICbased, optimized data path allows for high throughput performance Strong central management and reporting capabilities for smaller deployments through Pal Alto s Panorama management console Integration with Palo Alto s WildFire, which is a sandbox solution performing runtime code analysis of a suspect file 3 Copyright SwishData 2015 MARCH 2015
Weaknesses Caution rating was issued by NSS Labs because products running PANOS v6.0.3 are susceptible to severe evasion failures, which cannot be publicly disclosed without putting Palo Alto Networks customers at risk, since there are currently no known workarounds without upgraded to a newer version PANOS. This may also affect other versions released after the last known good version tested by NSS, PANOS v4.1.9. Performance declines below advertised throughput as additional capabilities on the firewall are enabled. Effectively, customers are forced to turn off some of the NGFW capabilities if they would like to retain high network traffic throughput on the device. Requires a 3rd party load balancer solution to perform clustering above 2 nodes. Unreasonably high TCO, as assessed by NSS Labs, which placed Palo Alto in the lower lefthand corner of NSS Labs SVM. Hardware ASIC performance does not translate into virtualized environment. 3 Next Generation Firewall Comparison Matrix Based on the research data from Miercom, NSS Labs, Gartner, and ESG Labs, we compiled a list of NGFW capability parameters and put them into a matrix for comparing NGFW products. Each capability was given a grade 1 through 5 as follows: 5 Excellent Capability is better than that offered by most competing products on the market We then rated each of the three NGFW products in 10 important capability areas. The results are shown in the NGFW comparison matrix on the next page. The McAfee NGFW was clearly superior to the Cisco ASA with Firepower and the Palo Alto NGFW, scoring 46 out of 50 possible points. In every category, the McAfee NGFW received a rating that was either higher than or equal to the other NGFWs. The discussion below describes each of the capability parameters used in the NGFW comparison matrix and explains how we assigned our ratings. Application Visibility Application visibility is the core NGFW capability. Different vendors use different techniques to identify applications within network traffic. Some vendors use basic techniques such as hash, string, and URL matching, while others employ sophisticated application fingerprinting methodologies. Within DoD, an agency conducted a number of tests to determine which NGFW products performed best in the area of application identification and categorization. The agency used Ixia XM12 and BreakingPoint (now acquired by Ixia) FireStorm network test appliances to generate application traffic and let the NGFW products identify the applications on the wire. Only Palo Alto and McAfee NGFWs were able to successfully identify all applications. In short, only Palo Alto and Intel Security have the special sauce to accurately do application fingerprinting within a firewall. For a copy of the report, please contact your Chief Information Security Officer (CISO). SwishData can help direct you to the right information source. Signaturebased Threat Detection 4 Good Capability is robust, but may present a few noncritical shortcomings 3 Fair Capability is adequate, but there are better products out there 2 Behind the Competition Capability competes poorly with that offered in other products 1 Poor Look for comparable solutions from another vendor Signaturebased threat detection is the basic capability of all modern firewalls, not just NGFWs. The signaturebased threat detection performance depends on how quickly the firewall signatures are updated after signatures for new threats emerge. One could argue that because McAfee has its Global Threat Intelligence (GTI) worldwide feed, McAfee NGFW would be updated more quickly than offerings from Cisco or Palo Alto. However, based on our research, we did not see GTIintegration yield better signaturebased threat detection performance. All three contenders did well in this category. 4 Copyright SwishData 2015 MARCH 2015
NGFW Capability Cisco ASA w/ FIREPOWER McAfee NGFW Application Visibility 3 5 5 Signaturebased Threat Detection 5 5 5 Web Security Effectiveness 4 4 2 Dangerous Website Filtering 4 4 3 AET Detection 3 5 3 Throughput 2 5 4 Scalability 2 5 3 High Availability 2 5 3 Management & Reporting 3 4 4 TCO 2 4 1 Palo Alto NGFW Total Score: 30 / 50 46 / 50 33 / 50 Web Security Effectiveness According to Miercom, web security effectiveness covers protection against drivebyinstallers, complex web exploits, phishing, and malicious redirects. Cisco offers a web security in a form of its IronPort web security appliance. However, some of the functionality is included in the ASA firewall with FirePOWER services. According to NSS Labs, the web capability is very good, which is why we opted for the 4 rating. Referencing Miercom web security tests, McAfee was a capable performer. However, most surprising was Palo Alto s poor URL filtering functionality, which is available via subscription. Palo Alto only yielded 3 percent block rate in Miercom s web security effectiveness test. Dangerous Website Filtering Dangerous website filtering refers to the security device s ability to detect and block various types of risky web content, such as sexual material, gambling, proxy avoidance, and hacking. Blocking these types of web content is an important aspect of controlling online access to minimize loss of user productivity, manage bandwidth costs, prevent potentially malicious content from entering the enterprise network and meet compliance requirements. Our grading was based on reports from Cisco ASA NSS Labs reports and Miercom web security testing. Application awareness is not all the same. With NGFW being the new big buzzword and every vendor wanting to jump on the NGFW bandwagon, many vendors have resorted to shortcuts. Some call their latest firewall offering an NGFW and claim that it does application awareness, when it only performs basic application categorization, if anything at all. If an unsuspecting customer were to procure this NGFWlabeled product, he or she would find NGFW capabilities to be woefully inadequate. AET Detection AET detection is major factor for organizations concerned with APTs and zeroday exploits. The pioneering vendor that began implementing AET detection methods within a firewall platform was Stonesoft. As Stonesoft NGFWs gained popularity, Intel Security acquired Stonesoft to compete with Palo Alto in the NGFW market. 5 Copyright SwishData 2015 MARCH 2015
While testing evasions at different layers of the network, Intel Security began to learn about more complex and dynamic evasions appearing in the wild. In 2010, Intel Security published a report on the discovery of AETs, and highlighted the vulnerabilities of most security devices at the time. Intel Security asserts that most security devices are still vulnerable to AETs today. Intel Security runs millions of evasion combinations in its labs daily, and shares its findings with the Computer Emergency Readiness Team (CERT) and numerous security vendors. The Evader tool was developed to provide inhouse testing capability for companies that deploy network security devices using deep packet inspection, such as IPS and NGFW. Companies can use Evader for realworld tests of their protection against AETs, thus enabling them to improve security levels and evaluate the results against vendor claims and published lab results. Evader is provided free of charge by Intel Security at http://evader.mcafee.com/. It is important to note that Evader is not a hacking tool or a penetration test harness. Evader simply tests if a known exploit can be delivered using AETs through currently installed security devices to a target host. When it comes to AET detection, McAfee NGFW is an undisputed leader with Cisco and Palo Alto trailing behind. The comparison matrix AET grades reflect this. Throughput Palo Alto claims it is the only vendor in the industry with an optimized data plane because of the proprietary ASICs used for wirespeed processing. However, this is only true in limited situations. From field experience, Palo Alto shows a steep decline in throughput performance as features are turned on. Therefore, to get the advertised performance numbers, many of the NGFW features need to be disabled, thereby lowering the security posture of the product. Cisco ASA with FirePOWER services experiences similar issues. Cisco has had backplane throughput limitations starting with its Catalyst switches; consequently, Cisco acquired Nuova Systems in 2008 to get the technology for Cisco Nexus switches. The same largely holds true with the updated ASA firewalls. They are low throughput, only going to 10 Gbps when application control and IPS are turned on. In contrast, Palo Alto can yield 60 Gbps and McAfee NGFW can do a whopping 120 Gbps with all features turned on. Scalability & High Availability When talking scalability, Cisco ASA does not do well with large environments. However, if the need is for a small business or a branch office, Cisco ASA could be completely adequate. Cisco also does not do clustering. Its firewalls operate in active/standby failover mode only. Palo Alto does well for small to midsize environments. Palo Alto NGFWs can work in pairs to form a single NGFW cluster. However, beyond that, one needs to use a third party load balancer to scale the NGFW deployment. McAfee NGFW can work in clusters of 16 nodes with terabit throughputs. McAfee NGFW also offers the ability to do capacity and software updates to the cluster without any disruption. It is the only vendor with that capability to date. Overall, the McAfee NGFW product does well for deployments small, midsized, and large. Moreover, in an effort to gain greater market share from Palo Alto, McAfee NGFW products are competitively priced. Management & Reporting Per Network World magazine s June 13, 2013 issue, Cisco still has significant work to do in improving the management, integration, threat mitigation and application controls. Palo Alto has its Panorama management console. Panorama provides the ability to manage a distributed network of firewalls from a centralized location. Using Panorama, one can view firewall traffic, manage all aspects of device configuration, push global policies, and generate reports on traffic patterns or security incidents. The issue with Panorama is that it does not scale in large networks. For large deployments, multiple Panorama appliances are required. McAfee NGFW offers a similar management solution called the Security Management Center (SMC). The SMC allows one to manage, monitor, log, and report on most Intel Security/McAfee products from one console. Additionally, the SMC can manage thirdparty switches, routers, and security appliances, and act as an external log server for other devices to send logs to. You can efficiently automate routine tasks, reuse elements and utilize numerous shortcuts and drillins. SMC supports management of up to 2,000 devices from a single appliance. 6 Copyright SwishData 2015 MARCH 2015
Total Cost of Ownership The best way to discuss the TCO aspect of NGFW products is to examine the NSS Labs Security Value Map (SVM), which breaks down NGFW cost into TCO by protected Mbps. 4 Solution Review and Recommendation NGFWs combine application awareness and deep packet inspection to give organizations more control over applications while also detecting and blocking malicious threats. In the past several years, it seems as if every vendor has begun offering an NGFW solution. However, as we have seen, only two vendors have a robust application visibility function that stands up to scrutiny: Palo Alto and Intel Security/McAfee. The other vendors may detect only some application traffic, while the rest will go uncategorized. Larger organizations need to be increasingly concerned with the advent of APTs and the risk they pose. The majority of APTs are delivered through covert channels by means of advanced evasion. This makes AET capabilities critical for any NGFW product considered by an organization. All NGFW products on the market tout evasion capabilities. However, as can be proven by the Evader tool, most fall short detecting even the basic of evasions. Intel Security is the undisputed leader when it comes to AETs. Palo Alto is catching on as well. In October 2014, Palo Alto delivered a silent update in its PANOS v6.05h3 code that fixed major evasion holes published by the NSS Labs. Cisco s evasion capabilities are still weak. Scalability and high availability are two other points to consider. If your organization does not anticipate growth, scalability may not be an issue. If your organization does not have a high throughput requirement, you may be fine with your firewall having just an active/standby HA mode. However, if you cannot tolerate any downtime, a more appropriate solution may be the one that can support clustering capabilities and hitless upgrades, all while maintaining high throughput. To conclude with some recommendations, for small to midsize organizations that would like to use Cisco because they are heavily invested in Cisco products, the Cisco ASA with FirePOWER services may be an adequate choice. However, your organization will be missing important security capabilities, and so will have to purchase a separate security appliance (e.g., an IPS/ IDS) to augment deficiencies in the ASA. Palo Alto is a good product that does very well in all but very large deployments. However, Palo Alto NGFW is incredibly costly and, as we have seen, does surprisingly poorly in the webfiltering category. Standing tall in our evaluation is Intel Security s McAfee NGFW. It exceeds Palo Alto in seven of the ten capability categories, including throughput, HA, and AET detection. McAfee NGFW equals Palo Alto in the other three categories. And because Intel Security is trying to recapture market share from Palo Alto, McAfee NGFW pricing is very competitive. About SwishData We re the cybersecurity and data performance architects. SwishData ensures the performance, affordability, and security of your agency s data infrastructure through both architecture and deployment. 17 Feagles Road Warwick, New York 10990 (703) 5318526 Phone (703) 8527904 Fax info@swishdata.com www.swishdata.com 7 Copyright SwishData 2015 MARCH 2015