eeye Retina Network Security Scanner v5.10 & REM Security Management Console PRODUCT REPORT ON PCI SUITABILITY

eeye Retina Network Security Scanner v5.10 & REM Security Management Console PRODUCT REPORT ON PCI SUITABILITY VULNERABILITY ASSESSMENT SCANNER (VA) NSS LABS CRITERIA VERSION: 1.2 REFERENCE: PCI DSS 1.1 AUGUST 25, 2008

Published by NSS Labs. 2008 NSS Labs CONTACT: 5115 Avenida Encinas Suite H Carlsbad, CA 92008 Tel: +1.847.553.4300 E-mail: info@nsslabs.com Internet: http://www.nsslabs.com All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the authors. This report shall be treated at all times as a confidential and proprietary report for internal use only. Please note that access to or use of this Report is conditioned on the following: 1. The information in this Report is subject to change by NSS Labs without notice. 2. The information in this Report is believed by NSS Labs to be accurate and reliable, but is not guaranteed. All use of and reliance on this Report are at your sole risk. NSS Labs is not liable or responsible for any damages, losses or expenses arising from any error or omission in this Report. 3. NO WARRANTIES, EXPRESS OR IMPLIED ARE GIVEN BY THE NSS LABS. ALL IMPLIED WARRANTIES, INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT ARE DISCLAIMED AND EXCLUDED BY NSS LABS. IN NO EVENT SHALL NSS LABS BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL OR INDIRECT DAMAGES, OR FOR ANY LOSS OF PROFIT, REVENUE, DATA, COMPUTER PROGRAMS, OR OTHER ASSETS, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 4. This Report does not constitute an endorsement, recommendation or guarantee of any of the products (hardware or software) tested or the hardware and software used in testing the products. The testing does not guarantee that there are no errors or defects in the products, or that the products will meet your expectations, requirements, needs or specifications, or that they will operate without interruption. 5. This Report does not imply any endorsement, sponsorship, affiliation or verification by or with any companies mentioned in this report. For PCI-related reports, this does not constitute an endorsement by the PCI Security Standards Council. 6. All trademarks, service marks, and trade names used in this Report are the trademarks, service marks, and trade names of their respective owners, and no endorsement of, sponsorship of, affiliation with, or involvement in, any of the testing, this Report or NSS Labs is implied, nor should it be inferred.

EXECUTIVE SUMMARY In Q3 of 2008, NSS Labs performed comprehensive testing of the eeye Retina Network Security Scanner v5.10 as a Vulnerability Assessment Scanner (VA) for internal use within payment card environments. This report contains the conclusions and associated data from a series of exacting tests performed on software installed in our real-world test lab. It is important to note that this test was designed to determine the viability of the vulnerability scanning product for internal use within payment card environments. As such, NSS Labs tested the scanner s ability to accurately identify and classify a much broader range of application vulnerabilities than would be required for an external vulnerability scanner such as those scanning services certified by the current PCI ASV program. For example: An external vulnerability scanner which identifies a SQL service that is available over the Internet should result in an immediate failure for the company being scanned, since it is never acceptable to directly query a live SQL server over the Internet. Conversely, an internal vulnerability scanner might expect to see the same SQL service running on the internal corporate network, and would be required to determine the version of SQL running, and maybe even perform some parameter checks to ascertain whether or not the version and/or configuration is vulnerable to exploit. Support for PCI DSS requirements was solid. NSS Labs found that the Retina Scanner v5.10 successfully passed 16 of 16 DSS requirements, and supported 45 of 47 others indirectly. There were only minor failures as the eeye product sailed through the process of validation without any major identified issues. Overall, out of 63 tested requirements, the product supports 61 (97%). Note: Users are advised to consult with their QSA regarding use and configuration of this product for compliance purposes. The effectiveness of Retina Network Security Scanner v5.10 was tested in its ability to accurately discover and classify live vulnerabilities on real operating systems and applications. NSS Labs real-world test environment included 250 hosts with over 25 different operating systems, 200 different applications and 100 patch levels. This is the most comprehensive and challenging environment of any test lab in the world, and represents extreme enterprise conditions. Retina detected a total of 99.0% of the vulnerabilities in our extensive enterprise network, correctly discovering and classifying a total of 687 of the 694 vulnerabilities. Of the 538 attacker initiated vulnerabilities, Retina missed none, giving a perfect detection total of 100%. Target initiated vulnerabilities, such as those in Internet Explorer or Adobe Acrobat, are much more difficult to accurately detect, yet Retina achieved a very respectable score of 95.5%, detecting 149 out of 156. The only detractor for this product was its tendency to produce too much vulnerability information on a given host which, while technically correct, might lead one to believe a host was vulnerable when it only had the potential to be vulnerable (depending on the configuration and use of the device).

Performance of the eeye Retina Network Security Scanner v5.10 was surprisingly robust. Running on an older 2GHz (single core) P4 server with 1GB of RAM, Retina was able to scan an entire Class C network (250 devices) in just 26 minutes. Retina s user interface provides a quick and easy means to manage a single instance, offering a straightforward way to configure the VA, view reports, etc. For larger deployments, the web-based REM Management Console has been designed to coordinate management, configuration, and reporting of large numbers of Retina Scanners across the enterprise. Reporting is mature, powerful and flexible. In our opinion, the eeye s Retina Network Security Scanner v5.10 is a robust Vulnerability Assessment Scanner and should be on any short list for e-commerce Datacenters, Corporate Datacenters, Corporate Perimeter and Retail Storefront environments. NSS Labs finds the eeye Retina Network Security Scanner v5.10 is suitable for use in: E-Commerce Datacenter / Hosting Center environments where there are many hosted services available to external users (i.e. E-Commerce Applications, HTTP, SMTP, IMAP, POP-3, DNS, and Outlook Web Access). Internal Datacenter environments where there are many services available to internal users (i.e. RPC, CIFS, Oracle Net, NTLM, XML, ERP Applications, DB Applications, HTTP, SMTP, IMAP, POP- 3, DNS, and MS Exchange). Corporate Perimeter environments where there are simple hosted services available to external users (i.e. HTTP, SMTP, IMAP, POP-3, DNS, and Outlook Web Access), as well as internal applications such as MS Word, Excel, and PowerPoint, Adobe Acrobat, and Corporate email, are likely to be available on the desktop. Retail Storefront environments where there are no hosted services available to external users (i.e. E-Commerce Applications, HTTP, SMTP, IMAP, POP-3, DNS, and Outlook Web Access), and communications are initiated from the internal network. Internal applications such as MS Word, Excel, and PowerPoint, Adobe Acrobat, and Corporate email, may be available on the desktop in the back office.

CONTENTS 1 Introduction... 1 2 The Product Under Test... 2 3 VA PCI Test Environment... 3 3.1 Testing VA Scanners... 3 3.2 VA test environment... 3 4 Results Summary... 6 4.1 About PCI DSS Functionality Validation... 6 4.2 PCI DSS Requirements Validation Map... 8 4.3 Performance... 10 4.4 Security Effectiveness Vulnerability Assessment Scanning... 11 4.5 NSS Test Methodologies... 13 4.6 Recommended Configurations... 15 5 Key Management & Scanning encrypted pages... 16 5.1 Key Management... 16 6 Stability & Capacity... 18 6.1 Detection under load... 18 6.2 System Capacity - real-time view of system utilization... 18 7 Logging and Reporting... 19 7.1 VA Scan Results... 19 7.2 Administrative Access Logging and Reporting... 20 7.3 updates and configuration changes... 20 7.4 Synchronization of System Clock... 21 7.5 Centralized Logging Over Secured Communications Channels... 21 8 Patches and Updates... 22 8.1 Support secure, non-refutable updates... 22 8.2 Online Updates... 22 8.3 Offline Updates... 22 9 Management & Administration... 23 9.1 PCI Default Configuration - No Default Usernames / Passwords... 23 9.2 Password Policy... 23 9.3 No Shared User Accounts... 24 9.4 Two-Factor Authentication... 24 9.5 Secured Management Interface... 24 Appendix A: Test Infrastructure... 25

1 INTRODUCTION In Q3 of 2008, NSS Labs performed comprehensive testing of the eeye Retina Network Security Scanner v5.10 against our Vulnerability Assessment Scanner criteria. This report contains the conclusions and associated data from a series of exacting tests performed on software installed in our real-world test lab. The NSS Labs Product Reports on PCI Suitability are designed to address the challenges faced by IT departments in selecting security products to address the compliance requirements of the Payment Card Industry s Data Security Standard (PCI DSS). This NSS Labs report provides readers with empirically validated evidence about a product s suitability for use in a payment card network. Fulfillment of specific PCI DSS v1.1 requirements, including logging and reporting Recommended Configuration Details for PCI network deployment Security Effectiveness Appropriate Usage Recommendations Product Stability and Reliability The NSS Labs Product Reports on PCI (VA) attests to the abilities of a Vulnerability Assessment Scanning product to accurately detect and report vulnerabilities within multiple versions of the following: Operating Systems - BSD, Linux, Sun Solaris, Microsoft Windows Web Servers - Apache, Lotus Domino, Microsoft IIS, Sun One Database Servers - IBM DB2, Microsoft SQL Server, MySQL, Oracle, PostreSQL, Sybase Mail Servers- Lotus Domino, Microsoft Exchange, Netscape Messaging Server, SendMail Firewalls - Check Point, Cisco, Gauntlet, Juniper, Fortinet, Linux IP chains/tables Routers & Switches Cisco, Juniper, Extreme, 3Com Common IP Services DNS, FTP, SMTP, RPC, COM, Syslog, File Sharing NSS Labs reports now implement the concept of Appropriate Usage (see NSS Labs whitepaper Evaluating Products based on Appropriate Usage ). Testing products based upon Appropriate Usage (applying a Use- Case based methodology) provides a clear picture of which security technologies are effective against a particular type of threat or attack. Thus, products can be evaluated based on their capabilities against specific deployment scenarios and protection requirements. Evaluated products are categorized for Retail Storefront, e-commerce Datacenter, Internal Datacenter, and Corporate Perimeter environments. eeye Retina Network Security Scanner v5.10 VA PCI p. 1

2 THE PRODUCT UNDER TEST eeye Retina Network Security Scanner v5.10 The eeye Retina Network Security Scanner v5.10 was installed and tested by NSS Labs on a Microsoft Windows Server 2003 system running a P4 2.33 GHz CPU, with a 1GB of RAM. The company also offers an appliance version, which was not tested. The REM Security Management Console was installed on a Microsoft Windows Server 2003 system running IIS and SQL on a P4 3GHz CPU, with a 2GB of RAM. More information at eeye Digital Security http://www.eeye.com. RETINA V5.10 REQUIREMENTS Microsoft Windows 2000, XP, 2003, Vista, or 2008 (x86, 32 bit only, Latest Service Packs Recommended) Microsoft.NET Framework 2.0 Intel Pentium IV 1.4 GHz or higher CPU 512 MB of RAM 80 MB of free disk space Network Card with TCP/IP Enabled Administrative access to run scans REM V3.5 REQUIREMENTS Microsoft Windows 2000 Server SP4 or Microsoft Windows 2003 Server SP2 (or higher) Microsoft IIS 6.0 (Internet Information Services or higher) Microsoft.NET Framework 2.0 (and ASP.NET on 2003) Intel Pentium IV 2.0 GHz or higher CPU 1GB of RAM or Higher 300 MB HDD for the software and 20 GB HDD for database, NTFS Required Microsoft SQL 2000 Server SP4 or SQL 2005 SP1 or higher Microsoft Internet Explorer 6.0 or higher Network Interface card, Network Connection, and Internet Access Sun Java 5.0 SE Update 4 or Higher Network Security Scanner Retina enables prioritized policy management, patch management, and vulnerability management. Network Vulnerability Assessment Identify network security vulnerabilities, missing application updates, plus zero day threats. Network Discovery and Policy Assessment Retina discovers all devices, operating systems, applications, patch levels, plus policy configurations. Vulnerability Management Enables prioritized policy management, patch management, and vulnerability assessment. Fast and Accurate Scans Accurately scan a Class C network of devices, operating systems and applications in ~25 minutes. Policy Compliance Identify and simplify your corporate and regulatory requirements (SOX, HIPAA, GLBA, PCI and others). eeye Retina Network Security Scanner v5.10 VA PCI p. 2

3 VA PCI TEST ENVIRONMENT 3.1 TESTING VA SCANNERS The ultimate goal of any attack into a computer system is to gain access to a target host and attempt to perform an unauthorized action. The unauthorized action could be reading of a system file, accessing a memory location, execution of malicious code, or any number of other actions. Unauthorized access of this nature is considered an intrusion. Computer systems are designed with many levels of protection to prevent unauthorized access and grant authorized access. However, intruders may circumvent these levels of protection by targeting vulnerable services, invoking back door privilege escalation, or replacing key operating system files. Network Vulnerability Assessment Scanning products are designed to discover and then interrogate systems on the network in order to classify the Operating System & Application versions as well as identify potentially dangerous configurations. Once the system in question has been properly identified and classified, a VA Scanner will cross-reference the information gathered during the scan/interrogation with its database of known vulnerabilities and produce a report detailing which systems are susceptible to which vulnerabilities for engineers to take action. Vulnerability Assessment Scanner products must properly identify vulnerabilities in operating systems and services without inadvertently compromising system or service / application integrity or stability. VA Scanners differ from their Penetration Testing cousins in that they do not actually exploit remote services; instead gathering relevant evidence either by logging into the system with administrative privileges and then querying the registry / scanning the file system for relevant information, or by looking for remote indicators such as TCP/IP Stack response, service banners, and so on. NSS Labs VA testing focuses on the ability of a VA Scanner to properly identify vulnerabilities including: remotely exploitable vulnerabilities, localized privilege escalation, catastrophic misconfiguration (i.e. allowing telnet to root without a password, or with an easily guessable password such as password ), as well as the existence of rootkits on a system. First, baseline vulnerabilities & successful attacks are determined for each host to be identified & classified using real-world exploits. Next, the target systems are restored to their pre-compromised state and the VA Scanner is launched and the results are recorded. Finally, the target host/applications are re-validated to ensure that the VA scanning does not interfere or prohibit legitimate usage. The overall effectiveness at detecting, identifying, and classifying vulnerabilities is then recorded and provided within this report. 3.2 VA TEST ENVIRONMENT NSS Labs maintains a farm of hundreds of operating systems and applications in varying security postures from lockdown to wide open as well varying patched states and build levels. eeye Retina Network Security Scanner v5.10 VA PCI p. 3

Vulnerable services are validated with real exploits that were caught in the wild from production environments, as well as by using various tools such as Core Impact, ImmunitySec Canvas, and Metasploit. The resulting test bed can uniquely validate vulnerability identification accuracy, infrastructure impact, and scanning performance of a Vulnerability Assessment Scanning solution. NSS Labs Real-world Test Bed is comprised of over 250 unique hosts with an 80/20 mix of client and server Operating Systems. 3.2.1 THE CLIENT OPERATING SYSTEMS INCLUDE: Windows 2000 & 2000 Pro (SP0-SP4 + multiple intermediate builds) Windows XP & XP Pro (SP0-SP3 + multiple intermediate builds) Windows XP Embedded Windows Vista (SP0-SP1 + multiple intermediate builds) SUSE Linux 9 & 10 (various builds) Red Hat Linux 3, 4, & 5 (various builds) Multiple Fedora builds Apple OSX 3.2.2 SERVER OPERATING SYSTEMS INCLUDE: Windows 2000 Server & Advanced Server (SP0-SP4 + multiple intermediate builds) Windows 2003 Server & Advanced Server (SP0-SP2 + (multiple builds)) SUSE Enterprise Linux 9 (multiple builds) SUSE Enterprise Linux 10 (multiple builds) Red Hat Enterprise Linux 3 (multiple builds) Red Hat Enterprise Linux 4 (multiple builds) Red Hat Enterprise Linux 5 (multiple builds) Sun Microsystems Solaris 8 (multiple builds) Sun Microsystems Solaris 9 (multiple builds) Sun Microsystems Solaris 10 (multiple builds) Free BSD 6.1-6.3 (multiple builds) Free BSD 7 OpenBSD 3.1 3.8 eeye Retina Network Security Scanner v5.10 VA PCI p. 4

OpenBSD 4.0 4.3 VA Scanners are tested against Internal/Core Datacenter, e-commerce Datacenter, and Enterprise Perimeter, and Retail storefront environments: eeye Retina Network Security Scanner v5.10 VA PCI p. 5

4 RESULTS SUMMARY 4.1 ABOUT PCI DSS FUNCTIONALITY VALIDATION This section provides a summary overview of the PCI DSS v1.1 Requirements validated by NSS Labs evaluation of the product. The PCI DSS is both a broad and very prescriptive set of requirements which span product functionality, human and automated processes, and network architectures. The scope of NSS Labs product validation is limited to what can be evaluated in our test labs. NSS Labs evaluates and validates product capabilities. It should be noted that capable products can be implemented and configured in ways that do not meet DSS requirements. NSS Labs cannot and does not validate the implementations of the product at specific customer sites how it is configured, and where it is deployed. That level of compliance validation per organization is the sole purview of Qualified Security Assessors. NSS Labs conducts product evaluations based on the official PCI DSS, guidelines, informational supplements, FAQs and other supporting documentation located at the PCI Security Standards Council s web site: https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml Recognizing that products can support PCI DSS in different ways, NSS Labs had developed three distinct classes of validation to which it adheres in the evaluation process. eeye Retina Network Security Scanner v5.10 VA PCI p. 6

The following legend outlines the scoring criteria used by NSS Labs engineers when evaluating product functionality for support of DSS requirements. Validation Description & Interpretation The product has been validated to meet the objectives of the specified PCI DSS requirement. PASS E.g. PCI DSS requirement 1.5: Implement IP masquerading to prevent internal addresses from being translated and revealed on the Internet. Use technologies that implement RFC 1918 address space, such as port address translation (PAT) or network address translation (NAT). NSS Labs Tests 5.7.1 and 5.7.2 validate that the VA Scanner tested is capable of enforcing this requirement on the subject network as a whole. The product was designed in such a way that it supports procedures and processes called for within PCI DSS. E.g. PCI DSS requirement 7.2: Restrict Access to computing resources and cardholder information only to those individuals whose job requires such access. NSS Tests 12.3 and 12.4 validate that the VA Scanner tested is capable of enforcing this restriction on itself though a device of its type is not intended to enforce nor capable of enforcing this functionality on other devices and systems on the network. The product has been found to not adequately meet the objectives of the specified PCI DSS requirement. FAIL E.g. PCI DSS requirement 10.5.2: Protect audit trail files from unauthorized modifications. Products that allow unauthorized modifications of log files would receive a fail. The requirement is neither directly nor indirectly applicable to the product. Most often used in the case of a direct procedural or policy requirement. E.g. DSS 2.1 Always change vendor-supplied defaults before installing a system on the network (for example, include passwords, simple network management protocol (SNMP) community strings, and elimination of unnecessary accounts). eeye Retina Network Security Scanner v5.10 VA PCI p. 7

4.2 PCI DSS REQUIREMENTS VALIDATION MAP The following chart depicts the PASS/FAIL status of each test correlated to the associated PCI DSS section to which it applies. PCI DSS NSS TEST DESCRIPTION NSS TEST ID 1.1 1.5 2.1 2.1.1 2.2 PCI Default Configuration - No Default Usernames / Passwords 9.1 2.2.1 2.2.4 2.3 Secured Management Interface 9.5 2.3 Separate Interface for Management 9.5.1 2.3 Administrative Access on Trusted Interface 9.5.1 2.4 3.52 3.6 Key Management 5.1 3.6.1 Key Management - Generation of Strong Keys 5.1.1 3.6.2 Key Management - Secure Key Distribution 5.1.2 3.6.3 Key Management - Secure Key Storage 5.1.3 3.6.4 Key Management - Periodic Changing of Keys 5.1.4 3.6.4 Key Management - Changing of Keys Automatically 5.1.5 3.6.4 Key Management - Changing of Keys At Least Annually 5.1.6 Key Management - Destruction and Revocation of Old 3.6.5 or Invalid Keys 5.1.7 3.6.6 3.6.8 Key Management - Destruction and Revocation of Old 3.6.9 or Invalid Keys 5.1.7 3.6.10 4.1 4.2 5.1 5.2 PASS 6.1 Ensure that all system components and software have the latest vendor-supplied security patches installed. Install relevant security patches within one month of 8 release. 6.1 Support Secure, Non-refutable Updates 8.1 eeye Retina Network Security Scanner v5.10 VA PCI p. 8

PCI DSS NSS TEST DESCRIPTION NSS TEST ID 6.1 Online Updates - Using Hosted Provider 8.2.1 Online Updates - Not Susceptible to Man in the Middle 6.1 Attacks 8.2.2 6.1 Offline Updates Removable Media 8.3.1 6.1 Offline Updates - Digitally Signed and Encrypted 8.3.2 6.2 6.6 7.1 7.2 No Shared User Accounts 9.3 7.2 No Shared User Accounts 9.3 8.1 No Shared User Accounts 9.3 8.2 Two-Factor Authentication 9.4 8.3 Two-Factor Authentication 9.4 8.4 Secured Management Interface 9.5 8.5 Password Policy Altering Case 9.2.3 Password Policy No Consecutive Repeating 8.5 Characters or Sequences 9.2.4 PCI Default Configuration - No Default Usernames / 8.5.1 Passwords 9.1 8.5.2-8.5.7 8.5.8 No Shared User Accounts 9.3 8.5.9 Password Policy Password Expiration 90 Day Max 9.2.5 8.5.10 Password Policy Password Length 9.2.1 8.5.11 Password Policy Enforces Non Alpha-Numeric 9.2.2 8.5.12 Password Policy No Repeat of Last Four Password 9.2.6 8.5.13-8.5.16 9 10.2.1 10.2.2 Administrative Access Logging and Reporting 7.2 10.2.3 Logs - Targeted Vulnerabilities Sorted by IP and Severity 7.1.1 10.2.3 Logs - Severity Levels Rated 7.1.2 10.2.3 Logs - Details for Each Vulnerability Found 7.1.3 10.2.3 Logs - Targeted Vulnerability Name 7.1.4 10.2.3 Logs - Severity Level 7.1.6 10.2.3 Logs - Comprehensive Explanation 7.1.7 10.2.4 10.2.6 eeye Retina Network Security Scanner v5.10 VA PCI p. 9

PCI DSS NSS TEST DESCRIPTION NSS TEST ID 10.2.7 Change Logs - User Identification 7.3.1 10.2.7 Change Logs - Type of Event 7.3.2 10.2.7 Change Logs - Date and Time 7.3.3 10.2.7 Change Logs - Success or Failure of an Action 7.3.4 10.2.7 Change Logs - Origination IP Address 7.3.5 10.2.7 Change Logs Resource Affected 7.3.6 updates and Updates and Configuration Changes Logging and PASS configuration Reporting 10.3 changes PASS 10.3.1 Change Logs - User Identification 7.3.1 PASS 10.3.2 Change Logs - Type of Event 7.3.2 PASS 10.3.2 Logs - Industry Reference Numbers 7.1.5 PASS 10.3.3 Change Logs - Date and Time 7.3.3 PASS 10.3.4 Change Logs - Success or Failure of an Action 7.3.4 PASS 10.3.5 Change Logs - Origination IP Address 7.3.5 PASS 10.3.6 Change Logs Resource Affected 7.3.6 Centralized Logging Over Secured Communications PASS 10.4 Channels 7.5 10.5 Masking / Omission of Restricted Card Holder Data PASS 10.5.1 Administrative Access on Trusted Interface 9.5.1 10.5.2 10.5.3 Masking / Omission of Restricted Card Holder Data 10.5.4-10.5.5 PASS 10.6 Logging and Reporting 7 10.7 11.1 PASS 11.2 Vulnerability Scans 7-9 11.3-11.5 12.1-12.10.4 4.3 PERFORMANCE NSS Labs has concluded that the eeye Retina Network Security Scanner v5.10 is appropriate for use in Internal Datacenter/Core, e-commerce, Perimeter and Retail Storefront environments where there are both hosted servers as well as desktop clients / laptops & PCs. While performance is impacted by enabling maximum capabilities, this should have little to no impact on the function of the software since scanning is not a real-time service, and the Retina product can scale simply by adding additional scan servers / appliances. eeye Retina Network Security Scanner v5.10 VA PCI p. 10

During our testing, the Retina Scanner performed admirably by scanning an entire Class C network of very vulnerable hosts in under 26 minutes. We would expect this time to decrease for less vulnerable networks, which would be nearly every production network with a reasonably good change control and security policy. In addition, we found the impact on the network during these this test was minimal as the Retina scanner relied heavily on local system information as opposed to untrusted scanning of services & applications. 4.4 SECURITY EFFECTIVENESS VULNERABILITY ASSESSMENT SCANNING Below are results displayed in the terms of Attack Source and Attack Impact. Attack Source defines whether an attack was launched directly by an external attacker ( Attacker Initiated ) or erroneously initiated by an internal user sitting at their PC or workstation ( Target Initiated, aka Client Initiated). Attack Impact defines whether a successful attack would have compromised a service ( Service Exposure i.e. an application such as Adobe Acrobat or Oracle database) or compromised the entire system ( Systems Exposure i.e. root access). System or Service Fault impact would make unavailable a specific service/application on the target system or crash the entire system. The security effectiveness of the eeye Retina Network Security Scanner was tested with live vulnerabilities on real operating systems and applications. It is important to note that the vendor was not aware in advance of the vulnerabilities selected for the test. The test results therefore reflect a very real-world scenario in which there is no ability to perform custom tuning for a lab environment. Thus, the security effectiveness results are quite impressive and differ demonstrably from any other public testing methodology currently in existence. Each vulnerability was individually verified by compromising a host in a number of ways prior to placing the VA Scanner into the test harness. These tests were performed using Immunity s Canvas, Core Impact and Metasploit as well as custom exploits against published vulnerabilities that NSS has harvested over the years. 4.4.1 ATTACK INITIATION The Retina Scanner s detection rate was surprisingly without flaw when it came to detecting web browser vulnerabilities such as Internet Explorer and Firefox. The vulnerabilities that were not detected were typically on more obscure / less used applications in situations wherein the user would have had to actively initiate a privilege escalation by approving actions by false documents and manually opening them despite warnings presented, or similar actions where the typical corporate end user could be expected to have received training sufficient to know better. Type Missed Caught Tested Caught % Attacker Initiated 0 538 538 100.0% Target Initiated 7 149 156 95.5% TOTAL 7 687 694 99.0% Retina Scanner detected 100% of Attacker Initiated vulnerabilities with 100% of System Exposure and 100% of System and Service Faults. We found this effectiveness to be excellent. eeye Retina Network Security Scanner v5.10 VA PCI p. 11

4.4.2 IMPACT TYPE NSS Labs evaluates and measures vulnerability severity using a number of methods. For the purposes of our reporting, we believe the impact type should reflect the resulting effect of an exploit rather than an arbitrary High, Medium or Low indication as is typically given by a system such as CVE. Such systems do not take into account the assets being protected, and therefore can lead to false estimations of severity. For example, a vulnerability labeled as LOW severity under CVE, could be of relatively HIGH importance if one has critical assets on that system. The most serious vulnerabilities were those which resulted in a remote system compromise, providing the attacker with the ability to execute arbitrary system level commands. Most exploits in this class that are weaponized will provide the attacker with a fully interactive remote shell on the target client or server. Retina Scanner proved strongest in this highly critical area, detecting 370 out of 371 (99.7%). Slightly less serious are the attacks resulting in an individual service compromise but not arbitrary system level command execution. Typical attacks in this category include service specific attacks such as SQL injection that enable the attacker to execute arbitrary SQL commands within the database service. These attacks are somewhat isolated to the service and do not immediately result in full system level access to the operating system and all services. However using additional localized system attacks it may be possible for the attacker to escalate from the service level to the system level. Of the 276 vulnerabilities in this category, Retina Scanner detected 274 (99.3%). Type Missed Tested Caught % System Exposure 1 371 99.7% Service Exposure 2 276 99.3% System or Service Fault 4 47 91.5% Finally, there are the attacks (often target initiated) which resulting in a system or service level fault that crashes the targeted service or application and which require administrative action to restart the service or reboot the system. These attacks do not enable the attacker to execute arbitrary commands. However the resulting impact to the business could be severe given that the attacker could crash the protected system or service. Of the 47 vulnerabilities in this category, Retina Scanner detected 43 of 47 (91.5%). It is apparent that eeye maintains a superior vulnerability research team that focuses on the most widely deployed operating systems and applications. We did note some amount of noise, with Retina reporting on vulnerabilities that may have been possible to exploit on the systems in question under the right circumstance, but were not exploitable as we had them configured. Yet it is difficult to fault them for this, since VA products often tout the number of vulnerabilities they are able to identify, and it is not in any vendors interest to potentially under-report, even when a vulnerability is not exploitable due to a configuration lockdown, if it is possible to misconfigure the host in such as way as to enable the vulnerability to be exploited. eeye Retina Network Security Scanner v5.10 VA PCI p. 12

We do believe that the first VA product to provide both a list of vulnerabilities on a host, as well as a list of exploitable vulnerabilities will have a significant advantage in the market since most administrators are time/resource constrained and would jump at the opportunity to reduce the workload from an investigation perspective and patching perspective. 4.5 NSS TEST METHODOLOGIES The following chart depicts the PASS/FAIL status of each NSS Labs test, correlated to the associated PCI DSS section to which it applies. Note that NSS Labs test ID s start with section 5 of this document. There is not always an applicable DSS reference for the test. These NSS Tests reflect recommended features of a product to be used in a payment card environment, and have been included as a best practice. RESULT NSS TEST ID 5.1 Key Management TEST DESCRIPTION PASS 5.1.1 Key Management - Generation of Strong Keys PASS 5.1.2 Key Management - Secure Key Distribution PASS 5.1.3 Key Management - Secure Key Storage PASS 5.1.4 Key Management - Periodic Changing of Keys PASS 5.1.5 Key Management - Changing of Keys Automatically PASS 5.1.6 Key Management - Changing of Keys At Least Annually PASS 5.1.7 Key Management - Destruction and Revocation of Old or Invalid Keys 6 Stability & Capacity PCI DSS ID 3.6 4.1 3.6.1 4.1 3.6.2 4.1 3.6.3 4.1 3.6.4 4.1 3.6.4 4.1 3.6.4 4.1 3.6.5 3.6.9 4.1 6.1 Detection Under Load 6.2 System Capacity - Real-time View of System Utilization 7 Logging and Reporting 7.1 VA Scan Results 10.2.3 7.1.1 Logs - Targeted Vulnerabilities Sorted by IP and Severity 10.2.3 7.1.2 Logs - Severity Levels Rated 10.2.3 7.1.3 Logs - Details for Each Vulnerability Found 10.2.3 7.1.4 Logs - Targeted Vulnerability Name 10.2.3 PASS 7.1.5 Logs - Industry Reference Numbers 10.3.2 7.1.6 Logs - Severity Level 10.2.3 7.1.7 Logs - Comprehensive Explanation 10.2.3 7.3 Updates and Configuration Changes Logging and Reporting eeye Retina Network Security Scanner v5.10 VA PCI p. 13

RESULT NSS TEST ID TEST DESCRIPTION PCI DSS ID 7.3.1 Change Logs - User Identification 10.2.7 7.3.2 Change Logs - Type of Event 10.2.7 7.3.3 Change Logs - Date and Time 10.2.7 7.3.4 Change Logs - Success or Failure of an Action 10.2.7 7.3.5 Change Logs - Origination IP Address 10.2.7 7.3.6 Change Logs Resource Affected 10.2.7 PASS 7.4 Synchronization of System Clock 10.4 PASS 7.5 Centralized Logging Over Secured Communications Channels 10.5 10.5.3 PASS 8 Patches and Updates 6.1 8.1 Support Secure, Non-refutable Updates 6.1 8.2 Online Updates 8.2.1 Online Updates - Using Hosted Provider 6.1 8.2.2 Online Updates - Not Susceptible to Man in the Middle Attacks 6.1 8.3 Offline updates 8.3.1 Offline Updates Removable Media 6.1 8.3.2 Offline Updates - Digitally Signed and Encrypted 6.1 9 Management & Administration 9.1 PCI Default Configuration - No Default Usernames / Passwords 9.2 Password Policy 2.2 8.5.1 9.2.1 Password Policy Password Length 8.5.10 9.2.2 Password Policy Enforces Non Alpha-Numeric 8.5.11 9.2.3 Password Policy Altering Case 8.5 9.2.4 Password Policy No Consecutive Repeating Characters or Sequences 8.5 9.2.5 Password Policy Password Expiration 90 Day Max 8.5.9 9.2.6 Password Policy No Repeat of Last Four Password 8.5.12 9.3 No Shared User Accounts 9.4 Two-Factor Authentication 9.5 Secured Management Interface 9.5.1 Separate Interface for Management 2.3 9.5.1 Administrative Access on Trusted Interface 7.2 8.1 8.5.8 7.2 8.2 8.3 2.3 8.4 2.3 10.5.1 eeye Retina Network Security Scanner v5.10 VA PCI p. 14

4.6 RECOMMENDED CONFIGURATIONS Very few products, if any, are ready to be installed directly out of the box. Furthermore, PCI DSS calls for a number of specific settings and configurations to be implemented in order to support compliance. Identifying which settings are required is a non-trivial task, especially given the wide variety of product types, and the plethora of product manufacturers, each with a number of distinct product lines and products. Thus, enabling a specific setting can vary greatly across products and vendors. NSS Labs reports for PCI strive to simplify the process of configuring a product to support PCI compliance. Therefore, we have included recommended configuration settings in each report. These are presented in short-hand with the intention of guiding a knowledgeable administrator where to find the specific settings. These can either be validated or modified as needed. In the following sections, PCI Test Methodologies are listed with details of the tests performed and the result. The appropriate audit reference and steps to view/modify the recommended configuration are included in the right side of the table. eeye Retina Network Security Scanner v5.10 VA PCI p. 15

5 KEY MANAGEMENT & SCANNING ENCRYPTED PAGES The VA Scanner must support strong cryptography and security protocols such as secure sockets layer (SSL) to safeguard sensitive cardholder data during transmission over open, public networks. 5.1 KEY MANAGEMENT Key Management is a crucial part of PCI compliance. The VA Scanner must be capable of supporting and enforcing diligent policies related to key management. 5.1.1 KEY MANAGEMENT - GENERATION OF STRONG KEYS The VA Scanner must be support the generation of strong keys. PASS 5.1.2 KEY MANAGEMENT - SECURE KEY DISTRIBUTION The VA Scanner must be support and enforce the secure distribution of keys. PASS This is accomplished via the REM Management Console 5.1.3 KEY MANAGEMENT - SECURE KEY STORAGE The VA Scanner must be support and enforce the secure storage of keys. PASS This is accomplished via the REM Management Console 5.1.4 KEY MANAGEMENT - PERIODIC CHANGING OF KEYS The VA Scanner must be support and enforce the periodic changing of keys. PASS This is accomplished via the REM Management Console 5.1.5 KEY MANAGEMENT - CHANGING OF KEYS AUTOMATICALLY The VA Scanner must be support and enforce the changing of keys as deemed necessary and recommended by the associated application (for example, re-keying); preferably automatically. PASS This is supported via the Retina scanner as a native function of scanning secure websites, etc. 5.1.6 KEY MANAGEMENT - CHANGING OF KEYS AT LEAST ANNUALLY The VA Scanner must be support and enforce the changing of keys on an annual or shorter period. PASS This is accomplished via the REM Management Console eeye Retina Network Security Scanner v5.10 VA PCI p. 16

5.1.7 KEY MANAGEMENT - DESTRUCTION AND REVOCATION OF OLD OR INVALID KEYS The VA Scanner must be support and enforce the destruction and revocation of old or invalid keys. PASS This is accomplished via the REM Management Console eeye Retina Network Security Scanner v5.10 VA PCI p. 17

6 STABILITY & CAPACITY The VA Scanner is required to maintain security effectiveness and continue to detect vulnerabilities even when the network becomes congested. 6.1 DETECTION UNDER LOAD The VA Scanner must not miss vulnerabilities on systems that were previously detected due to system (CPU/Memory) load for any reason. 6.2 SYSTEM CAPACITY - REAL-TIME VIEW OF SYSTEM UTILIZATION The VA Scanner should provide a real-time view of the system utilization that correlates to the measured performance thresholds. This feature is available both in the Retina scanner and in REM management console eeye Retina Network Security Scanner v5.10 VA PCI p. 18

7 LOGGING AND REPORTING 7.1 VA SCAN RESULTS 7.1.1 LOGS - TARGETED VULNERABILITIES SORTED BY IP AND SEVERITY The VA Scanner must log the targeted vulnerabilities sorted by IP address and severity, with the most critical vulnerabilities listed first. The product does provide this capability. 7.1.2 LOGS - SEVERITY LEVELS RATED Severity levels should be rated in accordance with the NIST CVSS standards and have a CVSS value assigned. The product does provide this capability. 7.1.3 LOGS - DETAILS FOR EACH VULNERABILITY FOUND The VA Scanner must log the details of each vulnerability found. The product does provide this capability. 7.1.4 LOGS - TARGETED VULNERABILITY NAME The VA Scanner must log the targeted vulnerability name. The product does provide this capability. 7.1.5 LOGS - INDUSTRY REFERENCE NUMBERS The VA Scanner must log the industry reference numbers such as CVSS, CVE, CAN, or Bugtraq ID. PASS The product does provide this capability. 7.1.6 SEVERITY LEVEL The VA Scanner must log the severity level of the event. The product does provide this capability. eeye Retina Network Security Scanner v5.10 VA PCI p. 19

7.1.7 LOGS - COMPREHENSIVE EXPLANATION The VA Scanner must log a comprehensive explanation of the event. The product does provide this capability. 7.2 ADMINISTRATIVE ACCESS LOGGING AND REPORTING The VA Scanner must log all actions by users with administrative privileges including modifications to any system or application logs. The product does provide this capability via the REM Management Console 7.3 UPDATES AND CONFIGURATION CHANGES All updates and configuration changes to the VA Scanner must be logged. 7.3.1 CHANGE LOGS - USER IDENTIFICATION The VA Scanner must log the identity of the user who caused the event. The product does provide this capability via the REM Management Console 7.3.2 CHANGE LOGS - TYPE OF EVENT The VA must log the type of event. The product does provide this capability via the REM Management Console 7.3.3 CHANGE LOGS - DATE AND TIME The VA must log the date and time of the event. The product does provide this capability via the REM Management Console 7.3.4 CHANGE LOGS - SUCCESS OR FAILURE OF AN ACTION The VA Scanner must log the success or failure of the action. The product does provide this capability via the REM Management Console 7.3.5 CHANGE LOGS - ORIGINATION IP ADDRESS The VA Scanner must log the source IP address of the event. The product does provide this capability via the REM Management Console eeye Retina Network Security Scanner v5.10 VA PCI p. 20

7.3.6 CHANGE LOGS RESOURCE AFFECTED The VA Scanner must log the resource affected by the event. The product does provide this capability via the REM Management Console 7.4 SYNCHRONIZATION OF SYSTEM CLOCK The VA Scanner must support the synchronization of system clock to facilitate accurate log entries. PASS The product does provide this capability via native Windows NTP as well as via the REM Management console 7.5 CENTRALIZED LOGGING OVER SECURED COMMUNICATIONS CHANNELS The VA Scanner must support centralized logging over secured communications channels. PASS The product does provide this capability via the REM Management Console eeye Retina Network Security Scanner v5.10 VA PCI p. 21

8 PATCHES AND UPDATES Ensure that all system components and software have the latest vendor-supplied security patches installed. Install relevant security patches within one month of release. PASS This test validates the VA Scanner s ability to determine the OS/Application build level and whether or not a vulnerability is present that would require a software patch. 8.1 SUPPORT SECURE, NON-REFUTABLE UPDATES Must support secure, non-refutable updates such as firmware, software, signature, or database updating. Updates are secured by a digital signature 8.2 ONLINE UPDATES 8.2.1 ONLINE UPDATES - USING HOSTED PROVIDER Online updates using a hosted provider. Secure connection to eeye internet portal 8.2.2 ONLINE UPDATES NOT SUSCEPTIBLE TO MAN IN THE MIDDLE ATTACKS Online updates must not be susceptible to man in the middle attacks. Updates are secured by a digital signature 8.3 OFFLINE UPDATES 8.3.1 OFFLINE UPDATES REMOVABLE MEDIA Offline updates using removable media or localized network connection. The product does provide this capability via the REM Management Console 8.3.2 OFFLINE UPDATES - DIGITALLY SIGNED AND ENCRYPTED Offline updates must be digitally signed and encrypted. Updates are secured by a digital signature. eeye Retina Network Security Scanner v5.10 VA PCI p. 22

9 MANAGEMENT & ADMINISTRATION 9.1 PCI DEFAULT CONFIGURATION - NO DEFAULT USERNAMES / PASSWORDS Upon initial setup of the VA Scanner, the administrator should be forced to change the default administrative user parameters and password. This is due to the sensitive nature of the content being stored by the VA Scanner and/or Management Console. FAIL * This is something to verify if you are a PCI Assessor. Otherwise, failing this requirement is not cause for failure of the product since this is an implied requirement of a VA scanner, not a direct requirement. 9.2 PASSWORD POLICY The VA must support the enforcement of password policies. 9.2.1 PASSWORD POLICY PASSWORD LENGTH The VA Scanner must be able to determine whether or not users have been required to create new passwords with a minimum length of seven characters. Through Retina / REM, it is possible to set the password policy for the hosts being scanned and to verify that the hosts are abiding by the policy. 9.2.2 PASSWORD POLICY ENFORCES NON ALPHA-NUMERIC The VA Scanner must be able to determine whether or not users have been required to create new passwords containing non alpha-numeric characters. PASS Through Retina / REM, it is possible to set the password policy for the hosts being scanned and to verify that the hosts are abiding by the policy. 9.2.3 PASSWORD POLICY ALTERING CASE The VA Scanner must be able to determine whether or not users have been required to create new passwords which include both UPPERCASE and lowercase letters. Through Retina / REM, it is possible to set the password policy for the hosts being scanned and to verify that the hosts are abiding by the policy. 9.2.4 PASSWORD POLICY NO CONSECUTIVE REPEATING CHARACTERS OR SEQUENCES The VA Scanner must be able to determine whether or not users have been prevented from creating passwords containing repeated or sequential characters (i.e 1111 or 1234. Through Retina / REM, it is possible to set the password policy for the hosts being scanned and to verify that the hosts are abiding by the policy. eeye Retina Network Security Scanner v5.10 VA PCI p. 23

9.2.5 PASSWORD POLICY PASSWORD EXPIRATION 90 DAYS MAXIMUM The VA Scanner must be able to determine whether or not users have been required to create new passwords no less often than every 90 days. Through Retina / REM, it is possible to set the password policy for the hosts being scanned and to verify that the hosts are abiding by the policy. 9.2.6 PASSWORD POLICY NO REPEAT OF LAST FOUR PASSWORDS The VA Scanner must be able to determine whether or not the password policy enforces that a new password not repeat any of the last four passwords. Through Retina / REM, it is possible to set the password policy for the hosts being scanned and to verify that the hosts are abiding by the policy. 9.3 NO SHARED USER ACCOUNTS The VA Scanner should support the creation of unique, non-shared IDs (i.e. one user account per administrative user), to be used for administrative changes. This is supported through REM Management Console 9.4 TWO-FACTOR AUTHENTICATION The VA Scanner should support the use of two-factor authentication on the management interfaces utilizing technologies such as TACACS and RADIUS. This is supported through REM Management Console. 9.5 SECURED MANAGEMENT INTERFACE The VA Scanner must not respond to unencrypted management protocols (SNMP, etc) on external (i.e. untrusted) interfaces. This can be set in both Retina and REM 9.5.1 SEPARATE INTERFACE FOR MANAGEMENT The VA should utilize a separate interface for management with a private IP address reachable only from a directly connected trusted management network. Since both Retina and REM are installed on Microsoft Windows Operating Systems, it is possible (and recommended by eeye) that management of the VA Scanner occur over a dedicated management interface to avoid interfering with ongoing scans. eeye Retina Network Security Scanner v5.10 VA PCI p. 24

APPENDIX A: TEST INFRASTRUCTURE Special thanks go to our test infrastructure partners who provide much of the equipment, software, and support that make this testing possible: eeye Retina Network Security Scanner v5.10 VA PCI p. 25