Acceptance Criteria for Penetration Tests According to PCI DSS Requirement 11.3 of the PCI DSS (Version 1.2.1, July 2009) defines the regular performance of penetration tests for all systems in scope as well as for all relevant network segments. Although the PCI DSS provides the precise framework by which penetration tests are to be conducted, it doesn t give any specifications regarding the methodology that has to be applied or the attack vectors that have to be chosen during the penetration test. Furthermore, acceptance criteria by which a penetration test can be evaluated as acceptable for PCI DSS conformity are not to be found in the standard itself. Therefore, the subject is further pointed out in the following, clarifying which aspects a penetration test must fulfil in order to meet the requirements of the PCI DSS. 1 Scope Requirement 11.3.2 specifies that the application layer has to be part of the scope of the penetration tests. However, the requirement does not make gradations regarding the systems that have to be examined. Thus, all systems in scope are to be examined likewise, independent if these store or process card data or if not. Therefore, systems which do not process card data but are in scope because e.g. they are in the same network segment or fulfil a decisive security function, are to be covered entirely by penetration tests. A web server which is accessible via Internet and a corresponding data base which is located behind a DMZ according to requirement 1.3.7 may serve as an example. Both systems are connected to an active directory-server whereby it is likewise assigned to the scope of both systems. A penetration test of the web server and its data base must therefore also include the active directory-server. In case a reporting-system, which does not process card data and does not access the data base, is located in the network segment of the data base it is also part of the scope due to its location in the same network segment and must therefore be covered by the penetration test. Furthermore, requirement 11.3.1 specifies that the network layer must also be an integral part of the penetration tests. Here, it must be considered that the network penetration test according to requirement 11.3.1 also has to cover network components. Therefore, firewalls, switches, routers, and network appliances within the PCI DSS relevant network segments have to be included as well. These in-scope network segments are thereby separated from out-of-scope network segments by a firewall that is operated in a PCI DSS compliant way. In the above mentioned example of the web server and its data base, firewalls that separate the web server and the data base are thus to be covered by the penetration test. Also, all routers and switches which control the data traffic between the web server, its data base and other relevant systems (such as the active directory server) are to be covered by the penetration test as well. Penetration tests have to be conducted not only in annual rhythm but according to requirement 11.3 also after each significant change of the technical infrastructure or the systems. Significant changes according to the definition in requirement 11.3 are for example: Upgrades of software, e.g. of the operating system from WinXP to Vista, of the firmware from firewall release 3.0 to 4.0 as well as version changes of applications such as the update from WinXP with Service Pack 2 to Service Pack 3, the change from Apache 1.3 to 2.0 or the change from Oracle 10.g to 11.i; 29. November 2010 SRC Security Research & Consulting GmbH Page 1 of 5
the exchange or adding of hardware components (such as gateways or network appliances, but not of defect hard disks or network interface cards); the adding of servers (such as the incorporation of a new reporting server into the PCI DSS environment of a data warehouse); the adding of entire network segments (such as the adding of entire server environments as a result of new business processes). 2 Minimum requirements for PCI DSS compliant Penetration Tests In order to provide users of the PCI DSS assistance in the realisation of PCI DSS compliant penetration tests, the PCI SSC has published a supporting document entitled Information Supplement: Penetration Testing 1. Here, further information about the realisation of the penetration tests that meet the requirements of the standard is given. Further requirements arise implicitly and explicitly from the PCI DSS itself. Thus, for instance requirement 11.3.b explicitly specifies that the penetration tests have to be carried out by qualified personnel. Further requirements only arise implicitly, for instance through the concurrence with other requirements of the PCI DSS. If a penetration test is conducted by an external service provider then requirement 12.8 has to be taken into consideration. In this case the service provider for example has to cover the aforementioned requirement of the qualified personnel. All requirements of the PCI DSS, whether included implicitly or explicitly, shall be presented in a summary below. Here, it has to be taken into account that the below list of criteria is complete at the date of its compilation which may change likewise with changes in the PCI DSS. 1. Realisation of vulnerability scans as starting point of the penetration test: As a rule of thumb it can be recorded that, according to the information supplement, a penetration test begins where a security scan, according to 11.2, ends. For this reason, the starting point for a PCI DSS compliant penetration test ideally is the initial realisation of a vulnerability scan for the gathering of information on the system. Attention should be paid to the fact that a penetration test is grounded on the results of the vulnerability scan and therefore cannot be terminated if the vulnerability scan does not bring forward obvious vulnerabilities. Rather, the penetration tester picks up the results of the scan to detect individual attack vectors via the manual evaluation of the information on the accessible system which the vulnerability scanner could not detect through its automated approach. The penetration tester uses the system information gathered through fingerprinting, banner detection etc. in order to collect further information on the system via manual actions like the provoking of error messages. As an alternative to the realisation of a vulnerability scan other methodologies for gaining information (Google search, retrieval of public information such as whois-lookups with domain names) are possible, however, a vulnerability scan is suitable as starting point of a penetration test by establishing a substantiated basis of information. 1 Available at https://www.pcisecuritystandards.org/security_standards/pci_dss_supporting_docs.shtml 29. November 2010 SRC Security Research & Consulting GmbH Page 2 of 5
2. Qualified Realisation of the Penetration Test: The penetration tests according to requirement 11.3 don t mandatorily have to be carried out by a QSA or ASV but rather by qualified personnel. If an internal member of staff possesses the expertise for the realisation of penetration tests and is able to verify this (e.g. through the respective individual certification such as Certified Ethical Hacker ) he can conduct the penetration test. Moreover, companies which specialise on the realisation of penetration tests can be contracted to carry out the penetration tests, provided that the requirements for the selection and handling of service providers (according to requirement 12.8) are met. 3. Organisational Independence of the Penetration Tester: In order to counteract the danger of courtesy expertises, penetration tests according to requirement 11.3.b may only be carried out by individuals who possess organisational independence from the organisation that is to be tested. For instance, these can be employees of other companies or company divisions (e.g. a CERT). 4. Precise Definition of a Methodology for the Realisation of a Penetration Test: Qualified penetration tests have to be conducted on the basis of a precisely defined methodology (e.g. the Realisation concept for penetration tests Durchführungskonzept für Penetrationstests of the BSI 2 ). Particularly before the start of the penetration test it has to be defined whether it is a whitebox- or blackbox-penetration test and the course of action has to be defined. Due to the fact that already existing information on systems which have to be tested can be used during a whitebox-test and thus the protracted phase of information gathering of a blackbox-test doesn t apply, whitebox-tests generally are a substantially more efficient method for the realisation of a penetration test. Usually, for this reason, the realisation of a whitebox-test is recommended. A common approach for penetration tests can be found, for example, in the SRC Whitepaper PCI DSS Security Scans & Penetration Tests. 5. Precise Goal of Results of the Penetration Test: In the context of the course of action a precise objective for the results of the penetration test has to be defined. Particularly the subject of the penetration test has to be clearly delimited; it has to be defined for instance, which security aspects are to be checked during the penetration test and if attack scenarios can be disregarded due to comprehensible reasons. Thus, for example, according to the information supplement in most cases it is not necessary to consider the risk of Denial of Service-Attacks (DoS-Attacks) because they pose no threat to the security of card data. The objective availability has to be applied to those systems only whose blackout could promote a compromise of card data (e.g. IDS/IPS-systems). For instance, the following goals for the penetration tests of different systems can be defined: Data base with card data: From the PCI DSS point of view, the availability of the data base as well as the integrity of the contained data is irrelevant (even though from a technical point of view or due to business reasons both can be of vast importance). 2 https://www.bsi.bund.de/contentbsi/publikationen/studien/pentest/index_htm.html 29. November 2010 SRC Security Research & Consulting GmbH Page 3 of 5
Whereas the confidentiality of the contained card data is of paramount importance. As goal for the penetration test of the data base the testing of the confidentiality protection should be in the foreground. Antivirus-Server: Confidentiality plays a subordinate role concerning the antivirusserver. Here, the safeguarding of the server s availability as well as the integrity of the used signatures respectively patterns is much more important. This should be considered accordingly during the definition of the goal. SFTP-Server that is being used for the transmission of card data: Similar to the case of the data base, here again, from the PCI DSS point of view, the confidentiality of the card data which is being transmitted or rather (buffered) stored with the help of the SFTP-server is in the foreground. From a technical point of view the integrity of the transmitted data and the availability of the service are possibly of paramount importance which is why it can be reasonable to cover it as well. However, from the PCI DSS point of view, likewise the example of the data base, this is not mandatory. Furthermore, criteria for the termination of the penetration test in case of failure of all performed attacks have to be defined. 6. Precise Definition of Attack Scenarios: During the planning of the penetration test a precise definition of the covered attack scenarios is to be conducted. According to requirement 11.3 these must cover network- and application layer and include all systems in scope. In case of high complexity of the technical infrastructure and/ or a multitude of involved systems it can be advantageous to carry out several penetration tests, each covering only one or few subsystem(s), instead of one penetration test covering all systems. Examples of possible attack scenarios include: - Attacks from the Internet (e.g. SQL-Injection) - Compromise through Trojans - Data theft (e.g. from a data base or SFTP-server) - DoS- or DDoS-attacks against systems with security functions (e.g. antivirus, IDS/IPS, etc.) - Internal attacks by discontent employees (e.g. placement of logical bombs, violation of administration rights, etc.) - Economic espionage 7. Internal and External Penetration Tests: In case systems and system environments are accessible not only internally but also externally (for example through service providers or publicly over the Internet) the penetration test according to requirement 11.3 has to include attacks from the inside as well as from the outside. In this case, if applicable, multiple attack scenarios as described under item 6 are necessary (e.g. one internal and one external attack scenario). 8. Precise Definition of Minimal Requirements of Penetration Tests: During the definition of the attack scenario minimal requirements of the penetration test are to be defined which 29. November 2010 SRC Security Research & Consulting GmbH Page 4 of 5
have to be tested in either case. These are, for instance, common attack methodologies that have to be tested in either case. These depend in each case on the examined system and cannot be provided in a generic way. Thus, for example, in the case of web applications at least the so called OWASP Top 10 in their respective up to date version have to be stringently considered according to requirement 11.3.2 3. Hereby, it has to be considered that these have to be included not only for the applications accessible over the Internet but also for each application that has been programmed on web technology (this also includes for instance browser based admin-interfaces, etc.). 9. Comprehensible Realisation and Documentation: The process of a penetration test has to be documented and designed in a comprehensible manner in order to allow the auditor an evaluation of the penetration test, its underlying methodology as well as the results. This particularly includes a precise documentation of methodology, goal, attack scenario(s), and minimal requirements of the penetration test (see topic 4 through 8) as well as a thorough documentation of the test progress and all results. 10. Elimination of Vulnerabilities Found & Retest: Vulnerabilities that have been found during the penetration test have to be eliminated according to requirement 11.3.a whereupon a retest of the corrections in form of a new penetration test has to be carried out. During the elimination of vulnerabilities it has to be taken into consideration that any changes on systems or system components always have to be carried out in line with the change management process according to requirement 6.4. Also, the changes on the basis of vulnerabilities found therefore have to pass through the regular change management process of the organisation and must be documented and approved accordingly. The retesting of the corrections that has to be carried out doesn t have to take place in the form of a complete penetration test but can be limited to the systematic testing of the eliminated vulnerabilities. Here, it has to be considered that this retesting also has to be conducted by a qualified and organisationally independent person. 3 http://www.owasp.org/index.php/category:owasp_top_ten_project 29. November 2010 SRC Security Research & Consulting GmbH Page 5 of 5