D5.1 Security Testing Methodology

SEVENTH FRAMEWORK PROGRAMME Theme SEC-2011.2.5-1 (Cyber attacks against critical infrastructures) D5.1 Security Testing Methodology Contract No. FP7-SEC-285477-CRISALIS Workpackage WP 5 - Vulnerability Discovery Author Marco Caselli, Frank Kargl Version 1.0 Date of delivery M12 Actual Date of Delivery M12 Dissemination level Public Responsible UT The research leading to these results has received funding from the European Community s Seventh Framework Programme (FP7/2007-2013) under grant agreement n 285477.

SEVENTH FRAMEWORK PROGRAMME Theme SEC-2011.2.5-1 (Cyber attacks against critical infrastructures) The CRISALIS Consortium consists of: Symantec Ltd. Project coordinator Ireland Alliander Netherlands Chalmers University Sweden ENEL Ingegneria e Innovazione Italy EURECOM France Security Matters BV Netherlands Siemens AG Germany Universiteit Twente Netherlands Contact information: Dr. Corrado Leita 2229 Route des Cretes 06560 Sophia Antipolis France e-mail: corrado_leita@symantec.com Phone: +33 673 41 91 27

Contents 1. Introduction 6 1.1. General Classification............................. 7 1.2. Standardization................................. 9 1.3. Testing Critical Infrastructures........................ 10 2. Standard 12 2.1. OSSTMM.................................... 12 2.1.1. General information.......................... 12 2.1.2. Overview................................ 14 2.2. NIST SP800-115................................ 19 2.2.1. General information.......................... 19 2.2.2. Overview................................ 22 2.3. ISSAF...................................... 29 2.3.1. General Information.......................... 29 2.3.2. Overview................................ 31 3. Requirements 37 3.1. Approach.................................... 37 3.2. Questionnaire.................................. 38 A. General information.............................. 38 B. Security requirements, design, and regulations................ 40 C. Testing and methodologies........................... 41 D. Research..................................... 46 E. Follow up.................................... 46 3.3. Further analyses................................ 46 4. Methodology 50 4.1. Overview.................................... 50 4.2. Workflow.................................... 51 4.2.1. Main Workflow............................. 51 4.2.2. Workflow phases............................ 53 3

4.2.3. Penetration Testing Steps....................... 57 4.3. Final Remarks................................. 60 5. Conclusion 63 4

Abstract Existing security testing methodologies and tools cannot be applied directly to critical environments, due to a number of safety and availability requirements. One of the goal of WP3 is to understand what assessment processes we can exploit in Industrial Control System (ICS) environments and what are the constraints given by industrial components and infrastructures. This document provides an overview of assessment methodologies used in standard IT. We analyze these methodologies to investigate their applicability in an ICS environment. We are also conducting a survey among stakeholders in order to detail existing best practices adopted by companies to test their infrastructures and discover vulnerabilities in their systems. From this we are proposing a new methodology tailored to support security testing in Critical Infrastructure environments. As the evaluation of the feedback and the methodology is requiring more work than originally foreseen and as our on-going investigations are also bringing up new aspects not foreseen in the original proposal (e.g., incident handling), this deliverable should be regarded only as a first version of D5.1 that we intend to extend in a version 2 either after year 2 or at the end of the project in order to address these additional aspects.

1. Introduction Security testing involves various techniques which complement security engineering, i.e., design and implementation of secure systems. In contrast, security testing implies testing of those systems with regards to their security properties. Security testing includes analysis-centered activities like a code review but also practical tests to penetrate a security system and find vulnerabilities and implement attacks. The latter is often termed penetration testing. Per definition, a penetration test is a simulation of an attack aimed at assessing the security of a computer system or a network. Simulation refers to the fact that the test is conducted by the system owner or on his request. This test has multiple phases and allows to highlight infrastructures and protocols weaknesses. During a penetration test, the penetration tester tries to discover and sometimes exploit known or unknown vulnerabilities in order to obtain useful information or to access targeted hosts. At the end, all security problems discovered are reported to the system owner together with an assessment of their impact on the system. The analysis should also propose technical or organizational mitigations to identified problems. In its most simplistic form, a penetration test only involves running some automated security scanner, like, e.g., Nessus [1]. However, it is generally agreed that serious penetration testing typically involves also manual test planing, preparation, and conduction to be able to flexibly react to the system under test. In such a case, the penetration tester should follow some structured methodology to ensure valid, reproducible, and well documented results. The development and use of such manual penetration testing methodologies is motivated by several reasons. First of all, there are numerous vulnerabilities that may be difficult or impossible to detect with automated tools. Network or application vulnerability scanning software are nowadays highly sophisticated. However, sometimes, high profile systems and infrastructures need a comprehensive and systematic approach to be compromised which cannot be achieved by automated tests. This is, e.g., related to the detection of high-risk vulnerabilities resulting from a combination of low-risk ones. As multiple minor weaknesses are exploited in a particular sequences, a penetration test may reveal all the possibilities identifying the possibly dangerous combinations. More importantly than identifying vulnerabilities, determining the feasibility of a particular set of attack vectors is another important target of a penetration test. As having 6

1.1. General Classification theoretically unassailable systems seems impossible in practice, companies and organizations are interested in running infrastructures that are practically secure. Therefore, the analysis becomes a way to rank vulnerabilities both by severity and realizability. The potential business impact is a further valuable indicator of importance for a vulnerability. Conducting practical attacks also aims at properly identifying necessary attacker effort which is a prerequisite for correct risk estimation. Assessing the magnitude of economical and operational problems related to a successful cyber attacks is usually the last phase of analysis of a penetration test. Based on the knowledge of business processes such analysis gives an accurate picture of the risk related to the infrastructure and to the work such infrastructure performs. Finally, penetration tests are the experiments with which companies try out the ability of network defenders to successfully detect and respond to cyber attacks. Their capacity of reaction is measured during the test and evaluated at the end of it. It will also provide evidence to support increased investments in security personnel and technology or it will suggest modifications in methods and strategies are the output of such kind of assessments. 1.1. General Classification Penetration tests can be performed in a broad variety of ways. The following criteria provide some orientation and classification. The amount of information known by the fake attackers is one of the main discriminating factor of penetration tests. The simplest classification describes three different types of tests: white-box, black-box, and gray-box. White-box penetration testing outlines a situation in which fake attackers have full knowledge of the system under attack. The goal of a white-box penetration test is to simulate the presence of an expert malicious insider. In some special cases such insider has also basic credentials for the target system. Among the information known to the fake attacker we can have: network diagrams, domain names, IP addresses, but also phone numbers, e-mail addresses. White-box testing of applications usually allows intruders to have also programs source code. White-box test design techniques include [2]: ˆ Control flow testing ˆ Data flow testing ˆ Branch testing FP7-SEC-285477-CRISALIS 7

1. Introduction ˆ Path testing ˆ Statement coverage ˆ Decision coverage Black-box penetration testing is the opposite of the previous one. This method examines infrastructures and applications functionalities without knowing anything about them. In general, the tester is aware of what the software is supposed to do but is not aware of how it does it [3]. Thus, specific inputs (e.g network packets, shell commands, etc.) return specific outputs without the intruder knowing how the mechanisms within the system work. The goal of a black-box penetration test is to simulate a typical external hacking or also cyber warfare scenario. Typical black-box tests design techniques include [4]: ˆ Decision table testing ˆ All-pairs testing ˆ State transition tables ˆ Equivalence partitioning ˆ Boundary value analysis Finally, Gray-box penetration testing is a combination of white-box and black-box methods. A gray-box tester partially knows the internal structure of the system under attack. This knowledge can include network topology, installed applications or also more in deep information like used algorithms and data structures. Since any combination of knowledge owned by the testers falls in the domain of gray-box penetration testing this method is the most widely used, especially in the field of web applications. Gray-box testing techniques include [5]: ˆ Matrix Testing ˆ Regression testing ˆ Pattern Testing ˆ Orthogonal array testing 8 SEVENTH FRAMEWORK PROGRAMME

1.2. Standardization Other ways to characterize penetration tests include the type of system access (external network, internal network, physical access), whether a test is conducted in pure physical ways or whether social engineering attacks are also performed, or whether testing includes performing (possibly disruptive) attacks or whether such attacks should only be identified but not performed. The specifics of a test need to be carefully defined in the test plan. 1.2. Standardization In this last decade the interest in penetration testing techniques has increased. Companies, organizations and governments provide numerous services on the Internet and need to assess the security of their systems. Given the proliferation of different penetration testing methods, some organizations and standardization bodies decided to propose comprehensive and usable methodologies. While many penetration testers follow a home-brewed self-defined testing approach, following standardized approaches has some general benefits. A standardized and generally accepted penetration methodology provides a certain confidence that no important steps or aspects of a test are forgotten by accident. As tests should be repeated after significant changes to the system under test, following the same methodology also provides the possibility to compare multiple tests to identify whether a system got more secure after mitigating measures have been taken. A standardized approach may also be required in cases where security tests are required from a legal or insurance perspective. A standardized approach and common reporting forms also allows to compare results with results from other installations. Finally, accepted standards simplify negotiations between a security tester and clients about the scope of such tests. So there are a number of good reasons to look into such standardized and commonly agreed approaches. Over the years, many standardized methodologies have been proposed. From most general to system-specific ones, both governments and private organizations have proposed different ways to approach penetration tests. Among the most well-known penetration testing techniques are: OSSTMM (Open Source Security Testing Methodology Manual), NIST SP800-115, ISSAF (Information Systems Security Assessment Framework), OWASP (Open Web Application Security Project), and PTES (Penetration Testing Execution Standard). Later in this document, we are going to give a broad overview of some of these approaches. In addition to methodologies, different kind of systems exist to assess specific contexts of security testing without a structured approach. Companies can decide not to follow a FP7-SEC-285477-CRISALIS 9

1. Introduction methodology but just perform a light assessment using different methods. In this cases they can exploit several guidelines and toolkits developed for numerous and different purposes. According to [6] toolkits implement and bundle in a convenient package a set of testing techniques, usually aimed at discovering specific classes of security problems. Guidelines instead organize the process of security testing, by collecting sets of best practices, comprehensively listing items to be tested, and structuring any other kind of useful advice. Finally, there are also collections of penetration testing tools such as Backtrack Linux or metasploit that are regularly used by penetration testers. 1.3. Testing Critical Infrastructures Despite the recent interest for penetration testing methodologies, none of the proposed methodologies specifically considers industrial control systems and critical infrastructures despite the fact that penetration testing is also performed in such environments on a routinely basis. We argue that standard methodologies should not be applied there without special consideration. These systems and the networks that interconnect them show many differences compared with standard IT networks. Since critical infrastructures control and automate real world processes or equipments which are potentially dangerous for humans, any activity different from the main purpose including penetration testing should be limited. So just running a full Nessus or Metasploit test in such an environment could have disastrous consequences. There are three main concerns that make penetration testing on critical infrastructures challenging. As introduced in the previous paragraph, any penetration attempt can have a consequence in the real world. What follows is an interesting example of such concerns. In a real incident, a gas utility conducting a penetration test on its corporate network performed this in a careless manner affecting some of the DCS systems. During the test this DCS system was locked up by mistake and the utility was not able to send gas through its pipelines for four hours leading to a large economic loss [7]. The strong relation between the system under test and physical components that people depend on is the first important difference with respect to standard IT. This implies that any CI penetration testing methodology needs to include safeguards to protect from such failures. Another important difference concerns the devices used in Critical Infrastructure, these systems usually have very limited resources compared to devices found in normal IT environments. The longevity of network components is greater and the cycle of replacement of devices may also be in the order of decades. This implies that devices 10 SEVENTH FRAMEWORK PROGRAMME

1.3. Testing Critical Infrastructures may be out of service, no patches may be available, and generally proposing mitigation strategies may take different approaches. This also means that critical infrastructures can be more easily overloaded with respect to other IT systems. A penetration testing methodology has to take care not to override the normal operation of every component in the ICS unless identifying DoS attacks is specifically in the scope of the test. Finally, due to the size of most of the critical infrastructures, a penetration testing has always to check if localized attacks have effects in different parts of the network which may be hard to assess due to indirect effects. At the end of a test the identification of a vulnerability is followed by the calculation of the damage that it can cause. Linking these two elements in a system with thousands of components requires the assistance of experienced personnel both in the IT field and in the field related to the industrial process under control. As a consequence, penetration tests cannot be conducted by security experts in isolation. We think that a penetration testing methodology for critical infrastructure must explicitly describe how to deal with the previous issues. Moreover, it has to identify specific methods, tests, and tools already used in standard penetration testing techniques and explain how to tune these in order to be suitable for industrial control systems. Finally, such an ICS penetration testing methodology must provide where necessary new instruments and customized methods to test IT and networking components not found in standard ICT (e.g., Process Control Units, Remote Terminal Units, field networks, etc.). FP7-SEC-285477-CRISALIS 11

2. Standard In this chapter, we are going to review multiple penetration testing methodologies proposed by standardization bodies and other organizations. The aim is to provide a broad overview over the state of the art before defining specific requirements for CI penetration testing and then proposing an own CRISALIS penetration testing methodology for Critical infrastructures that is distilled from those existing approaches. 2.1. OSSTMM 2.1.1. General information The Open Source Security Testing Methodology Manual (OSSTMM) [8] is an open standard methodology for security tests. Developed by Pete Herzog at the end of 2000 as an ethical hacking framework, it has rapidly grown to become a comprehensive methodology to assure security at operational level. Version 3, released in 2008, encompasses tests for every security aspect: from personnel qualification to physical security, from control of communication to electronic systems safety. As every standard methodology, it is designed to be consistent and repeatable. Moreover, it is openly available and thus allows a free dissemination and free use. Figure 2.1 shows a comparison among OSSTMM and common security tests in terms of accuracy and thoroughness. By its nature, the methodology can be adapted to any kind of audit operation (e.g. penetration tests, ethical hacking, security and vulnerability assessments, red/blue teaming, etc.). The primary purpose always remains the description of a security examination path and the definition of a way to consistently correlate test results. The usage of the OSSTMM allows the auditor to perform a validated set of tests and to qualify its examination as a certified OSSTMM audit. Differently from other methodologies, mitigations to security flaws are not required as part of an OSSTMM audit. Recommendations may well be part of the final results, but common operational situations often imply that auditors have a limited view and knowledge of the client environment. In these cases it is difficult to propose proper and feasible solutions. For this reason solutions are usually avoided in final reports but should we worked out together with the system owner after the test. This is an aspect 12

2.1. OSSTMM Figure 2.1.: OSSTMM comparison with common security tests (from [8]) that is well in line with our findings from the previous chapter as it will also apply to CI. An important aspect of the OSSTMM is the compliance with general security policies. The methodology is developed taking into account major legislation and regulations. Furthermore, OSSTMM defines three different types of compliance and a set of rules to deal with all of them. The three types of compliance defined in the methodology are: ˆ Legislation: enforces the security test to be compliant with region/country legislation. The lack of this requirement could lead to criminal charges. ˆ Regulation: enforces the security test to be performed accordingly to standard regulation within industry sector. A failure in this task could lead to a dismissal of the company from the group. ˆ Policy: enforces the security test to be done according to business and organization policies. Violation could cause a dismissal of the responsible from the company. OSSTMM provides a list of national and international regulations already proved to be compliant to the proposed methodology. FP7-SEC-285477-CRISALIS 13

2. Standard 2.1.2. Overview Before describing the workflows, the OSSTMM introduces several concepts to outline what kind of security tests can be performed and what the scope of such tests can be. Figure 2.2 presents the six different security test types defined by OSSTMM. Possible security tests are however not limited to these cases and can be tuned to more specific requirements. Figure 2.2.: OSSTMM security test types map (from [8]) ˆ Blind: this type of security test describes a situation in which the attacker does not know anything about the target. On the other side, the defender knows everything about the audit operations and it is prepared to take countermeasures. A blind security test is primarily used to evaluate attacker skills. ˆ Double Blind: as before the attacker does not know anything about its target but, this time, the target is not notified in advance about the security test. A double blind audit is used to test both the skills of the attacker and the preparedness of the defender. ˆ Gray Box: in a gray box security test the attacker has limited knowledge of the target and can better engage the defender. The defender still has the advantage of full knowledge about the audit in progress. As in the blind security test, this case is mostly used to evaluate attacker s skills. 14 SEVENTH FRAMEWORK PROGRAMME

2.1. OSSTMM ˆ Double Gray Box: in this type of security test both the attacker and the defender have limited knowledge about their counterparts. Such situation is used to evaluate both the contenders. With respect to double blind this test increases the level of accuracy and pervasiveness. ˆ Tandem: in the Tandem security test the attacker has full knowledge of the target and the defender is fully prepared to bear the test. Such situation is used to improve the thoroughness of the audit as the attacker has a full view of all tests and their responses. ˆ Reversal: as in the previous situation, the attackers knows everything about the target while the defender knows nothing about its contender. The purpose of this test is to evaluate every operational aspect of the defender. Once the test type of interest is identified, it is necessary to identify the scope of the audit. OSSTMM divides such scope into three different channels: COMSEC (communication security), PHYSSEC (physical security), SPECSEC (spectrum security). A thorough security audit would require testing all three channels but usually the tests are tuned on more specific needs. For this reason, the methodology makes a further partitioning, addressing these three channels into five logical sections. Channel PHYSSEC is divided into Human and Physical, channel SPECSEC contains Wireless Communication, and channel SPECSEC is divided into Data Networks and Telecommunications. What follows is a description of the five logical sections: ˆ Human: concentrates on the human element and its communication processes. ˆ Physical: focuses on physical security testing and comprises tangible elements of security where interaction requires physical effort or an energy transmitter to manipulate. ˆ Wireless Communications: concentrates on security tests on wireless electronic communications and signals. ˆ Data Networks: comprises tests on electronic systems and data networks. ˆ Telecommunications: focuses on digital/analog telecommunications where interaction takes place over established telephone network lines. The full methodology is divided into seventeen modules. Every module defines a specific target and several tasks that are needed to achieve it. Each module has an input FP7-SEC-285477-CRISALIS 15

2. Standard and an output. The input represents the information needed to perform each task while the output is the result of the accomplished tasks. Figure 2.3 shows the methodology flow. OSSTMM uses the same workflow for all the channels. The seventeen modules and the methodology flow apply in the same way to the five logical sections. However, while the methodology remains the same, each channel will differ in the proposed tasks. The methodology is divided into four phases: Regulatory Phase (modules 1 to 3), Definitions Phase (modules 4 to 7), Information Phase (modules 8 to 13) and Interactive Controls Test Phase (modules 14 to 17). Each phase faces a different level of depth in the audit. The regulatory phase focuses on understanding the audit requirements, scope and constraints. The definitions phase improves the description of the scope and of the inner interactions and processes. The information phase uncovers misplaced and mismanaged information flows in the target. Finally, the interactive controls test phase focuses on practical penetration testing. This is often the final phase of a security test. This ensures that practical attacks do not affect less invasive tests. In what follows we briefly describe the focus of every module. ˆ Posture Review: it is used to define the scope and identify what tests must be performed. This module focuses on rules, norms, regulations, legislations and policies applicable to the target. ˆ Logistics: it defines which limitations the audit process has. It relates to interaction constraints such as physical distance, communication speed, etc. ˆ Active Detection Verification: it takes into account any practical restriction imposed against performing interactive tests on the targets. ˆ Visibility Audit: it determines available targets within the scope. This module provides the first in depth description of the targets and how they interact with the scope. ˆ Access Verification: it measures the breadth and depth of interactive access points. It verifies if such access point to the target exists and if it requires authentication. ˆ Trust Verification: it identifies trust relationships from and between the targets. This module verifies if targets accept interaction freely and without credentials. ˆ Controls Verification: it measures the effectiveness of process-based attributes like non-repudiation, confidentiality, privacy and integrity. 16 SEVENTH FRAMEWORK PROGRAMME

2.1. OSSTMM Figure 2.3.: OSSTMM methodology flow (from [8]) FP7-SEC-285477-CRISALIS 17

2. Standard ˆ Process Verification: it verifies the existence, effectiveness and maintenance of security levels defined for the targets. ˆ Configuration/Training Verification: it explores the default conditions under which targets operate regularly to understand their intents, business justification and reasoning. ˆ Property Validation: it checks the use of illegal or unlicensed intellectual property or application within the targets. ˆ Segregation Review: it checks the amount of personally identifiable information available depending on the applied rules and legislations. ˆ Exposure Verification: it searches for freely available information which uncovers indirected or unwanted visibility of the targets. ˆ Competitive Intelligence Scouting: it searches for freely available information which could harm or adversely affect the target. ˆ Quarantine Verification: it determines the effectiveness of authentication in terms of black and white listing quarantines. ˆ Privileges Audit: it measures the impact of misusing credentials and privileges and the consequences of unauthorized escalations of privileges. ˆ Survivability Validation/ Service Continuity: it checks the resistance of the target to excessive or adverse changes. ˆ Alert and Log Review/End Survey: it is a review of the performed audit activities. The outcome is a comprehensive assessment of the tested channels. Results are presented through Risk Assessment Values (RAVs) that are a set of security metrics providing an overview on the state of the channels (e.g. measurement of the attack surface, the amount of uncontrolled interactions with a target, etc.). To finally measure the thoroughness of the test performed, the OSSTMM suggests to conclude the work with a Security Test Audit Report (STAR). This step requires to record in the appropriate form the following information: ˆ Date and time of test ˆ Duration of test 18 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 ˆ Auditors and analyst involved ˆ Test type ˆ Scope of test ˆ Test Index ˆ Channel tested ˆ Test Vectors ˆ Verified test and metrics calculations of the operational protection levels, loss controls and security limitations ˆ Knowledge of which tests have been completed, not completed, or only partially completed, and to what extent ˆ Any issue regarding the test and the validity of the results ˆ Test error margins ˆ Any processes which influence the security limitations ˆ Any unknowns anomalies Overall, OSSTMM provides a very comprehensive framework that addresses a large variety of practical aspects. It is specifically useful to ensure completeness of our later approach. However, it is also relatively generic and is not going into details of specific systems and is not addressing the CRISALIS scenarios at all. 2.2. NIST SP800-115 2.2.1. General information The National Institute of Standard and Technology (NIST) has designed its own security assessment methodology in the Special Publication 800-115 [9]. The purpose of this publication was to provide guidelines for organizations on planning, conducting and evaluating information security testing. Even if the overall goal is to propose an overview of main key elements of technical security assessments, NIST SP800-115 provides also practical recommendations and technical information relating to penetration tests. FP7-SEC-285477-CRISALIS 19

2. Standard NIST defines the security assessment as the process of determining how effectively an entity being assessed meets specific security objectives. From an high-level point of view, there are three possible ways to accomplish such target: ˆ through Testing, by exercising the assessment objects comparing actual and expected behaviors ˆ through Examination, by analyzing the assessments objects and understand their functioning ˆ through Interviewing, by conducting extensive discussions with organizations and experts to identify possible problems A methodology that implements one of the previous points has to contain at least three different phases: ˆ Planning: this phase is necessary to collect all the information needed for the assessment. This concerns elements like assets, threats, policies in use, etc. Moreover, the assessment should have a final goal and a set of objectives to work towards. ˆ Execution: this phase concerns the operational part of the assessment. The Execution implements operations of environment discovery and analysis, vulnerability identification and result validation. Practical activities for this phase obviously differ from case to case. However, the main goal is to bring to light problems at technical and organizational level. ˆ Post-Execution: this phase describes all the analysis aim at identifying vulnerability causes and mitigation strategies. The development of a final report is the main output of the Post-Execution. As already discussed, NIST SP800-115 offers an overview of security assessment key points. However it suggests several practical techniques to implement penetration testing. According to the authors, the most commonly used pentesting techniques can be grouped in the following categories: ˆ Review Techniques define how to examine and analyze assets and their security policies. ˆ Target Identification and Analysis Techniques define how to identify assessments targets and how to describe their properties. 20 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 ˆ Target Vulnerability Validation Techniques: define how to verify the presence of vulnerabilities within the targets and how to test their security levels. The NIST methodology focuses on how these techniques can be used together, but does not provide a list of technical activities to be used in any specific circumstance. In addition to practical penetration testing techniques, the methodology proposes several non-technical methods that can be used to enforce the assessment (e.g. physical security testing). Finally, NIST SP800-115 concentrates on how singular tests and final reports can be compared with each others. The possibility to compare two security assessments relies on the correct representation of the Testing Viewpoint. Every test can be performed from different viewpoints. This implies different information owned by the auditors or different physical location of the pentesters. Results of comparisons among different assessments always need to consider the kind of testing. NIST SP800-115 defines four testing types: ˆ External Security Testing: this type of testing is conducted from outside the organization s security perimeter. The goal of this security test is to reproduce the view of an external attacker and to draw the attention to vulnerabilities already visible from outside the company premises (e.g. from the Internet). External testing always begins with discovery techniques aiming at examining organization public information. ˆ Internal Security Testing: differently from the previous one, this kind of testing is performed within an organization s perimeter (e.g. internal network). The purpose of the Internal Security Testing is to impersonate a trusted insider or also an attacker that has penetrated external defenses. This test allows pentesters to have some level of access to the network and, thus, to internal information. These elements make it possible to assess internal security mechanisms. ˆ Overt Security Testing: also known as White Hat testing, defines an Internal or External Testing in which pentesters have a full knowledge of organization s systems and processes. Company staff is also aware of the test and it usually tries to limit the testing impact. This kind of test is often used as company training. ˆ Covert Security Testing: also known as Black Hat testing, uses an adversarial approach by not giving any knowledge about the company to the pentesters. In this case, company IT staff is usually also not aware of the test and its response is used for testing the technical and organizational security controls within the company. FP7-SEC-285477-CRISALIS 21

2. Standard 2.2.2. Overview In this section we describe in detail the three groups of techniques discussed before and the three phases into which the methodology is logically structured. Review Techniques describe the process of passively gathering information related to the organization and the consequent analysis operations. This information regards both policies and equipments (e.g. systems, networks, etc.). Since these techniques are passive, they do not interfere with company operations and systems. For this reason, this is usually the first phase in every security assessment. Possible review techniques are: ˆ Documentation Review examines the technical aspects of used polices and procedures. Security flaws or weaknesses in such elements can lead to missing or improperly implemented security controls. The results of the analysis can also be used to better tune the following techniques. ˆ Log Review determines if important information and operations are properly logged and recorded. This is a necessary element to prove that all the systems are working according to the applied policies and procedures. Log analysis can show also misconfigured systems and malicious operations (e.g. attempted or successful attacks). ˆ Ruleset Review checks the collection of rules and signature-based security mechanisms used to detect malicious operations or implement organization s processes. Routers, firewalls and intrusion detection systems are the main targets of this analysis. Router access control lists are checked to identify whether rules are no longer needed or enforce the refusal of unknown traffic. The review refines in depth firewall rules by checking unnecessary open ports and allowed services. Moreover, it searches for possible backdoors and malware. Finally, IDS rulesets are verified to be up to date and tuned to organization s needs. ˆ System Configuration Review searches for weaknesses in systems and applications configurations (e.g. programs not enforcing security or non-compliant with security policies). This review usually concerns: unnecessary services running on servers, improper user account and password settings, etc. ˆ Network Sniffing implements the passive monitoring of a data network. It can rely on communication flows, protocols decoding, and header/payload analysis. It is used to map organization communication infrastructures and gathering information about used systems, applications and services. It can also identify unau- 22 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 thorized and inappropriate activities or capture unencrypted personal information (e.g. usernames and passwords). ˆ File Integrity Checking provides a way to check file modifications. It is usually implemented through checksum. Target Identification and Analysis Techniques focus on identifying active devices through open ports and running services and analyze them looking for vulnerabilities. NIST SP800-115 discusses several different techniques belonging to this group: ˆ Network Discovery concentrates on discovering active and responding host on a network. Active techniques send various types of packets to the network waiting for responses. This system is usually an improvement of the passive analysis discussed in the Review Techniques as it is able to find hosts not currently involved in any communication. During this analysis, pentesters have to face firewalls and intrusion detection systems that usually identify many instances of scans. Active discovery generally produce network noise and communication delays. ˆ Network Port and Service Identification is usually performed in parallel with the network discovery. This check involves the use of port scanners able to understand services and application running on remote hosts. Information gathered during the process allows pentesters to exploit OS fingerprinting tools. Such applications guess operating systems working on the network and provide a description about their versions and updates. ˆ Vulnerability Scanning, like the previous two operations, collects information about hosts in the network. Moreover, it also attempts to identify specific vulnerabilities on discovered hosts. Vulnerability scanners usually provide: a check on the compliance of security policies, a list of targets for the following penetration testing, and preliminary information on how to mitigate discovered vulnerabilities. ˆ Wireless Scanning: as the number of wireless devices is increasing, these techniques are becoming more and more important. Wireless scanning techniques can be further divided in: passive scanning, active scanning, bluetooth scanning, and device location tracking. An example of passive scanning involves the analysis of MAC addresses (e.g. vendor identification or white/black listing of devices). An active scanning can be used instead to verify authentication mechanisms and data encryption. Bluetooth security issues have been extensively discussed in literature [10], [11]. Bluetooth scanning is more difficult than wireless one due to the its short range. However, a confirmation of compliance with organization security FP7-SEC-285477-CRISALIS 23

2. Standard requirements is needed to have a comprehensive security assessment. Finally, wireless networks can be used to locate organization s devices. This check enforces the reconstruction of the company s network topology. Target Vulnerability Validation Techniques uses the information found in the two previous phases and further explore the existence of potential vulnerabilities. The target is usually not limited to identifying the vulnerability but also to show the security exposure by trying to exploit it. This procedure is often risky for the organization infrastructure and processes. For this reason is usually the last active work performed on the network. In this way, it will not compromises other tests. NIST SP800-15 defines and implements its penetration testing method in this phase. The targets of the penetration testing can be numerous. Pentesters are interested in understanding how well the system tolerates real attacks, identifying necessary countermeasures, and evaluating defenders abilities. Moreover, this attack simulation can show what level of skills attackers need to compromise organization s systems. The proposed penetration testing is divided in four steps as show in Figure 2.4. Figure 2.4.: NIST SP 800-115 penetration testing steps The Planning step prepares the necessary information and background for the attack. The Discovery step integrates information already owned by pentesters with a new session of network scanning. With respect to the other phases, the attackers try to acquire information about: host names and IP addresses, employee names and other contacts, and system information (e.g. names and shared resources). In some cases also techniques as dumpster diving can be used to enforce the process. The Discovery step improves also the vulnerability analysis and finalizes the list of targets. The Attack step is the core of every penetration testing. During the attack a continuous feedback 24 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 is provided back to the discovery step as succeeding to access a system gives pentesters additional knowledge of the infrastructure under attack. Figure 2.5 shows an example of attack. Figure 2.5.: NIST SP 800-115 attack example (from [9]) NIST SP800-115 organizes the most common vulnerabilities exploited during the Attack step in the following categories: ˆ Misconfiguration: misconfigured security settings (e.g. insecure default settings). ˆ Kernel Flaws: security flaws in the core code implementing an operating system that can endanger the whole system. ˆ Buffer Overflows: inadequate checks on input sizes. This misbehavior can cause arbitrary code been executed with the privileges of the running program. ˆ Insufficient Input Validation: missing checks on the content of input strings, files, etc. Network applications without filters can run malicious commands (e.g. SQL injection). FP7-SEC-285477-CRISALIS 25

2. Standard ˆ Symbolic Links: as the operating system allows to change permissions related to files, a malicious user can trick the system by creating symbolic links and modifying permission passing through them. ˆ File Descriptor Attacks: malformed file descriptors can grant access to the related files. ˆ Race Conditions: an attacker can take advantage of high privileges processes by modifying their execution order and manipulate results. ˆ Incorrect File and Directory Permission: incorrect permissions can cause various lacks of information (e.g. reading and writing password files). The last step, called the Reporting, occurs simultaneously with the other three steps and records all the results. At the end of the test, pentesters report the identified vulnerabilities, provide a risk rating, and propose possible mitigation techniques for discovered weaknesses. Security Assessment Planning provides information on creating an assessment policy, scheduling tests, identify the most suitable approach, and addressing possible problems. It also enforces assessments to be compliant to current regulations and legislations. According to NIST SP800-115 this planning activity should identify: ˆ Organizational requirements assessments have to take into account ˆ Roles and responsibilities ˆ Adherence to already established methodologies ˆ Assessment frequency ˆ Documentation requirements In this phase organizations have to choose assessment techniques and customize them to fit the requirements. Organizations should also carefully consider risks related to use such techniques on their infrastructures. The selection of auditors and the identification of needed skills is another fundamental step of the planning. Assessors experience can determine the success or the failure of the evaluation. Organizations can choose the type of assessment (e.g. External/Internal and Overt/Covert) and finally propose technical tools and resources available to the assessors. At the end, the assessment plan should answer to the following basic questions: 26 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 ˆ What is the scope of the assessment? ˆ Who is authorized to conduct the assessment? ˆ What are the assessment s logistics? ˆ How should sensitive data be handled? ˆ What should occur in the event of an incident? Security Assessment Execution phase highlights key points for assessors to consider throughout the execution phase. Assessors have to be coordinated with various entities and stakeholders within the organization. Appropriate organization s personnel should help and supervise the assessment for the whole duration of the activity. During operations assessors can face different kind of problems (e.g. technical, operational, political, etc.). These can include: ˆ Resistance: to the assessments, from entities within the organization (e.g. network administrators or end users). ˆ Lack of Realism: that indicates the preemptive modification to system settings in order to make the infrastructure more secure to pass the tests. ˆ Immediate Mitigation: that describes the adjustment and the resolution onthe-fly of weakness found during the assessment. This can disturb following tests and compromise the final report. ˆ Time: time constraints are usually a problem as security assessments are often an exception more than a normal operation integrated in the development and deployment cycle. ˆ Resources: organization can make a stand on providing and maintaining adequate resources for security assessments. ˆ Evolving Technology: tools used for security assessments as well as used techniques can be out-of-date. ˆ Operational Impact: even if a security assessment should prevent any impact on infrastructure functioning, there is always a chance of accidental problems. For this reason, an incident response plans should be already in place during testing. FP7-SEC-285477-CRISALIS 27

2. Standard Assessors can make some preliminary analysis on the assessment during the assessment itself. This preliminary analysis of the causes of the noticed problems becomes the input for the last phase of the security assessment. Most common problem causes can be organized in the following sets: ˆ Insufficient patch management ˆ Insufficient threat management ˆ Lack of security baselines ˆ Poor integration of security into the system development life cycle ˆ Security architecture weaknesses ˆ Inadequate incident response procedures ˆ Inadequate training for end users and network/systems administrators ˆ Lack of security policies or policy enforcement Finally, the Post-Testing Activities concentrates on finalizing the analysis and proposing mitigation actions and recommendations. Security testing reports should be used by the organization as follows: ˆ As a reference point for corrective action ˆ In defining mitigation activities to address identified vulnerabilities ˆ As a benchmark for tracking an organization s progress in meeting security requirements ˆ To assess the implementation status of system security requirements ˆ To conduct cost/benefit analysis for improvements to system security ˆ To enhance other life cycle activities (e.g. risk assessment) ˆ To meet reporting requirements Overall, NIST SP800-115 also provides a well-structured and complete approach that goes into more technical detail than OSSTMM. From this perspective, it is interesting to discuss whether such details are necessary and helpful and make a methodology more applicable or whether they just inevitably restrict its use to certain use cases. 28 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF 2.3. ISSAF 2.3.1. General Information The Information Systems Security Assessment Framework (ISSAF) [12] is a peer reviewed structured framework designed by the Open Information Systems Security Group (OISSG). The methodology defined by ISSAF covers all the aspects related to security assessments: from an high-level perspective (e.g. business impact and organizational models) to practical techniques (e.g. security testing of passwords, systems, network, etc.). The framework is divided in four main phases structured in several working packages (named activities ). The four phases are respectively: Planning, Assessment, Treatment, and Accreditation. Figure 2.6 shows ISSAF workflow. Figure 2.6.: ISSAF framework overview (from [12]) The Planning phase concerns all the operation needed to define a project plan for the FP7-SEC-285477-CRISALIS 29

2. Standard whole assessment process. Before proceeding with the operational part, organizations have to motivate the assessment to substantiate the underlying concerns. Moreover, organizations have to prepare a business plan to cover costs and identifying available personnel and resources. The main steps in this phase are: ˆ Information Gathering ˆ Project Chartering ˆ Resource Identification ˆ Budgeting ˆ Cash Flow ˆ Work Breakdown Structure ˆ Project kick-off The Assessment phase defines the approach needed to evaluate the Information Security Risks within an enterprise. This phase focuses on the risk assessment process and addresses every component involved. It is divided into two categories: Inherent Risk Identification and Controls Assessments. The first consists of the following activities: ˆ Identification of Assessment entities: during this activity the organization spots the targets of the assessment (e.g. processes, assets, facilities, etc.). ˆ Identification of Threats and Vulnerabilities: this activity describes how to search, list and describe organization s vulnerability. ˆ Impact Assessment: this activity measures the impact of an exploited vulnerability to the business of the organization ˆ Likelihood Assessment: during this activity the organization evaluates the likelihood that a vulnerability is exploited. Control Assessment explores instead the following fields: ˆ Evaluation of Legal and Regulatory Compliance: this activity evaluates the position of the organization with respect to current regulations and legislations. 30 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF ˆ Evaluation of Enterprise Information Security Policy: this activity focuses on understanding current organization s approach on implementing and maintaining its security policies. ˆ Evaluation of Enterprise Information Security Organization and Management: this activity directly follows the previous one, as a review on how the Information Security is organized inside the company. ˆ Assessment of Enterprise Information Systems Security and Controls: this activity is in charge of reviewing different security aspects of the organization. For example: physical security, environmental security, technical controls (e.g. network security, host security, etc.), social engineering, etc. ˆ Evaluation of Enterprise Security Operations Management: during this activity the organization gains an understanding of the risk of specific operations processes. The operational areas involved in this operations are: Capacity Management, Vulnerability Management, User Management, etc. ˆ Evaluation of Enterprise Business Continuity Management: this activity evaluates organization capability of ensuring continuous availability of the Information Technology infrastructure. The output of the Assessment phase is reported within the Inherent Risk and the Residual Risk documents. The Treatment phase implements a platform for taking decision about the aforementioned risks. Its work-flow goes through the selection of safeguards, the development of implementation plan for countermeasures, and the development of a decision making process. In this phase the organization decides to accept, mitigate, or avoid vulnerabilities and associated risks identified in the previous phase. Finally, the process of Accreditation defines the steps needed to an organization to obtain the ISSAF certification. 2.3.2. Overview In this section we will review the Penetration Testing methodology proposed by IS- SAF leaving aside all practical details regarding specific network components assessing methods. ISSAF penetration testing methodology evaluates organization s network and systems. It consist of three phases and nine assessment steps. The three phases are: Planning and Preparation; Assessment; Reporting, Clean-up and Destroy Artifacts. FP7-SEC-285477-CRISALIS 31

2. Standard The Planning and Preparation phase comprises all the assessment s preliminary operations. These regards: requirements definition, actions to be performed, and expectations. ISSAF requires the identification of a contact person from the target company and those who will perform the tests. During the kick-off meeting both parties have to agree on the specific engagement team, the exact dates, times of the test, escalation path and other arrangements. The meeting has to be concluded with the signing of a formal Assessment Agreement that will provide mutual legal protection. The Assessment phase is the core of the penetration testing methodology. ISSAF approach is a path that leads the attacker to nine operation steps as shown in Figure 2.7. ˆ Information Gathering: ISSAF defines the information gathering in the broader sense possible. It comprises both technical and non-technical methods. The former looks for DNS records and exploit tools like WHOIS. The latter implements some social engineering methods (e.g. using search engines and mailing lists to find information about the target). The more a pentester knows about his victim the more he will improve his chances to successfully attack it. During this phase, the attacker does not always need to establish a contact with the target. Important information can be collected also from public sources (e.g. the Internet). As security assessments are generally limited in time and resources, information gathering is important to identify points that will be most likely vulnerable and focus on them. ˆ Network Mapping: this phase directly follows the previous one and refines the information gathering with a more technical approach. The target is to identify what devices are working in the network and outline its topology. Network mapping involves the use of fingerprinting and network analysis tools. More in details, ISSAF specifies different classes of actions to be performed: Find live hosts Port and service scanning Perimeter network mapping (e.g. router, firewalls) Identifying critical services Operating System fingerprinting Identifying routes using Management Information Base (MIB) Service fingerprinting 32 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF Figure 2.7.: ISSAF pentesting overview (from [12]) FP7-SEC-285477-CRISALIS 33

2. Standard At the end of this phase, pentesters will have the possibility to confirm or dismiss some preliminary made hypotheses regarding target systems. ˆ Vulnerability Identification: once specific targets are selected, the attackers will perform several activities to detect actual exploitable vulnerabilities in the system. ISSAF lists some of the activities used to detect weak points: Identify vulnerable services using service banners Perform vulnerability scans Perform false positive and false negative verification Enumerate discovered vulnerabilities Estimate probable impact Identify attack paths and scenarios for exploitation ˆ Penetration: this phase is the first actual operation against the target. The attacker will try to circumvent security measures and gain access to the system under attack. ISSAF methodology emphasizes the following steps within the penetration: Find proof of concept code/tools to use for the attack Develop new tools/scripts if needed Test proof of concept code/tools before use them to avoid problems during the actual pentesting Use proof of concept code against target Verify or disprove the existence of vulnerabilities Document findings ˆ Gaining Access & Privileges Escalation: in this phase pentesters will confirm the possibility to access the target. To do that the attacker has to compromise all the intermediate systems in the path leading to the physical system under attack (e.g. firewalls, routers, etc.). Moreover, the attacker will usually access the system with the least privileges possible and he will try to obtain administrator s rights exploiting local vulnerabilities with root-kits. ˆ Enumerating Further: this phase lists malicious activities the attacker can perform within the compromised system. Such activities are: Obtain encrypted passwords for off-line cracking Obtain passwords by using sniffing or other techniques 34 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF Sniff traffic and analyze it Gather cookies and use them to exploit sessions and for password attacks E-mail address gathering Identifying new routes and networks Mapping internal networks ˆ Compromise Remote Users/Sites: this phase focuses on the presence of secure communications (e.g. Virtual Private Networks) between remote users/sites and enterprises network. In this scenario, the attackers can try to extend his control to remote users and detached networks. In case of success, the attacker will repeat all the previous steps in the new environment. ˆ Maintaining Access: in this phase pentesters make sure to control the system also in the future. As a vulnerability can always be patched by system operators an attacker can install malicious code in the system to create covert channels and backdoors. Such verification shows the degree of exposure that a system can have once compromised. The Maintaining Access phase is often not performed as part of a penetration test due to the risks involved (e.g. a real attacker capable to take possession of the covert channel and gain access to the system). ˆ Covering Tracks: this phase ensures attackers to be invisible to later analysis. Usually, during a penetration testing, attackers should be as open as possible about their operations (unless the company expressly requests otherwise). There are several actions that pentesters can perform to hide their traces: Hide modified files and used tools Clear logs by checking history and edit log files Defeat integrity checking Defeat Anti-viruses Implement customized Root-kits ˆ (optional) Audit: system audits should be performed after completing a penetration test. This phase can usually bring to light further potential vulnerabilities within the system. The Reporting, Clean up and Destroy artifacts phase concludes the assessment. According to ISSAF a minimal reporting should consist of: FP7-SEC-285477-CRISALIS 35

2. Standard ˆ Verbal Reporting: that describes the feedback the company has to have immediately after the pentesting. The attackers will give to the company an overview on their finding and they will focus on major critical issues. ˆ Final Reporting: that is an in depth presentation with detailed results of the tests. Each discovered vulnerability has to be presented with related countermeasures and a proposal on how to implement them. The report should follow a well documented structure. ISSAF proposes a list of information needed for the document: Management Summary Scope of the project Tools that have been used Dates & times of the actual tests on the systems Outputs of tests performed A list of all identified vulnerabilities with included recommendations on how to solve problems found A list of action points Finally, all information that is created and/or stored during the test on target systems should be removed. If this is not possible, such information (e.g. files and directories) should be mentioned to system operators to proceed to their removal. This concludes the discussion of existing penetration testing methodologies. In a later version, we may extend this to other approaches like OWASP, PTES, or CHECK. 36 SEVENTH FRAMEWORK PROGRAMME

3. Requirements 3.1. Approach Within the CRISALIS project we merge the expertise of different stakeholders. Academia, SCADA vendors, and SCADA operators are three of the main contributors to our assessment methodology for critical infrastructure. However, to further enlarge the information basis that our approach is built upon, we decided to create a questionnaire about testing security infrastructure. The purposes of interviewing CI and security experts are several. There is the need to understand what are the requirements and the expected feedbacks. Moreover, we also need to validate the benefits of establishing a methodology tuned to CIs and also evaluate its comprehensiveness and accuracy. Using a questionnaire seemed the most efficient way to reach a broader community and inquire about the state of the art of CI testing in general and to identify possible steps forward toward the definition of the more comprehensive concept of CI vulnerability assessment. The questionnaire is then extended by more detailed interviews with some stakeholders. The questionnaire is divided into four sections: General information; Security requirements, design and regulations; Testing and methodologies; and Research. The General Information section focuses on characteristics of the interviewed persons. Information on area of expertise and experience will be used to evaluate the later responses and comments. The security requirements, design and regulations section collects information about deployed security mechanisms and authentication schemata. Moreover, the section investigates on what limitations there are in deploying security mechanisms in Critical Infrastructures. The testing and methodologies section is the core part of the questionnaire. It aims to identify currently used security practices focusing on: kind of tests performed, test frequency, and tools used. Furthermore, the section asks about the expectations the experts have from a CI security methodology. Finally, the Research section concludes the questionnaire asking about CI security research status. We believe that integration of CRISALIS expertise and questionnaire outputs will provide the necessary background from which to build a comprehensive security assessment 37

3. Requirements methodology for critical infrastructure. Such a methodology is supposed to take into account already existing best-practice provided by industry and different, but relevant, IT fields. The output has to be compliant with CI experts and operators needs and should allow them to have a common evaluation framework to compare their infrastructures. Section 3.2 presents the questionnaire we used to collect information. The questionnaire is also available online at Critical Infrastructure Security & Penetration Testing [13]. 3.2. Questionnaire Critical Infrastructure Security & Penetration Testing Industry/Researcher Questionnaire University of Twente December 2012 This questionnaire is anonymous unless you want to participate in the follow up mentioned in section E. The purpose of this work is exploring the need for a standard penetration testing methodology for Critical Infrastructure. With the following questions we will collect ideas and requirements useful to realize such methodology. N.B. If you have never worked with Critical Infrastructure please fulfill sections B and C with your opinion on how these systems are supposed to work and be tested. Questions marked with * are intended only for Critical Infrastructures operators/deployers. A. General information 1. In which area are you working? SCADA Vendor SCADA End User SCADA Consultant 38 SEVENTH FRAMEWORK PROGRAMME

A. General information Security Consultant Academia Other: Comments: 2. What is your domain of interest? (Multiple selections are possible) Water Energy Oil & Gas Transportation Other: Comments: 3. Briefly describe your field of work/research and your job. 4. How much experience do you have in working with Critical Infrastructures (CIs)? (little) (a lot) Comments: 5. How much experience do you have in security? (little) (a lot) Comments: FP7-SEC-285477-CRISALIS 39

3. Requirements 6. How important is computer security in your domain? (little) (a lot) Comments: B. Security requirements, design, and regulations 7. What types of security components do you use/manifacture/deploy? (Multiple selections are possible) Firewall Application White-listing Intrusion Detection System Intrusion Prevention System Forensic Analysis Tool Other: Comments: 8. Rank these limitations to security in order of importance from 1 (High importance) to 6 (Low importance). ˆ Costs are a big factor limiting the amount of research and spending on security. ˆ Time constraints. solutions. No time to investigate all potential security issues and ˆ Off-the-shelf security technology are not suitable for CIs and their proprietary software. ˆ The underlying hardware used in most CI cannot handle modern security features. ˆ Regulations/laws do not allow all security features. ˆ There is no incentive to secure every aspect of CIs. ˆ Other: Comments: 40 SEVENTH FRAMEWORK PROGRAMME

C. Testing and methodologies 9. * How important is the correct functioning of security components in your system? (little) (a lot) Comments: 10. * How is the access to the systems information and specification controlled? (Multiple selections are possible) Only those who have a need-to-know can access such information For some specific aspects a very small set of people has access to such information Specific area of the company terrain are only accessible by authorized personel Only the manufacters/suppliers have the full specification of the components Open source software is used Also internally, the security tests can mostly be done as black-box tests Other: Comments: C. Testing and methodologies 11. What kind of security tests are done? (Multiple selections are possible) Stress tests and Denial-of-Service tests Penetration tests Security code review Formale code verification Other: Comments: 12. How often are these tests performed? (Multiple selections are possible) At design time FP7-SEC-285477-CRISALIS 41

3. Requirements During development During the setup Regularly during operation When some changes occur Other: Comments: 13. Are security tests mainly black-box (system s specifications unknown) or white-box (full disclosure of information to pentesters)? Black-box White-Box Unknown Comments: 14. * Is it possible to test the system remotely (passing through public networks)? Yes No Unknown Comments: 15. * Is the security testing outsourced, i.e. done by externals? No, everything is tested in-house. No, generally everything is done in-house but in some cases external security testers are included. Yes, but only specific elements are tested by externals. Yes, the whole system is tested for security by externals. Unknown Other: Comments: 42 SEVENTH FRAMEWORK PROGRAMME

C. Testing and methodologies 16. Is the testing performed within company premises? Yes, always. If outsourced, it can be performed also outside company premises. Unknown Other: Comments: 17. For web servers security/penetration testing methodologies and tools exist. Are there any methods/procedures/standards used for testing CI security? No, I am not aware of any general procedures/standards. No, there are no standards but some methodologies specifically designed for each CI exist. Yes, internally a set of documented procedures exist which are followed for each CI. Yes, published/standardized methods are used for testing wherever possible. Unknown Other: Comments: 18. Can you mention some methods used? 19. What are the procedures for security testing CIs? How is the security tested? (Multiple selections are possible) A security test is done on the whole live system. FP7-SEC-285477-CRISALIS 43

3. Requirements A security test is done when the system is offline. A security test is done on a backup system. Different components are tested separately Unknown Other: Comments: 20. The security tests are done by... (Multiple selections are possible) The team working on the component(s). An internal security team. An external security team. Unknown Other: Comments: 21. Are there any CI-specific tools/software for the testing? (Multiple selections are possible) No, I am not aware of any CI-specific tool. Some simple tools are developed when necessary. Yes, existing software tools are used for testing specific components. Yes, we developed our own tools/software for testing the security of the ICS. Unknown Comments: 22. Referring to previous question, can you mention any of such tools/software? 44 SEVENTH FRAMEWORK PROGRAMME

C. Testing and methodologies 23. Is there a need for a (standard) CI penetration testing methodology? No. There is no incentive to do thorough security testing. A little. The methodologies/documented procedures will not be used. We do a flexible security testing strategy that is decided upon for each individual case. Yes. General guideline methodologies are good, but there is no need for more specific methodologies. Yes. A CI-specific and comprehensive methodology would make security testing more organized/ repeatable/ comparable. Unknown Comments: 24. This CI penetration testing methodology should... (Multiple selections are possible) include a guideline for testing the whole CI. include a guideline for testing specific components. include general guidelines about each testing aspect. include very detailed information about each testing aspect. suggest software/tools to be used. state how to document the results. Other: Comments: 25. Are there any areas/aspects/topics that need special attention? FP7-SEC-285477-CRISALIS 45

3. Requirements D. Research 26. Should more research be done in the area of CI security (either at university or at companies)? No, there is no incentive to do thorough security testing. No, there is already a lot being done (also in non-ci areas that can be applied to CI). Yes, more time should be spent in studying CIs security. Other: Comments: 27. Additional comments: E. Follow up 28. Would you agree to be contacted for a more in depth discussion? Yes No 29. If yes, please provide us your email address: Thanks for your time. In case you provided your email address, we will contact you to organize a 1 hour phone interview. 3.3. Further analyses At the moment of this writing (April 2013) the distribution and filling out of the questionnaire is still in progress. Due to the limited feedbacks we collected until now, it is 46 SEVENTH FRAMEWORK PROGRAMME

3.3. Further analyses difficult to generalize the results. However we are starting to see some trends and common believes in the answers. Interviewed people belong to Academia, SCADA vendors and utilities, consulting companies, etc. This is one of the main reasons why we propose a two step approach to D5.1 were we provide a preliminary proposal now that is later extended and validated based on further input. We still provide a preliminary summary of the found requirements in this section. As we expected, Critical Infrastructures and Security seem to be two inseparable concepts. All the participants to the survey agreed on considering computer security a priority in implementing and managing Critical Infrastructures. However there are some limitations in deploying security. Most of the interviewed experts identified costs as the biggest obstacle to properly ensure security. Costs are always a big factor limiting the amount of research and spending on security. Almost equally important as costs are time constraints. Several people complain about the limited time available to investigate all potential security issues and solutions. Ranked a bit lower, Off-the-Shelf security technologies in combination with unusual hardware used in SCADA seem issues easier to resolve. On one hand, Off-the-Shelf tools and techniques can be customized and used on CIs and their proprietary software. On the other hand, CI s underlying software, even outdated, can handle modern security features. Finally, participants agreed on the fact that laws and regulations already in place do not impede operators to deploy new security features to their infrastructures. It can be deduced that a penetration testing technique and, more in general, an assessment methodology for Critical Infrastructures has to focus on reducing cost and being time efficient. Standard methodologies have to be reduced to a minimum sequence of activities in order to save money and time. Automation of testing steps will play an important role. Penetration testers and auditors have to cut all disposable processes from their work and maximize the throughput. At the same time, external testers may have a clearer time and cost budget they can plan with. According to the results of our interviews there are no preferences in choosing a specific kind of security mechanisms with respect to the others. Firewalls, Application White-Listing, Intrusion Detection and Prevention Systems and also Forensic Analysis tools are widely deployed and used in Critical Infrastructures. So a penetration testing methodology should not focus on one of them but include them all. The same is true for the different kinds of security test. In the questionnaire we proposed four types of tests asking which of these are already implemented in Critical Infrastructures. Our experts confirm that Stress tests and Denial-of-Service tests, Penetration tests, Security code reviews, and Formal code verification are quite known and used in CI field. For this reason, a security assessment can freely exploit one or more of them depending on the purpose. FP7-SEC-285477-CRISALIS 47

3. Requirements All tests are so far mostly done during CI development and setup. According to our results, only a small percentage of the survey participants considers a live system suitable for a penetration test. However, situations in which the system is off-line for maintenance seem suitable to run specific tests. The reason behind these considerations relies on the concern that a test may affect the proper functioning of Critical Infrastructure and endanger both physical processes, people, and company business. Therefore, security assessments must not have an impact on live systems. Every action must be evaluated beforehand with respect to this risk. In any case, security operators have to be ready to face problematic or dangerous situations that can be caused by a penetration test. Questions about strategies and constraints on penetration testing reveal that there is no common agreement on the issue. First of all, Critical Infrastructures use both White- Box and Black-box penetration testing depending on the purpose of the test. Together, the two techniques contribute to perform a comprehensive assessment that focuses both on vulnerabilities and defense reaction capabilities. However, some experts exclude one or the other for different reasons. Time constraints and potential dangerous impacts are some of them. Remotely accessing the network is another big issue. Not all the operators are available to open their systems to Wide Area Networks (WAN) or to the Internet. On the other hand, some of them agree on passing through public networks to better simulate real attacks. Finally, some operators agree on outsourcing security testing and giving access to the system to third parties. These tests are usually performed within company premises. In any case, outsourced tests cannot substitute internal testing. However, almost all the interviewed persons agreed that an outsourced test is always specific for a component or a subsystem but never general and comprehensive of the whole Critical Infrastructure. The majority of respondents is not aware of any specific penetration testing methodology for Critical Infrastructure. Some experts suggested that (non-public) customized methodologies for specific tests exist but we can safely conclude that no comprehensive security assessment methodology has been developed and published yet. We know from the results that several Off-the-Shelf tools and software are used to test Critical Infrastructures. Some of the experts provide examples. Metasploit [14], Nessus [1], Achilles [15, 16], Mu Dynamics products [17] but also standard IT fuzzers and code checking tools are used in the CI context. In several cases, operators and vendors developed their own tools or customized standard one for specific purposes. At the end of the questionnaire we evaluated how a new assessment methodology specifically tuned on Critical Infrastructure would be received within the community. Almost all the interviewed agreed on the need for a standard CI penetration testing methodology. Such an opinion is however divided in two different connotations. On the 48 SEVENTH FRAMEWORK PROGRAMME

3.3. Further analyses one hand, half of the experts say that general guidelines methodologies are good, but there is no need for more specific methodologies. On the other hand, the others ask for a CI-specific and comprehensive methodology that would make security testing more organized/repeatable/comparable. Again, the requests about what a methodology should do are various and different among each other. Almost all the responses want the methodology to suggest specific software/tools to be used. The experts also emphasize the need for a comprehensive evaluation capable to provide general guidelines about each testing aspect. Moreover, it is important to define how to document testing results in order to have a common background useful for comparisons. Assessing specific components with respect to a general evaluation of the infrastructure seems a priority. This is probably due to the difficulty of analyzing results related to large and, sometimes, geographically distributed infrastructures. It is worth noting that almost no one asks for very detailed information about each testing aspect. This is related to the difficulties that a detailed methodology will have to face with respect to already in place policies and procedures inside specific Critical Infrastructures. Thanks to the questionnaire and interview feedback, we have a first insight on what operators and vendors want from a CI assessment methodology. The sometimes varying answers reflects the real world situation where there is no one-size-fits-all solution. From system to system and from company to company specific needs lead to the development and customization several different testing methods. The idea behind this deliverable is to collect such different information and create a general methodology based on common elements among already existing methods. At the same time, the new CRISALIS penetration testing methodology needs to be flexible enough to be adaptable to different scenarios and environments. FP7-SEC-285477-CRISALIS 49

4. Methodology 4.1. Overview In this section we focus on formalizing and merging all the concepts described in the previous two sections. The output of this analysis will be a new methodology for a security assessment methodology for Critical Infrastructures. We need to note that the current proposal should only be seen as a draft and that the concluded survey together with experiences with applying our approaches to testbeds available in CRISALIS will lead to a refinement and extension of this methodology in the second version of this document. Our work has two different starting points. On one hand we have the inputs provided by the questionnaire and the interviews. This is a necessary contribution for different reasons. First of all, experts in the field (e.g. SCADA vendors, utilities, security consultants, etc. ) have provided their expertise that we are building on. Thanks to their knowledge we were able to identify problems and concerns linked to the application domain and constraints in an industrial environment. Moreover, they allow us tailor or methodology to their actual needs. The interviewees explained how assessments and penetration tests are performed today in CIs and what they consider is still missing in these evaluations. Our CRISALIS penetration testing methodology addresses these issues and provides a generalized approach capable of taking the best of specific assessment methods already used, unifying differences and overcoming limitations. On the other hand, we have examined a number of relevant security assessment methodologies adopted from standard IT. OSSTMM, NIST SP800-115 and ISSAF are well known and extensively used. However, we see some shortcomings. E.g., several of the steps may not always be needed in the Critical Infrastructure field. Our idea of a security assessment methodology for Critical Infrastructure is a process that adopts the overall structure, the terminology, and certain individual elements or steps from these standard methodologies. However, at the same time, it leaves aside details regarding IT elements that are of secondary importance (or are completely missing) in the industrial field. Even if Industrial Control Systems and standard IT networks are merging more and more, assessing Critical Infrastructure must include also verifying specific components and properties of such heterogeneous systems such as PLC. The scope can therefore be 50

4.2. Workflow narrowed but some additional steps must also be included. Our work maps the needs identified by the experts on standard security methodologies. We do not want to create a new standard from scratch but rethink the existing ones. The result will be a lightwheight security assessment methodology involving a penetration testing methodology tuned to industry components and constraints. The use of a common schema to evaluate Critical Infrastructure will allow operators, vendors and security consultants to share information and assessments in an easier way and will make results more meaningful. A uniform approach will allow to easily spot major problems and threats inside Critical Infrastructure and will eventually contribute to a general improvement of the security of such systems. 4.2. Workflow 4.2.1. Main Workflow As introduced in the previous paragraph, we conceptually split our methodology in two major parts: ˆ General Assessment: it covers all the aspects of a security assessment. It involves three sub-phases: a Pre-Assessment in which a company identifies the goals of the assessment and solves any bureaucratic and legal issue regarding the following operations; the actual Practical Assessment in which auditors and pentesters perform practical tests on the assets; and the Post-Assessment phase in which the company evaluates the results and decides about countermeasures and information disclosure. ˆ Practical Assessment: it represents the intermediate part of the General Assessment. It focuses on practical tests performed on Critical Infrastructures. The Practical Assessment is also divided in three different sub-phases: the Preparation in which pentesters discuss technical details with system operators and prepare tools and strategies to be used during the test; the Testing where all the tests are performed; and the Analysis in which the results of the tests are documented and detailed. Figure 4.1 shows the methodology workflow. There are two different contact points between the two methodology s parts. The former links the Pre-Assessment phase to the Preparation one while the latter links the Analysis phase to the Post-Assessment one. In both the cases there is a substantial FP7-SEC-285477-CRISALIS 51

4. Methodology Figure 4.1.: Critical Infrastructure Methodology movement of information flowing from a phase to the other. We will denote these two activities as Information Flows one ( 1 ) and two ( 2 ). Finally, there are two possible loops in the workflow. The first feedback is given by the Analysis phase. At the end of the practical assessment it is possible to restart it without pass to the Post-Assessment. This situation describes a case in which the requirements identified in the General Assessment do not match with the actual testing. The second loop concentrates on the Testing phase and will be described in 4.2.3. Before discussing each working phase in details, it is worth noting that the reason to separate the methodology in two different parts is twofold. The first reason concerns the usability of the methodology. Such schemes allows operators to perform one of the two parts independently. Let us imagine a situation in which the procedures described for the Practical Assessment are not compliant with company policies or they are simply not suitable for company s purposes. In this case, operators can exploit the General Assessment scheme to perform all the accessory operations and substitute the Practical Assessment with a different model in line with their expectations. Thanks to the proposed methodology they will have just to take care about the Information Flows and modify them properly. In the same way, operators can use just the Practical Assessment with the condition to provide the right input and to modify the output by taking into 52 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow account requests provided by a different security assessment. The second reason behind the proposed scheme regards the maintainability. Critical Infrastructures have changed a lot since the 70s and they are likely going to change even more in the future. Moreover, legislations and company policies also change over time. The possibility to substitute a part of the methodology without altering the rest is a definite advantage in terms of cost and time. In case of such modifications, operators will have to modify the Information Flows to link the new part to the rest of the methodology. 4.2.2. Workflow phases In what follows we describe the five phases and the two Information Flows more in details: ˆ Pre-Assessment: the Pre-Assessment is the starting point of our security evaluation methodology. In this phase the company opens the discussion about the security assessment and explains its motivations and its goals. This phase should involve at least three different actors: the Chief Security Officer (CSO) or whoever is representative for this function and may decide to approve the starting of assessment operations; the Chief Financial Officer (CFO) or whoever is representative for this function and may decide on the budget allocated for the assessment operations and about acceptable risks; and ICS and SCADA operators that will be responsible to contribute to the discussion with the technical point of view. The discussion deals with several topics. First of all, the actors have to decide about security assessment s purposes. The main goal and the motivations behind the following operations have to be stated clearly. What follows is the identification of company assets. The actors have to focus on one or more targets according to the proposed objectives. Critical Infrastructures are complex systems and, for this reason, it is unlikely to have always a comprehensive assessment. Most of the time, specific sections or processes of the Critical Infrastructure will be checked and analyzed. An additional and very important step is a detailed risk analysis that clearly identifies activities endangering the safety of the system and its environment. This needs to be documented as well as an agreement to either skip those steps or perform them with specific safety guards. During the Pre-Assessment, involved actors have to agree on the time plan and decide on how to distribute resources to the five phases of the methodology. Finally they have to investigate and ensure that legal and contract obligations regarding assessment operations are met. This last step overlaps with the Control Assessment proposed by ISSAF and already discussed in 2.3.1. FP7-SEC-285477-CRISALIS 53

4. Methodology ˆ Information Flow 1 : at the end of the Pre-Assessment phase its participants have to prepare a document summarizing all the results of the discussion. The Information Flow describes the work the operators will do for translating such outcomes in practical directives for the pentesters. For example, the assets identified in the previous phase will become systems and components to be tested. In the same way, the main goal will be divided in several sub-goals which will be mapped into specific tests. Such information will be the input of the Preparation phase. ˆ Preparation: the Preparation is the first phase of the Practical Assessment. This phase involves two actors: ICS and SCADA operators that report the outcomes of the Pre-Assessment phase; and the pentesters which are in charge of performing necessary tests. The two actors can can be from the same organization if the purposes or the constraints of the security assessment respectively allow or do not permit the presence of third parties (e.g. external companies, trained pentesters, etc.). Operators and pentesters have to discuss several issues. First of all, they will decide on what kind of pentesting schema to follow. They will decide on the type of penetration testing as proposed by OSSTMM in 2.1.2 (e.g. Blind testing, Grey testing, etc.). Moreover, they will choose on the timing of the attack (e.g. working hours, weekends, etc.). Finally, they will decide whether to attack the live system, or a backup, or performing all the tests when the system is off-line. The attack impact is one of the main concerns that has to be faced during the Preparation. Critical Infrastructures cannot be put in danger by a penetration testing even if the reason to perform the analysis is verifying that possibility. During this phase operators should decide and implement specific safeguards to put in place during tests and prepare an emergency plan with countermeasure to take in case one of the security tests succeed in penetrating and corrupting the system. This plan has to be ready also in case of defensive black box testing. In that situation operators participating to the test will not be aware of the plan unless a problem occurs. In this phase operators have to verify if all the objectives defined during the Pre-Assessment are achievable. Otherwise they should motivate any problem that represents an obstacle to the test. The final step of this phase is the setup of the environment and of all the resources needed to the pentesting activity. ˆ Testing: the Testing phase implements the security tests that have been discussed during the Preparation. Its duration depends on different factors such as: accuracy of the tests (e.g. comprehensive tests vs. vulnerability identification), pentester skills, allocated budget, etc. During the entire phase pentesters have to log all 54 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow the performed operations and related discoveries. Such report will be the input of the Analysis phase. Security operators responsible for the emergency plan will follow the operations from a different point of view. They will check the presence of penetration side effects on the Critical Infrastructure that the pentesters are not aware off. Such information will be integrated to the pentest log at the end of the Testing phase. It is worth noting that the testing plan arranged in the Preparation phase can change accordingly to the evolution of the tests. Unexpected obstacles can interfere with the penetration test and undermine the feasibility of some operations. Such situations have to be also detailed in the log. In case a test shows severe vulnerabilities that can endanger the Critical Infrastructure, the safeguards and emergency plan should be triggered and prevent real damage. The effect of such an event is the interruption of the methodology workflow and the immediate feedback to the Chief Security Officer. The level of severity needed to trigger the emergency plan has to be identified in the Preparation phase. The testing phase will be extensively discussed in section 4.2.3. ˆ Analysis: the Analysis phase concludes the Practical Assessment. As in the Preparation phase, pentesters and operators are the two actors involved. The former will report about the findings of the Testing phase and will propose a risk rating of the security to the Critical Infrastructure. The latter will critically review the testing, contextualizing the problems and and justifying any design or maintenance choices. Moreover, operators have to perform two important tasks: check if the status of the system has returned to normal after the testing session; and apply fixes to minor and easy-to-fix problems and vulnerabilities. At the end of the Testing phase, the pentesters will have to undo permanent modifications to the system, like removing any file or software installed inside the Critical Infrastructure or unpluging any machine used for pentesting. During the Analysis, operators will check the correctness of such activity and will run some integrity checks to verify that the Infrastructure is brought back to a safe state. It is probable that problems such as misconfigurations and other simple activities from the penetration induce some errors into the system. Operators should fix such issues without reporting them to the next stage of the methodology. In this sense, the Analysis phase is also a filter identify the more severe security problems and what types of problems a later security deployment may create. ˆ Information Flow 2 : at the end of the Analysis phase operators and pentesters will prepare a document that, starting from the report of the Testing phase, summarizes findings and security issues. The format of the documentation can be based FP7-SEC-285477-CRISALIS 55

4. Methodology on the the ones proposed by ISSAF in its Final Reporting 2.3.2 and partially also on the final requirements of the OSSTMM 2.1.2. Some information needed to the document describing the Analysis phase are: Date and time of test Duration of test Auditors and analyst involved Tools that have been used Test type Scope of test Comprehensive list of performed tests Knowledge of which tests have been completed, not completed, or only partially completed, and to what extent Test error margins Vulnerability discovered/exploited with recommendations on how to solve problems found Unknown anomalies Operators will have to motivate and justify such documentation during the Post- Assessment phase. ˆ Post-Assessment: the Post-Assessment closes our security assessment methodology. With respect to the Pre-Assessment phase the presence of the Chief Financial Officer is no longer needed as any decision regarding new operations will be taken afterwards. Therefore, in this phase, the discussion involves ICS and SCADA operators and the Chief Security Officer or whoever represents this role in the company. Operators will explain the steps performed during the Practical Assessment. Pentester can also participate to the meeting. Their task will be to present their findings focusing on major problems encountered. Operators will also propose countermeasures agreed on the basis of the discussion in the Analysis phase with the pentesters. The CSO will evaluate the proposed countermeasures deciding on what actions to take to solve the most pressing issues. It is worth noting that, during the Post-Assessment phase, participants will also discuss the possible impact of vulnerability disclosures. The output of this phase is the final result of the security assessment methodology. Such documentation is both an high-level overview about company s vulnerabilities, possible mitigations, limitations in deploying security and relation to security legislation or policies on the 56 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow one hand and on the other hand a low-level overview describing specific problems and vulnerabilities. The conclusions of the assessment match the list proposed by the NIST in 2.2.2 (Insufficient patch management, Lack of security baselines, Inadequate incident response procedures, etc.). The output of the Post-Assessment methodology can be used for following security assessments (e.g. to identify solved and still in place problems and vulnerabilities) or as a starting point for different kinds of assessment (e.g. risk assessment introduced in CRISALIS deliverable 2.2). Figure 4.2 shows a summary of the concepts discussed above. We will now discuss the penetration testing in more detail. Figure 4.2.: Critical Infrastructure Methodology Detailed Schema 4.2.3. Penetration Testing Steps The penetration testing process is the core of the assessment methodology. During the Testing phase, pentesters go through five operational steps: Information Gathering, Network Mapping, Vulnerability Identification, Penetration, and Maintaining Control. FP7-SEC-285477-CRISALIS 57

4. Methodology Each step groups a set of operations necessary for the next step. We also envision a circular representation of the testing steps as the penetration testing process is not necessarily one-time. A successful attack and the consequent control of a part of the system by the pentesters could open new attack vectors. In this case, pentesters can traverse again through the five pentesting steps exploiting new targets or attack vectors. The process ends when all the prepared attack scenarios as well as those discovered during the activity have been tried and evaluated. The specifications of the five penetration testing steps are based on the schema proposed by ISSAF in 2.3.2 as we consider this the most appropriate and comprehensive approach as far as the actual penetration testing is concerned. In what follows we describe each step in detail: ˆ Information Gathering: the Information Gathering step focuses on systems for collecting and analyzing information regarding the Critical Infrastructure. This information concerns several elements such as working personnel, company policies, used hardware/software, etc.. The Information Gathering involves both technical and non-technical methods. Depending on the type of testing (Black Box vs. White Box), pentesters can have some initial information at their disposal that helps them to understand the environment they are going to attack. It is unlikely to find useful information about the Critical Infrastructure internal functioning on the Internet. However, the knowledge of the working staff and their skills and backgrounds can give some ideas on what pentesters will have to face once the attack has started. In case of a White Box penetration testing the Information Gathering is important to quickly identify points that will be most likely vulnerable and focus on them. ˆ Network Mapping: the Network Mapping step directly follows the Information Gathering and contributes to acquire more information about the Critical Infrastructure. In this case, pentesters concentrate only on hardware and software. The main target is to identify what devices are working inside the Critical Infrastructure, identify communication patterns the network topology. Different tools can be used during this activities (e.g., sniffers, fingerprinters, etc.). Approaches to fingerprint devices and map networks are investigated in WP4 of CRISALIS and are presented in deliverables 4.1 and 4.2.. Such tools can be highly useful to automate some work in this step and thus save time and costs. During this step pentesters will perform the following operations: Topology discovery and finding live hosts 58 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow Differentiate from network components (e.g. switch, firewalls, etc. ), ICS devices (PLC, RTU, etc.) and standard computers (servers, laptops, etc.). Port and Service scanning Operating System and Services Fingerprinting Critical Services Identification The Network Mapping step will allow pentesters to confirm or discard attack hypotheses formulated during the Preparation phase or the Information Gathering step. ˆ Vulnerability Identification: once one or more targets are selected, pentesters will perform a vulnerability scanning. The most commonly used tools for vulnerability identification in Critical Infrastructure are Achilles [15] [16] and Nessus [1]. The list of the activities of this step comprises but is not limited to: Perform vulnerability scans Enumerate discovered vulnerabilities Perform vulnerabilities verification (e.g. trying to eliminate false-positive) Estimate possible impact of exploiting found vulnerabilities and verify that attack can be performed safely Identify suitable attack vectors for exploitation ˆ Penetration: during the Penetration step, pentesters try to gain access and obtain control of Critical Infrastructure s components. This is the most dangerous operation of the Testing phase as the attackers try to subvert normal operations of the system or a part of it. The Metasploit Framework [14] is one of the most widely adopted tool to exploit software vulnerabilities. In its professional version, the framework also provides several exploits against Programmable Logic Controllers (PLC). However, most of the times, pentesters will resort to develop customized scripts and tools to exploit ICS-specific vulnerabilities. This is the case of the fuzzers developed within the CRISALIS project and presented in deliverable 5.2. What follows is a list of operations performed by pentesters during the Penetration step: Find or develop suitable code/scripts/tools to use for the attack Test developed code/scripts/tools before using them on the Critical Infrastructure (to avoid problem during pentesting and unwanted effect on the whole system) FP7-SEC-285477-CRISALIS 59

4. Methodology Use code/scripts/tools against the Critical Infrastructure Verify or disprove the existence of vulnerabilities ˆ Maintaining Control: once in control of a machine, attackers will verify what information can be retrieved from the machine and if they can use the compromised computer to find new targets on the network. Pentesters will search for valuable data like: Passwords Running processes Information regarding the physical process under control Generic contacts (e.g. emails) If the compromised machine is at the edge of the network it may also link to different networks. Pentesters have to evaluate this possibility and, if this is the case, restart the pentesting process against the new target. Attackers have always to verify the possibility to maintain access to the machine for future activities by installing malicious code (e.g., root-kits and backdoors). This guarantees to maintain control even if the exploited vulnerability is patched. Whether this should be conducted needs to be decided in the planing phase. Finally, pentesters have to cover tracks of the intrusion. Such operations are usually performed: Hiding modified files and tools Cleaning logs Restoring integrity checks At the end of this step the Testing restarts on a new target or ends up letting pentesters proceed with the Analysis phase. Figure 4.3 proposes a comprehensive schema of the assessment methodology plus the chain of the five penetration testing steps. 4.3. Final Remarks The presented CRISALIS penetration testing approach represent a first draft towards a security assessment methodology for Critical Infrastructures. To the best of our knowledge, this work is the first attempt to formalize Critical Infrastructure assessments and pentestings. With our proposal we tried to incorporate requests and advices given by Critical Infrastructure experts with existing standard methodologies. The result is a 60 SEVENTH FRAMEWORK PROGRAMME

4.3. Final Remarks Figure 4.3.: Critical Infrastructure Methodology plus Penetration Testing Steps FP7-SEC-285477-CRISALIS 61

4. Methodology light-weight but comprehensive list of working phases linked by specific information and documentation exchanges that should lead pentesters through a fast, cost-saving and effective evaluation of a system. Our methodology differentiates among two logical parts. The first, called General Assessment, concentrates on the administrative process that a company must follow to organize a security assessment. The second, called Practical Assessment, regards the technical preparation of the evaluation process and the consequent analysis. Finally, the process shown in 4.2.3 describes the actual penetration testing. In this phase we discussed the steps pentesters have to follow to check and evaluate a Critical Infrastructure. As described in the introduction of this chapter, the use of separated but connected logical components improve both usability and maintainability of the methodology. The proposed methodology is not refined in arbitrary level of detail and yet needs to be tested in a Critical Infrastructure. This will be done once the survey and interview results are fully available as this will also provide a detailed understanding of the level of detail that is actually required. In the meantime, we have conducted some initial penetration testing experiments in one of the CRISALIS testbeds where one new vulnerability was found. These tests also helped to shape and refine this methodology as we were able to identify importance and feasibility of different steps. It also highlighted that there are insufficient structures to report newly found vulnerabilities of incidents to manufacturers and users. This should be addressed in a second version of this deliverable. 62 SEVENTH FRAMEWORK PROGRAMME

5. Conclusion In this deliverable, we have started with motivating the need and benefits for a methodology for security penetration testing for Critical Infrastructures and Industrial Control Systems. Benefits include time and cost savings for penetration testers, easier comparison of results, and better confidence in completeness and correctness of results. We analyzed a series of existing methodologies regarding their applicability in the CRISALIS context. Based also on the preliminary feedback from a survey and interviews we have conducted, we identified requirements and the need to provide an adapted and revised approach. While we make a first proposal for such an approach already in this version of the deliverable, we suggest to provide a second version of the document by the end of Y2, as this will allow us to finish the survey, use this to provide an updated and refined methodology, and finally also include results from own penetration testing experiments in our testbeds. 63

Bibliography [1] R. Deraison, H. Meer, and C. Van Der Walt. Nessus network auditing. Syngress Media Incorporated, 2004. [2] Wikipedia. White-box testing Wikipedia, the free encyclopedia, 2004. [Online; accessed 14-January-2013]. [3] R. Patton. Software testing. Sams, 2005. [4] Wikipedia. Black-box testing Wikipedia, the free encyclopedia, 2003. [Online; accessed 14-January-2013]. [5] Wikipedia. Gray-box testing Wikipedia, the free encyclopedia, 2006. [Online; accessed 14-January-2013]. [6] M. Prandini and M. Ramilli. Towards a practical and effective security testing methodology. In Computers and Communications (ISCC), 2010 IEEE Symposium on, pages 320 325. IEEE, 2010. [7] D.P. Duggan, M. Berg, J. Dillinger, and J. Stamp. Penetration testing of industrial control systems. Sandia National Laboratories, 2005. [8] P Herzog. Osstmm 3 the open source security testing methodology manual. barcelona, españa: Isecom, 2010. [9] Karen Scarfone, Murugiah Souppaya, Amanda Cody, and Angela Orebaugh. Nist special publication 800-115: Technical guide to information security testing and assessment, 2008. [10] Creighton T Hager and Scott F MidKiff. An analysis of bluetooth security vulnerabilities. In Wireless Communications and Networking, 2003. WCNC 2003. 2003 IEEE, volume 3, pages 1825 1831. IEEE, 2003. [11] Creighton T Hager and Scott F Midkiff. Demonstrating vulnerabilities in bluetooth security. In Global Telecommunications Conference, 2003. GLOBECOM 03. IEEE, volume 3, pages 1420 1424. IEEE, 2003. 64

Bibliography [12] Balwant Rathore, Mark Brunner, Miguel Dilaj, Omar Herrera, Piero Brunati, Rama K Subramaniam, Subash Raman, and Umesh Chavan. Issaf 0.2.1 - information systems security assessment framework, 2006. [13] Marco Caselli and Frank Kargl. Critical infrastructure security & penetration testing questionnaire. https://docs.google.com/spreadsheet/viewform?formkey= dg15cuv5ck9hwlg1etvuvk51znnyb1e6mq/, 2012. [14] LLC Metasploit. The metasploit framework, 2007. [15] Achilles test platform. http://www.wurldtech.com/product_services/ discover_analyze/achilles_test_platform/. [16] Achilles test software. http://www.wurldtech.com/product_services/ discover_analyze/achilles_test_software/. [17] Mu dynamics software. http://www.mudynamics.com/products/. FP7-SEC-285477-CRISALIS 65