D5.1 Security Testing Methodology

SEVENTH FRAMEWORK PROGRAMME Theme SEC-2011.2.5-1 (Cyber attacks against critical infrastructures) D5.1 Security Testing Methodology Contract No. FP7-SEC-285477-CRISALIS Workpackage WP 5 - Vulnerability Discovery Author Marco Caselli, Frank Kargl, Tobias Limmer Version 2.0 Date of delivery M24 Actual Date of Delivery M24 Dissemination level Public Responsible UT The research leading to these results has received funding from the European Community s Seventh Framework Programme (FP7/2007-2013) under grant agreement n 285477.

SEVENTH FRAMEWORK PROGRAMME Theme SEC-2011.2.5-1 (Cyber attacks against critical infrastructures) The CRISALIS Consortium consists of: Symantec Ltd. Project coordinator Ireland Alliander Netherlands Chalmers University Sweden ENEL Ingegneria e Innovazione Italy EURECOM France Security Matters BV Netherlands Siemens AG Germany Universiteit Twente Netherlands Contact information: Dr. Matthew Elder Symantec Limited Ballycoolin Business Park (GA11-35) Blanchardstown Dublin 15 Ireland e-mail: matthew_elder@symantec.com

Contents 1. Introduction 6 1.1. General Classification of Penetration Tests.................. 8 1.2. Standardization................................. 9 1.3. Testing Critical Infrastructures........................ 10 2. Standard 12 2.1. OSSTMM.................................... 12 2.1.1. General information.......................... 12 2.1.2. Overview................................ 14 2.2. NIST SP800-115................................ 19 2.2.1. General information.......................... 19 2.2.2. Overview................................ 22 2.3. ISSAF...................................... 29 2.3.1. General Information.......................... 29 2.3.2. Overview................................ 31 2.4. NESCOR Guide to Penetration Testing for Electric Utilities........ 36 2.4.1. General Information.......................... 36 2.4.2. Overview................................ 38 3. Requirements 51 3.1. Approach.................................... 51 3.2. Questionnaire.................................. 52 A. General information.............................. 53 B. Security requirements, design, and regulations................ 54 C. Testing and methodologies........................... 55 D. Research..................................... 60 E. Follow up.................................... 60 3.3. Analysis..................................... 61 3.3.1. Security Deployment Limitations................... 61 3.3.2. Security solutions............................ 62 3.3.3. Security tests.............................. 62 3

3.3.4. Pentesting Strategies.......................... 63 3.3.5. Penetration Testing Methodologies for Critical Infrastructures... 64 3.3.6. Final Remarks............................. 65 4. CRISALIS CI Security Testing Methodology 67 4.1. Overview.................................... 67 4.2. Workflow.................................... 68 4.2.1. Main Workflow............................. 68 4.2.2. Workflow phases............................ 70 4.2.3. Penetration Testing Steps....................... 74 4.3. Final Remarks................................. 79 5. Tools 82 5.1. Kali Linux.................................... 82 5.2. Metasploit Framework............................. 82 5.3. Nessus Vulnerability Scanner......................... 84 5.4. Achilles Test Platform............................. 85 5.5. SamuraiSTFU.................................. 85 5.6. SCADA Strangelove Tools........................... 86 5.7. Others Ressources............................... 87 6. Vulnerability Disclosure 88 7. Conclusion 90 4

Abstract Existing security testing methodologies and tools cannot be applied directly to critical infrastructures due to a number of safety and availability requirements. One of the goals of WP5 is to understand which assessment processes we can exploit within Industrial Control System (ICS) environments and to describe the constraints given by industrial devices and infrastructures. This document provides an overview of assessment methodologies used in standard IT. We analyzed these methodologies to investigate their applicability in an ICS environment. Moreover, we realized a survey among stakeholders in order to detail existing best practices adopted by companies to test their infrastructures and discover vulnerabilities in their systems. Due to the low number of responses we decided to use the survey as a guideline for in depth interviews with security experts and pentesters to better understand ICS related problematics and possible solutions. As result of these activities, we propose a new methodology tailored to support security testing in Critical Infrastructure environments.

1. Introduction This deliverable describes the security and penetration testing methodology developed by the CRISALIS project. It aims at providing a generic framework for security testers in critical infrastructure environments without being a fixed recipe to follow. Instead, it should provide an overview over important phases, discuss important considerations to keep in mind and provide an overview on available tools. The methodology is derived from surveying security testing methodologies in other areas than CI and expert knowledge from CI security experts collected through an online questionnaire and in-depth interviews. This report describes version two of the methodology that was created during the 2nd year of the project. Compared to version one, we have included a description of the NESCOR electric utility testing methodology. Likewise, we extended and completed our collection of input from CI security experts by conduction a series of additional interviews. Both led to an update of our methodology compared to version one. Next, we included two additional chapters: one that describes available tools for CI security testing and one that briefly discusses the issue of responsible disclosure once security vulnerabilities have been found. The term security testing encompasses a large set of techniques and methodologies that aim at verifying if an information system owns and maintains the level of security is intended to have. Security testing techniques include different kinds of activities: analysis-centered activities (e.g. document-based design review), software analyses (e.g. code review) and also practical tests on live systems with the purposes of find vulnerabilities, penetrate infrastructures or disrupt their functioning. These practical tests are commonly known under the name of penetration testing. A penetration test represents a simulated attack to an infrastructure, a specific device, or a network with the purpose to evaluate implemented security systems. The attack is simulated as it is conducted by the system owner or on his request. A penetration test has multiple phases and allows to highlight infrastructures and protocols weaknesses. During a penetration test, the pentesters try to discover or directly exploit known and unknown vulnerabilities in order to obtain useful information or to access targeted hosts. The purpose of a penetration test is always well defined. With respect to a general security assessment the penetration testing has not to be comprehensive but it has to achieve a specific goal (e.g. steal data, disrupt services, etc.) Both a security 6

test and a penetration test foresee a final report in which all the findings are listed and analyzed. While the penetration testing focuses on how the pentesters achieve their goal, the security testing presents a comprehensive report of the security problems with an assessment of their impact on the system. In both cases, the final analyses should propose technical or organizational mitigations to overcome observed problems. In its most simplistic form, a security test involves running some automated security scanner, such as Nessus [1]. However, it is generally agreed that comprehensive analyses typically involve activities such as manual test planing, preparation, and postexamination to be able to flexibly react to the system under test. In such a case, the security tests but especially penetration tests should follow some structured methodology to ensure valid, reproducible, and well documented results. The development and use of such manual penetration testing methodologies is motivated by several reasons. First of all, there are numerous vulnerabilities that may be difficult or impossible to detect with automated tools. Network or application vulnerability scanning software are nowadays highly sophisticated. However, sometimes, high profile systems and infrastructures need human thinking more than brute force to be compromised. Even if automated tests are implemented to be comprehensive there is a chance that the time needed to finish the assessment exceeds the time granted for the testing activity. In this case, a systematic approach dynamically adjusted on the basis of the responses of the infrastructure to the tests can achieve better results. Moreover, automated tests can miss specific kinds of vulnerabilities. This is the case related to the detection of critical effects result of a combination of low-risk vulnerabilities exploited in a specific sequence. As important as the identification of single vulnerabilities, the analysis on the feasibility of a particular attack vectors is another important target of security assessments and penetration tests. As it is impossible to have entirely secure systems in practice, companies and organizations are interested in running infrastructures that are practically secure. Therefore, the security assessments become a way to rank vulnerabilities both by severity and realizability. The potential business impact is a further valuable results of security testing. Conducting practical attacks allows to properly measuring necessary effort from the attackers and use it as an input for a correct risk estimation. Assessing the magnitude of economical and operational problems related to successful cyber-attacks is usually the last phase of analysis of a security assessments and penetration tests. Finally, companies use penetration tests to assess the ability of their network operators to successfully detect and respond to cyber-attacks. Their capacity of reaction is measured during the test and evaluated at the end of it. These kind of experiments can provide the necessary evidences to support increasing investments in security personnel FP7-SEC-285477-CRISALIS 7

1. Introduction and technology or, at least, suggests changes in the implemented methods and strategies. 1.1. General Classification of Penetration Tests Penetration tests can be performed in a broad variety of ways. The following criteria provide some orientation and classification. The amount of information known by the attackers (the pentesters) is one of the main discriminating factor of penetration tests. The simplest classification describes three different types of tests: white-box, black-box, and gray-box. ˆ White-box penetration testing outlines a situation in which fake attackers have full knowledge of the system under attack. The goal of a white-box penetration test is to simulate the presence of an expert malicious insider. In some special cases such insider has also basic credentials for the target system. Among the information known to the pentester we can have: network diagrams, domain names, IP addresses, but also phone numbers and e-mail addresses. White-box testing of applications usually allows intruders to have also programs source code. ˆ Black-box penetration testing is the opposite of the white-box penetration testing. This method examines infrastructures and applications functionalities without knowing anything about them. In general, the tester is aware of what the software is supposed to do but is not aware of how it does it [2]. Thus, specific inputs (e.g network packets, shell commands, etc.) return specific outputs without the intruder knowing how the mechanisms within the system work. The goal of a black-box penetration test is to simulate a typical external hacking or also cyber-warfare scenarios. ˆ Gray-box penetration testing is a combination of white-box and black-box methods. A gray-box tester partially knows the internal structure of the system under attack. This knowledge can include network topology, installed applications or also more in deep information like used algorithms and data structures. Since any combination of knowledge owned by the testers falls in the domain of gray-box penetration testing this method is the most widely used, especially in the field of web applications. Other ways to characterize penetration tests include the type of system access (external network, internal network, physical access) and the type of attacks (physical attacks vs social engineering attacks). Moreover, depending on the testing requirements, some tests 8 SEVENTH FRAMEWORK PROGRAMME

1.2. Standardization can simply identify vulnerabilities without exploiting them or conclude the attack in the more disruptive way possible. The aforementioned characteristics of penetration tests need to be carefully defined in the test plan. 1.2. Standardization In this last decade the interest in penetration testing techniques has increased. Companies, organizations and governments provide numerous services on the Internet and need to assess the security of their systems. Given the proliferation of different penetration testing methods, some organizations and standardization bodies have decided to propose comprehensive and usable methodologies. While many penetration testers follow a home-brewed self-defined testing approach, following standardized approaches has some general benefits. A standardized and generally accepted penetration methodology provides a certain confidence that no important steps or aspects of a test are forgotten by accident. As tests should be repeated after significant changes to the target system, following the same methodology also provides the possibility to compare multiple tests to identify whether a system got more secure after mitigation measures have been taken. A standardized approach may also be required in cases where security tests are required from a legal or insurance perspective. A standardized approach and common reporting forms also allows to compare results with the measured on other installations. Finally, accepted standards simplify negotiations between a security tester and clients about the scope of such tests. So there are a number of good reasons to look into such standardized and commonly agreed approaches. Over the years, many standardized methodologies have been proposed. From most general to system-specific ones, both governments and private organizations have proposed different ways to implement penetration tests. Among the most well-known penetration testing techniques are: OSSTMM (Open Source Security Testing Methodology Manual), NIST SP800-115 and ISSAF (Information Systems Security Assessment Framework).Later in this document, we are going to give a broad overview of some of these approaches. In addition to methodologies, there are a number of different standardized ways to assess specific contexts of security testing without a structured approach. Companies can decide not to follow a methodology but just perform a light assessment using different methods. In this cases they can exploit several guidelines and toolkits developed for numerous and different purposes. According to [3] toolkits implement and bundle FP7-SEC-285477-CRISALIS 9

1. Introduction in a convenient package a set of testing techniques, usually aimed at discovering specific classes of security problems. Guidelines instead organize the process of security testing, by collecting sets of best practices, comprehensively listing items to be tested, and structuring any other kind of useful advice. Finally, there are also collections of penetration testing tools such as Kali Linux or Metasploit that are regularly used by penetration testers. 1.3. Testing Critical Infrastructures Despite the recent interest for penetration testing methodologies, none of the proposed methodologies specifically considers industrial control systems (ICSs) and critical infrastructures (CIs) despite the fact that penetration testing is also performed in such environments on a routinely basis. To the best of our knowledge the only guide specifically tuned on ICS is the one developed within the US National Electric Sector Cybersecurity Organization Resource (NESCOR) tuned on penetration testing for electric utilities. We argue that standard methodologies should not be applied to critical infrastructures without special consideration. These systems and the networks that interconnect them show many differences compared with standard IT networks. Since critical infrastructures control and automate real world processes or equipments which are potentially dangerous for humans, any activity different from the main purpose including penetration testing should be limited. So just running a full Nessus or Metasploit test in such an environment could have disastrous consequences. There are three main concerns that make penetration testing on critical infrastructures challenging. As introduced in the previous paragraph, any penetration attempt can have a consequence in the real world. What follows is an interesting example of such concerns. In a real incident, a gas utility conducting a penetration test on its corporate network performed this in a careless manner affecting some of the distributed control systems (DCS). During the test this DCS system was locked up by mistake and the utility was not able to send gas through its pipelines for four hours leading to a large economic loss [4]. The strong relation between the system under test and physical components that people depend on is the first important difference with respect to standard IT. This implies that any CI penetration testing methodology needs to include safeguards to protect from such failures. Another important difference concerns the devices used in Critical Infrastructure, these systems usually have very limited resources compared to devices found in normal IT environments. The longevity of network components is greater and the cycle of replacement of devices may also be in the order of decades. This implies that devices 10 SEVENTH FRAMEWORK PROGRAMME

1.3. Testing Critical Infrastructures may be out of service, no patches may be available, and generally proposing mitigation strategies may take different approaches. This also means that critical infrastructures can be more easily overloaded with respect to other IT systems. A penetration testing methodology has to take care not to override the normal operation of every component in the industrial control systems unless identifying Denial of Service (DoS) attacks is specifically in the scope of the test. Finally, due to the size of most of the critical infrastructures, a penetration testing has always to check if localized attacks have effects in different parts of the network which may be hard to assess due to indirect effects. At the end of a test the identification of a vulnerability is followed by the calculation of the damage that it can cause. Linking these two elements in a system with thousands of components requires the assistance of experienced personnel both in the IT field and in the field related to the industrial process under control. As a consequence, penetration tests cannot be conducted by security experts in isolation. We think that a penetration testing methodology for critical infrastructure must explicitly describe how to deal with the previous issues. Moreover, it has to identify specific methods, tests, and tools already used in standard penetration testing techniques and explain how to tune these in order to be suitable for industrial control systems. Finally, such an ICS penetration testing methodology must provide where necessary new instruments and customized methods to test IT and networking components not found in standard ICT (e.g., Process Control Units, Remote Terminal Units, field networks, etc.). FP7-SEC-285477-CRISALIS 11

2. Standard In this chapter, we are going to review multiple penetration testing methodologies proposed by standardization bodies and other organizations. The aim is to provide a broad overview over the state of the art before defining specific requirements for CI penetration testing and then proposing an own CRISALIS penetration testing methodology for Critical infrastructures that is distilled from those existing approaches. 2.1. OSSTMM 2.1.1. General information The Open Source Security Testing Methodology Manual (OSSTMM) [5] is an open standard methodology for security tests. Developed by Pete Herzog at the end of 2000 as an ethical hacking framework, it has rapidly grown to become a comprehensive methodology to assure security at operational level. Version 3, released in 2008, encompasses tests for every security aspect: from personnel qualification to physical security, from control of communication to electronic systems safety. As every standard methodology, it is designed to be consistent and repeatable. Moreover, it is openly available and thus allows a free dissemination and free use. Figure 2.1 shows a comparison among OSSTMM and common security tests in terms of accuracy and thoroughness. By its nature, the methodology can be adapted to any kind of audit operation (e.g. penetration tests, ethical hacking, security and vulnerability assessments, red/blue teaming, etc.). The primary purpose always remains the description of a security examination path and the definition of a way to consistently correlate test results. The usage of the OSSTMM allows the auditor to perform a validated set of tests and to qualify its examination as a certified OSSTMM audit. Differently from other methodologies, mitigations to security flaws are not required as part of an OSSTMM audit. Recommendations may well be part of the final results, but common operational situations often imply that auditors have a limited view and knowledge of the client environment. In these cases it is difficult to propose proper and feasible solutions. For this reason solutions are usually avoided in final reports but should we worked out together with the system owner after the test. This is an aspect 12

2.1. OSSTMM Figure 2.1.: OSSTMM comparison with common security tests (from [5]) that is well in line with our findings from the previous chapter as it will also apply to CI. An important aspect of the OSSTMM is the compliance with general security policies. The methodology is developed taking into account major legislation and regulations. Furthermore, OSSTMM defines three different types of compliance and a set of rules to deal with all of them. The three types of compliance defined in the methodology are: ˆ Legislation: enforces the security test to be compliant with region/country legislation. The lack of this requirement could lead to criminal charges. ˆ Regulation: enforces the security test to be performed accordingly to standard regulation within industry sector. A failure in this task could lead to a dismissal of the company from the group. ˆ Policy: enforces the security test to be done according to business and organization policies. Violation could cause a dismissal of the responsible from the company. OSSTMM provides a list of national and international regulations already proved to be compliant to the proposed methodology. FP7-SEC-285477-CRISALIS 13

2. Standard 2.1.2. Overview Before describing the workflows, the OSSTMM introduces several concepts to outline what kind of security tests can be performed and what the scope of such tests can be. Figure 2.2 presents the six different security test types defined by OSSTMM. Possible security tests are however not limited to these cases and can be tuned to more specific requirements. Figure 2.2.: OSSTMM security test types map (from [5]) ˆ Blind: this type of security test describes a situation in which the attacker does not know anything about the target. On the other side, the defender knows everything about the audit operations and it is prepared to take countermeasures. A blind security test is primarily used to evaluate attacker skills. ˆ Double Blind: as before the attacker does not know anything about its target but, this time, the target is not notified in advance about the security test. A double blind audit is used to test both the skills of the attacker and the preparedness of the defender. ˆ Gray Box: in a gray box security test the attacker has limited knowledge of the target and can better engage the defender. The defender still has the advantage of full knowledge about the audit in progress. As in the blind security test, this case is mostly used to evaluate attacker s skills. 14 SEVENTH FRAMEWORK PROGRAMME

2.1. OSSTMM ˆ Double Gray Box: in this type of security test both the attacker and the defender have limited knowledge about their counterparts. Such situation is used to evaluate both the contenders. With respect to double blind this test increases the level of accuracy and pervasiveness. ˆ Tandem: in the Tandem security test the attacker has full knowledge of the target and the defender is fully prepared to bear the test. Such situation is used to improve the thoroughness of the audit as the attacker has a full view of all tests and their responses. ˆ Reversal: as in the previous situation, the attackers knows everything about the target while the defender knows nothing about its contender. The purpose of this test is to evaluate every operational aspect of the defender. Once the test type of interest is identified, it is necessary to identify the scope of the audit. OSSTMM divides such scope into three different channels: COMSEC (communication security), PHYSSEC (physical security), SPECSEC (spectrum security). A thorough security audit would require testing all three channels but usually the tests are tuned on more specific needs. For this reason, the methodology makes a further partitioning, addressing these three channels into five logical sections. Channel PHYSSEC is divided into Human and Physical, channel SPECSEC contains Wireless Communication, and channel SPECSEC is divided into Data Networks and Telecommunications. What follows is a description of the five logical sections: ˆ Human: concentrates on the human element and its communication processes. ˆ Physical: focuses on physical security testing and comprises tangible elements of security where interaction requires physical effort or an energy transmitter to manipulate. ˆ Wireless Communications: concentrates on security tests on wireless electronic communications and signals. ˆ Data Networks: comprises tests on electronic systems and data networks. ˆ Telecommunications: focuses on digital/analog telecommunications where interaction takes place over established telephone network lines. The full methodology is divided into seventeen modules. Every module defines a specific target and several tasks that are needed to achieve it. Each module has an input FP7-SEC-285477-CRISALIS 15

2. Standard and an output. The input represents the information needed to perform each task while the output is the result of the accomplished tasks. Figure 2.3 shows the methodology flow. OSSTMM uses the same workflow for all the channels. The seventeen modules and the methodology flow apply in the same way to the five logical sections. However, while the methodology remains the same, each channel will differ in the proposed tasks. The methodology is divided into four phases: Regulatory Phase (modules 1 to 3), Definitions Phase (modules 4 to 7), Information Phase (modules 8 to 13) and Interactive Controls Test Phase (modules 14 to 17). Each phase faces a different level of depth in the audit. The regulatory phase focuses on understanding the audit requirements, scope and constraints. The definitions phase improves the description of the scope and of the inner interactions and processes. The information phase uncovers misplaced and mismanaged information flows in the target. Finally, the interactive controls test phase focuses on practical penetration testing. This is often the final phase of a security test. This ensures that practical attacks do not affect less invasive tests. In what follows we briefly describe the focus of every module. ˆ Posture Review: it is used to define the scope and identify what tests must be performed. This module focuses on rules, norms, regulations, legislations and policies applicable to the target. ˆ Logistics: it defines which limitations the audit process has. It relates to interaction constraints such as physical distance, communication speed, etc. ˆ Active Detection Verification: it takes into account any practical restriction imposed against performing interactive tests on the targets. ˆ Visibility Audit: it determines available targets within the scope. This module provides the first in depth description of the targets and how they interact with the scope. ˆ Access Verification: it measures the breadth and depth of interactive access points. It verifies if such access point to the target exists and if it requires authentication. ˆ Trust Verification: it identifies trust relationships from and between the targets. This module verifies if targets accept interaction freely and without credentials. ˆ Controls Verification: it measures the effectiveness of process-based attributes like non-repudiation, confidentiality, privacy and integrity. 16 SEVENTH FRAMEWORK PROGRAMME

2.1. OSSTMM Figure 2.3.: OSSTMM methodology flow (from [5]) FP7-SEC-285477-CRISALIS 17

2. Standard ˆ Process Verification: it verifies the existence, effectiveness and maintenance of security levels defined for the targets. ˆ Configuration/Training Verification: it explores the default conditions under which targets operate regularly to understand their intents, business justification and reasoning. ˆ Property Validation: it checks the use of illegal or unlicensed intellectual property or application within the targets. ˆ Segregation Review: it checks the amount of personally identifiable information available depending on the applied rules and legislations. ˆ Exposure Verification: it searches for freely available information which uncovers indirected or unwanted visibility of the targets. ˆ Competitive Intelligence Scouting: it searches for freely available information which could harm or adversely affect the target. ˆ Quarantine Verification: it determines the effectiveness of authentication in terms of black and white listing quarantines. ˆ Privileges Audit: it measures the impact of misusing credentials and privileges and the consequences of unauthorized escalations of privileges. ˆ Survivability Validation/ Service Continuity: it checks the resistance of the target to excessive or adverse changes. ˆ Alert and Log Review/End Survey: it is a review of the performed audit activities. The outcome is a comprehensive assessment of the tested channels. Results are presented through Risk Assessment Values (RAVs) that are a set of security metrics providing an overview on the state of the channels (e.g. measurement of the attack surface, the amount of uncontrolled interactions with a target, etc.). To finally measure the thoroughness of the test performed, the OSSTMM suggests to conclude the work with a Security Test Audit Report (STAR). This step requires to record in the appropriate form the following information: ˆ Date and time of test ˆ Duration of test 18 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 ˆ Auditors and analyst involved ˆ Test type ˆ Scope of test ˆ Test Index ˆ Channel tested ˆ Test Vectors ˆ Verified test and metrics calculations of the operational protection levels, loss controls and security limitations ˆ Knowledge of which tests have been completed, not completed, or only partially completed, and to what extent ˆ Any issue regarding the test and the validity of the results ˆ Test error margins ˆ Any processes which influence the security limitations ˆ Any unknowns anomalies Overall, OSSTMM provides a very comprehensive framework that addresses a large variety of practical aspects. It is specifically useful to ensure completeness of our later approach. However, it is also relatively generic and is not going into details of specific systems and is not addressing the CRISALIS scenarios at all. 2.2. NIST SP800-115 2.2.1. General information The National Institute of Standard and Technology (NIST) has designed its own security assessment methodology in the Special Publication 800-115 [6]. The purpose of this publication was to provide guidelines for organizations on planning, conducting and evaluating information security testing. Even if the overall goal is to propose an overview of main key elements of technical security assessments, NIST SP800-115 provides also practical recommendations and technical information relating to penetration tests. FP7-SEC-285477-CRISALIS 19

2. Standard NIST defines the security assessment as the process of determining how effectively an entity being assessed meets specific security objectives. From an high-level point of view, there are three possible ways to accomplish such target: ˆ through Testing, by exercising the assessment objects comparing actual and expected behaviors ˆ through Examination, by analyzing the assessments objects and understand their functioning ˆ through Interviewing, by conducting extensive discussions with organizations and experts to identify possible problems A methodology that implements one of the previous points has to contain at least three different phases: ˆ Planning: this phase is necessary to collect all the information needed for the assessment. This concerns elements like assets, threats, policies in use, etc. Moreover, the assessment should have a final goal and a set of objectives to work towards. ˆ Execution: this phase concerns the operational part of the assessment. The Execution implements operations of environment discovery and analysis, vulnerability identification and result validation. Practical activities for this phase obviously differ from case to case. However, the main goal is to bring to light problems at technical and organizational level. ˆ Post-Execution: this phase describes all the analysis aim at identifying vulnerability causes and mitigation strategies. The development of a final report is the main output of the Post-Execution. As already discussed, NIST SP800-115 offers an overview of security assessment key points. However it suggests several practical techniques to implement penetration testing. According to the authors, the most commonly used pentesting techniques can be grouped in the following categories: ˆ Review Techniques define how to examine and analyze assets and their security policies. ˆ Target Identification and Analysis Techniques define how to identify assessments targets and how to describe their properties. 20 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 ˆ Target Vulnerability Validation Techniques: define how to verify the presence of vulnerabilities within the targets and how to test their security levels. The NIST methodology focuses on how these techniques can be used together, but does not provide a list of technical activities to be used in any specific circumstance. In addition to practical penetration testing techniques, the methodology proposes several non-technical methods that can be used to enforce the assessment (e.g. physical security testing). Finally, NIST SP800-115 concentrates on how singular tests and final reports can be compared with each others. The possibility to compare two security assessments relies on the correct representation of the Testing Viewpoint. Every test can be performed from different viewpoints. This implies different information owned by the auditors or different physical location of the pentesters. Results of comparisons among different assessments always need to consider the kind of testing. NIST SP800-115 defines four testing types: ˆ External Security Testing: this type of testing is conducted from outside the organization s security perimeter. The goal of this security test is to reproduce the view of an external attacker and to draw the attention to vulnerabilities already visible from outside the company premises (e.g. from the Internet). External testing always begins with discovery techniques aiming at examining organization public information. ˆ Internal Security Testing: differently from the previous one, this kind of testing is performed within an organization s perimeter (e.g. internal network). The purpose of the Internal Security Testing is to impersonate a trusted insider or also an attacker that has penetrated external defenses. This test allows pentesters to have some level of access to the network and, thus, to internal information. These elements make it possible to assess internal security mechanisms. ˆ Overt Security Testing: also known as White Hat testing, defines an Internal or External Testing in which pentesters have a full knowledge of organization s systems and processes. Company staff is also aware of the test and it usually tries to limit the testing impact. This kind of test is often used as company training. ˆ Covert Security Testing: also known as Black Hat testing, uses an adversarial approach by not giving any knowledge about the company to the pentesters. In this case, company IT staff is usually also not aware of the test and its response is used for testing the technical and organizational security controls within the company. FP7-SEC-285477-CRISALIS 21

2. Standard 2.2.2. Overview In this section we describe in detail the three groups of techniques discussed before and the three phases into which the methodology is logically structured. Review Techniques describe the process of passively gathering information related to the organization and the consequent analysis operations. This information regards both policies and equipments (e.g. systems, networks, etc.). Since these techniques are passive, they do not interfere with company operations and systems. For this reason, this is usually the first phase in every security assessment. Possible review techniques are: ˆ Documentation Review examines the technical aspects of used polices and procedures. Security flaws or weaknesses in such elements can lead to missing or improperly implemented security controls. The results of the analysis can also be used to better tune the following techniques. ˆ Log Review determines if important information and operations are properly logged and recorded. This is a necessary element to prove that all the systems are working according to the applied policies and procedures. Log analysis can show also misconfigured systems and malicious operations (e.g. attempted or successful attacks). ˆ Ruleset Review checks the collection of rules and signature-based security mechanisms used to detect malicious operations or implement organization s processes. Routers, firewalls and intrusion detection systems are the main targets of this analysis. Router access control lists are checked to identify whether rules are no longer needed or enforce the refusal of unknown traffic. The review refines in depth firewall rules by checking unnecessary open ports and allowed services. Moreover, it searches for possible backdoors and malware. Finally, IDS rulesets are verified to be up to date and tuned to organization s needs. ˆ System Configuration Review searches for weaknesses in systems and applications configurations (e.g. programs not enforcing security or non-compliant with security policies). This review usually concerns: unnecessary services running on servers, improper user account and password settings, etc. ˆ Network Sniffing implements the passive monitoring of a data network. It can rely on communication flows, protocols decoding, and header/payload analysis. It is used to map organization communication infrastructures and gathering information about used systems, applications and services. It can also identify unau- 22 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 thorized and inappropriate activities or capture unencrypted personal information (e.g. usernames and passwords). ˆ File Integrity Checking provides a way to check file modifications. It is usually implemented through checksum. Target Identification and Analysis Techniques focus on identifying active devices through open ports and running services and analyze them looking for vulnerabilities. NIST SP800-115 discusses several different techniques belonging to this group: ˆ Network Discovery concentrates on discovering active and responding host on a network. Active techniques send various types of packets to the network waiting for responses. This system is usually an improvement of the passive analysis discussed in the Review Techniques as it is able to find hosts not currently involved in any communication. During this analysis, pentesters have to face firewalls and intrusion detection systems that usually identify many instances of scans. Active discovery generally produce network noise and communication delays. ˆ Network Port and Service Identification is usually performed in parallel with the network discovery. This check involves the use of port scanners able to understand services and application running on remote hosts. Information gathered during the process allows pentesters to exploit OS fingerprinting tools. Such applications guess operating systems working on the network and provide a description about their versions and updates. ˆ Vulnerability Scanning, like the previous two operations, collects information about hosts in the network. Moreover, it also attempts to identify specific vulnerabilities on discovered hosts. Vulnerability scanners usually provide: a check on the compliance of security policies, a list of targets for the following penetration testing, and preliminary information on how to mitigate discovered vulnerabilities. ˆ Wireless Scanning: as the number of wireless devices is increasing, these techniques are becoming more and more important. Wireless scanning techniques can be further divided in: passive scanning, active scanning, bluetooth scanning, and device location tracking. An example of passive scanning involves the analysis of MAC addresses (e.g. vendor identification or white/black listing of devices). An active scanning can be used instead to verify authentication mechanisms and data encryption. Bluetooth security issues have been extensively discussed in literature [7], [8]. Bluetooth scanning is more difficult than the wireless one due to its short FP7-SEC-285477-CRISALIS 23

2. Standard range. However, a confirmation of compliance with organization security requirements is needed to have a comprehensive security assessment. Finally, wireless networks can be used to locate organization s devices. This check enforces the reconstruction of the company s network topology. Target Vulnerability Validation Techniques uses the information found in the two previous phases and further explore the existence of potential vulnerabilities. The target is usually not limited to identifying the vulnerability but also to show the security exposure by trying to exploit it. This procedure is often risky for the organization infrastructure and processes. For this reason it is usually the last active work performed on the network. In this way, it will not compromises other tests. NIST SP800-15 defines and implements its penetration testing method in this phase. The targets of the penetration testing can be numerous. Pentesters are interested in understanding how well the system tolerates real attacks, identifying necessary countermeasures, and evaluating defenders abilities. Moreover, this attack simulation can show what level of skills attackers need to compromise organization s systems. The proposed penetration testing is divided in four steps as show in Figure 2.4. Figure 2.4.: NIST SP 800-115 penetration testing steps The Planning step prepares the necessary information and background for the attack. The Discovery step integrates information already owned by pentesters with a new session of network scanning. With respect to the other phases, the attackers try to acquire information about: host names and IP addresses, employee names and other contacts, and system information (e.g. names and shared resources). In some cases also techniques as dumpster diving can be used to enforce the process. The Discovery step improves also the vulnerability analysis and finalizes the list of targets. The Attack 24 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 step is the core of every penetration testing. During the attack a continuous feedback is provided back to the discovery step as succeeding to access a system gives pentesters additional knowledge of the infrastructure under attack. Figure 2.5 shows an example of attack. Figure 2.5.: NIST SP 800-115 attack example (from [6]) NIST SP800-115 organizes the most common vulnerabilities exploited during the Attack step in the following categories: ˆ Misconfiguration: misconfigured security settings (e.g. insecure default settings). ˆ Kernel Flaws: security flaws in the core code implementing an operating system that can endanger the whole system. ˆ Buffer Overflows: inadequate checks on input sizes. This misbehavior can cause arbitrary code been executed with the privileges of the running program. ˆ Insufficient Input Validation: missing checks on the content of input strings, files, etc. Network applications without filters can run malicious commands (e.g. FP7-SEC-285477-CRISALIS 25

2. Standard SQL injection). ˆ Symbolic Links: as the operating system allows to change permissions related to files, a malicious user can trick the system by creating symbolic links and modifying permission passing through them. ˆ File Descriptor Attacks: malformed file descriptors can grant access to the related files. ˆ Race Conditions: an attacker can take advantage of high privileges processes by modifying their execution order and manipulate results. ˆ Incorrect File and Directory Permission: incorrect permissions can cause various lacks of information (e.g. reading and writing password files). The last step, called the Reporting, occurs simultaneously with the other three steps and records all the results. At the end of the test, pentesters report the identified vulnerabilities, provide a risk rating, and propose possible mitigation techniques for discovered weaknesses. Security Assessment Planning provides information on creating an assessment policy, scheduling tests, identify the most suitable approach, and addressing possible problems. It also enforces assessments to be compliant to current regulations and legislations. According to NIST SP800-115 this planning activity should identify: ˆ Organizational requirements assessments have to take into account ˆ Roles and responsibilities ˆ Adherence to already established methodologies ˆ Assessment frequency ˆ Documentation requirements In this phase organizations have to choose assessment techniques and customize them to fit the requirements. Organizations should also carefully consider risks related to use such techniques on their infrastructures. The selection of auditors and the identification of needed skills is another fundamental step of the planning. Assessors experience can determine the success or the failure of the evaluation. Organizations can choose the type of assessment (e.g. External/Internal and Overt/Covert) and finally propose technical tools and resources available to the assessors. At the end, the assessment plan should answer to the following basic questions: 26 SEVENTH FRAMEWORK PROGRAMME

2.2. NIST SP800-115 ˆ What is the scope of the assessment? ˆ Who is authorized to conduct the assessment? ˆ What are the assessment s logistics? ˆ How should sensitive data be handled? ˆ What should occur in the event of an incident? Security Assessment Execution phase highlights key points for assessors to consider throughout the execution phase. Assessors have to be coordinated with various entities and stakeholders within the organization. Appropriate organization s personnel should help and supervise the assessment for the whole duration of the activity. During operations assessors can face different kind of problems (e.g. technical, operational, political, etc.). These can include: ˆ Resistance: to the assessments, from entities within the organization (e.g. network administrators or end users). ˆ Lack of Realism: that indicates the preemptive modification to system settings in order to make the infrastructure more secure to pass the tests. ˆ Immediate Mitigation: that describes the adjustment and the resolution onthe-fly of weakness found during the assessment. This can disturb following tests and compromise the final report. ˆ Time: time constraints are usually a problem as security assessments are often an exception more than a normal operation integrated in the development and deployment cycle. ˆ Resources: organization can make a stand on providing and maintaining adequate resources for security assessments. ˆ Evolving Technology: tools used for security assessments as well as used techniques can be out-of-date. ˆ Operational Impact: even if a security assessment should prevent any impact on infrastructure functioning, there is always a chance of accidental problems. For this reason, an incident response plans should be already in place during testing. FP7-SEC-285477-CRISALIS 27

2. Standard Assessors can make some preliminary analysis on the assessment during the assessment itself. This preliminary analysis of the causes of the noticed problems becomes the input for the last phase of the security assessment. Most common problem causes can be organized in the following sets: ˆ Insufficient patch management ˆ Insufficient threat management ˆ Lack of security baselines ˆ Poor integration of security into the system development life cycle ˆ Security architecture weaknesses ˆ Inadequate incident response procedures ˆ Inadequate training for end users and network/systems administrators ˆ Lack of security policies or policy enforcement Finally, the Post-Testing Activities concentrates on finalizing the analysis and proposing mitigation actions and recommendations. Security testing reports should be used by the organization as follows: ˆ As a reference point for corrective action ˆ In defining mitigation activities to address identified vulnerabilities ˆ As a benchmark for tracking an organization s progress in meeting security requirements ˆ To assess the implementation status of system security requirements ˆ To conduct cost/benefit analysis for improvements to system security ˆ To enhance other life cycle activities (e.g. risk assessment) ˆ To meet reporting requirements Overall, NIST SP800-115 also provides a well-structured and complete approach that goes into more technical detail than OSSTMM. From this perspective, it is interesting to discuss whether such details are necessary and helpful and make a methodology more applicable or whether they just inevitably restrict its use to certain use cases. 28 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF 2.3. ISSAF 2.3.1. General Information The Information Systems Security Assessment Framework (ISSAF) [9] is a peer reviewed structured framework designed by the Open Information Systems Security Group (OISSG). The methodology defined by ISSAF covers all the aspects related to security assessments: from an high-level perspective (e.g. business impact and organizational models) to practical techniques (e.g. security testing of passwords, systems, network, etc.). The framework is divided in four main phases structured in several working packages (named activities ). The four phases are respectively: Planning, Assessment, Treatment, and Accreditation. Figure 2.6 shows ISSAF workflow. Figure 2.6.: ISSAF framework overview (from [9]) The Planning phase concerns all the operation needed to define a project plan for the FP7-SEC-285477-CRISALIS 29

2. Standard whole assessment process. Before proceeding with the operational part, organizations have to motivate the assessment to substantiate the underlying concerns. Moreover, organizations have to prepare a business plan to cover costs and identifying available personnel and resources. The main steps in this phase are: ˆ Information Gathering ˆ Project Chartering ˆ Resource Identification ˆ Budgeting ˆ Cash Flow ˆ Work Breakdown Structure ˆ Project kick-off The Assessment phase defines the approach needed to evaluate the Information Security Risks within an enterprise. This phase focuses on the risk assessment process and addresses every component involved. It is divided into two categories: Inherent Risk Identification and Controls Assessments. The first consists of the following activities: ˆ Identification of Assessment entities: during this activity the organization spots the targets of the assessment (e.g. processes, assets, facilities, etc.). ˆ Identification of Threats and Vulnerabilities: this activity describes how to search, list and describe organization s vulnerability. ˆ Impact Assessment: this activity measures the impact of an exploited vulnerability to the business of the organization ˆ Likelihood Assessment: during this activity the organization evaluates the likelihood that a vulnerability is exploited. Control Assessment explores instead the following fields: ˆ Evaluation of Legal and Regulatory Compliance: this activity evaluates the position of the organization with respect to current regulations and legislations. 30 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF ˆ Evaluation of Enterprise Information Security Policy: this activity focuses on understanding current organization s approach on implementing and maintaining its security policies. ˆ Evaluation of Enterprise Information Security Organization and Management: this activity directly follows the previous one, as a review on how the Information Security is organized inside the company. ˆ Assessment of Enterprise Information Systems Security and Controls: this activity is in charge of reviewing different security aspects of the organization. For example: physical security, environmental security, technical controls (e.g. network security, host security, etc.), social engineering, etc. ˆ Evaluation of Enterprise Security Operations Management: during this activity the organization gains an understanding of the risk of specific operations processes. The operational areas involved in this operations are: Capacity Management, Vulnerability Management, User Management, etc. ˆ Evaluation of Enterprise Business Continuity Management: this activity evaluates organization capability of ensuring continuous availability of the Information Technology infrastructure. The output of the Assessment phase is reported within the Inherent Risk and the Residual Risk documents. The Treatment phase implements a platform for taking decision about the aforementioned risks. Its work-flow goes through the selection of safeguards, the development of implementation plan for countermeasures, and the development of a decision making process. In this phase the organization decides to accept, mitigate, or avoid vulnerabilities and associated risks identified in the previous phase. Finally, the process of Accreditation defines the steps needed to an organization to obtain the ISSAF certification. 2.3.2. Overview In this section we will review the Penetration Testing methodology proposed by IS- SAF leaving aside all practical details regarding specific network components assessing methods. ISSAF penetration testing methodology evaluates organization s network and systems. It consist of three phases and nine assessment steps. The three phases are: Planning and Preparation; Assessment; Reporting, Clean-up and Destroy Artifacts. FP7-SEC-285477-CRISALIS 31

2. Standard The Planning and Preparation phase comprises all the assessment s preliminary operations. These regards: requirements definition, actions to be performed, and expectations. ISSAF requires the identification of a contact person from the target company and those who will perform the tests. During the kick-off meeting both parties have to agree on the specific engagement team, the exact dates, times of the test, escalation path and other arrangements. The meeting has to be concluded with the signing of a formal Assessment Agreement that will provide mutual legal protection. The Assessment phase is the core of the penetration testing methodology. ISSAF approach is a path that leads the attacker to nine operation steps as shown in Figure 2.7. ˆ Information Gathering: ISSAF defines the information gathering in the broader sense possible. It comprises both technical and non-technical methods. The former looks for DNS records and exploit tools like WHOIS. The latter implements some social engineering methods (e.g. using search engines and mailing lists to find information about the target). The more a pentester knows about his victim the more he will improve his chances to successfully attack it. During this phase, the attacker does not always need to establish a contact with the target. Important information can be collected also from public sources (e.g. the Internet). As security assessments are generally limited in time and resources, information gathering is important to identify points that will be most likely vulnerable and focus on them. ˆ Network Mapping: this phase directly follows the previous one and refines the information gathering with a more technical approach. The target is to identify what devices are working in the network and outline its topology. Network mapping involves the use of fingerprinting and network analysis tools. More in details, ISSAF specifies different classes of actions to be performed: Find live hosts Port and service scanning Perimeter network mapping (e.g. router, firewalls) Identifying critical services Operating System fingerprinting Identifying routes using Management Information Base (MIB) Service fingerprinting 32 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF Figure 2.7.: ISSAF pentesting overview (from [9]) FP7-SEC-285477-CRISALIS 33

2. Standard At the end of this phase, pentesters will have the possibility to confirm or dismiss some preliminary made hypotheses regarding target systems. ˆ Vulnerability Identification: once specific targets are selected, the attackers will perform several activities to detect actual exploitable vulnerabilities in the system. ISSAF lists some of the activities used to detect weak points: Identify vulnerable services using service banners Perform vulnerability scans Perform false positive and false negative verification Enumerate discovered vulnerabilities Estimate probable impact Identify attack paths and scenarios for exploitation ˆ Penetration: this phase is the first actual operation against the target. The attacker will try to circumvent security measures and gain access to the system under attack. ISSAF methodology emphasizes the following steps within the penetration: Find proof of concept code/tools to use for the attack Develop new tools/scripts if needed Test proof of concept code/tools before use them to avoid problems during the actual pentesting Use proof of concept code against target Verify or disprove the existence of vulnerabilities Document findings ˆ Gaining Access & Privileges Escalation: in this phase pentesters will confirm the possibility to access the target. To do that the attacker has to compromise all the intermediate systems in the path leading to the physical system under attack (e.g. firewalls, routers, etc.). Moreover, the attacker will usually access the system with the least privileges possible and he will try to obtain administrator s rights exploiting local vulnerabilities with root-kits. ˆ Enumerating Further: this phase lists malicious activities the attacker can perform within the compromised system. Such activities are: Obtain encrypted passwords for off-line cracking Obtain passwords by using sniffing or other techniques 34 SEVENTH FRAMEWORK PROGRAMME

2.3. ISSAF Sniff traffic and analyze it Gather cookies and use them to exploit sessions and for password attacks E-mail address gathering Identifying new routes and networks Mapping internal networks ˆ Compromise Remote Users/Sites: this phase focuses on the presence of secure communications (e.g. Virtual Private Networks) between remote users/sites and enterprises network. In this scenario, the attackers can try to extend his control to remote users and detached networks. In case of success, the attacker will repeat all the previous steps in the new environment. ˆ Maintaining Access: in this phase pentesters make sure to control the system also in the future. As a vulnerability can always be patched by system operators an attacker can install malicious code in the system to create covert channels and backdoors. Such verification shows the degree of exposure that a system can have once compromised. The Maintaining Access phase is often not performed as part of a penetration test due to the risks involved (e.g. a real attacker capable to take possession of the covert channel and gain access to the system). ˆ Covering Tracks: this phase ensures attackers to be invisible to later analysis. Usually, during a penetration testing, attackers should be as open as possible about their operations (unless the company expressly requests otherwise). There are several actions that pentesters can perform to hide their traces: Hide modified files and used tools Clear logs by checking history and edit log files Defeat integrity checking Defeat Anti-viruses Implement customized Root-kits ˆ (optional) Audit: system audits should be performed after completing a penetration test. This phase can usually bring to light further potential vulnerabilities within the system. The Reporting, Clean up and Destroy artifacts phase concludes the assessment. According to ISSAF a minimal reporting should consist of: FP7-SEC-285477-CRISALIS 35

2. Standard ˆ Verbal Reporting: that describes the feedback the company has to have immediately after the pentesting. The attackers will give to the company an overview on their finding and they will focus on major critical issues. ˆ Final Reporting: that is an in depth presentation with detailed results of the tests. Each discovered vulnerability has to be presented with related countermeasures and a proposal on how to implement them. The report should follow a well documented structure. ISSAF proposes a list of information needed for the document: Management Summary Scope of the project Tools that have been used Dates & times of the actual tests on the systems Outputs of tests performed A list of all identified vulnerabilities with included recommendations on how to solve problems found A list of action points Finally, all information that is created and/or stored during the test on target systems should be removed. If this is not possible, such information (e.g. files and directories) should be mentioned to system operators to proceed to their removal. 2.4. NESCOR Guide to Penetration Testing for Electric Utilities 2.4.1. General Information The NESCOR Guide to Penetration Testing for Electric Utilities was developed by Justin Searle in 2012. The document is a support to security assessments involving smart grids but focuses only on penetration testing. IT does not provide a detailed, step-by-step procedure but focuses on high-level descriptions of penetration testing tasks and overall goals. According to the authors, the NESCOR guide should help to organize the whole testing process in phases and tasks breaking down the complexity of the process. The NESCOR guide is divided into nine main parts interconnected with each other as shown in Figure 2.8. The tests start with the definition of a scope of the engagement. Then, the pentesters have to analyze the architecture of the target system and, 36 SEVENTH FRAMEWORK PROGRAMME

2.4. NESCOR Guide to Penetration Testing for Electric Utilities if possible, replicate it in a safe environments. The guide suggests to always proceed with the penetration testing in a test environment. The testing process is divided into four main tasks: Server OS Penetration Tasks, Server Application Penetration Tasks, Network Communication Penetration Tasks, and Embedded Device Penetration Tasks. The guide ends with an overall analysis on the previous tasks and a final process of result interpretation and reporting. Figure 2.8.: NESCOR schema (from [10]) Two important elements enrich the description of the penetration testing activity. The first regards the time constraint. In addition to goals and descriptions, each task described within the document is labeled with an estimated level of effort. The authors described four time windows to specify how much time an experienced tester would take to complete each task. These windows are: 1-4 hours (low effort), 5-16 hours (medium effort), 17-40 hours (high effort), 41+ (extremely high effort). FP7-SEC-285477-CRISALIS 37

2. Standard The second element discussed in the guide represents the likelihood that an utility should consider performing a specific task. This element represents also a relative risk for the target subsystem to be compromised in a real attack scenario. This element is shown by the colors: ˆ Green: frequently performed tasks that require the most basic penetration testing skills ˆ Yellow: commonly performed tasks that require moderate pentesting skills ˆ Orange: occasionally performed tasks that require high level expertise ˆ Red: infrequently performed tasks requiring specialized skills In what follows we describe the phases outlined in Figure 2.8. 2.4.2. Overview The Penetration Test Scoping phase is used to organize tests and tools based on both overall and specific goals. In this section, the document gives a brief explanation on how to use the methodology and provide short advices on the frequency the penetration testing should be performed on an infrastructure. The Architecture review and the Target System Setup are instead a detailed description of the infrastructure under analysis and its comparison with some reference architectures to which the guide is aimed for. Most important, in these two sections, the document advices pentesters to perform tests on non-production systems and devices. In case where staging environments do not exist, the pentesters should implement only low-risk, non-intrusive penetration testing activity on production systems. Considered infrastructures are: ˆ Advanced Metering Infrastructure (AMI): includes components that goes from the deployed meters to the headend and data center servers. Pentesters are supposed to have physical access to the infrastructure and the possibility to eavesdrop dataflows. The guide warns on the low chance to find specialized tools for every kind of AMI infrastructure and device and advices the tester to develop new tools based on the specific needs. ˆ Demand Response (DR): describes devices exploited in the path from the energy sources to the gateways to the Demand-Response servers. Such gateways usually involve Energy Management and Control Systems (EMCS) and Home Area 38 SEVENTH FRAMEWORK PROGRAMME

2.4. NESCOR Guide to Penetration Testing for Electric Utilities Networks (HANs). DR infrastructure may include other intermediate devices such as Building Automation Systems (BASs) and Data Collection Units (DCUs). ˆ Distributed Energy Resources (DER): are cyber-physical systems that provide energy or additional services to the power grid. This infrastructure can involve generators and storage devices but also electric vehicles. Pentesting these components requires testers to have again physical access to the devices. ˆ Distribution Grid Management (DGM): includes a number of different subsystems used within utility s electric grid. Involved devices are: automated reclosures and switches, load monitors, substation relays, remote fault indicators and capacitor banks. It is unlikely that a penetration testing focuses on all the previous devices. However, testers should focus at least on a subset of controlled field devices and the vendor s management server used to configure them. ˆ Electric Transportation (ET): involves components from Electric Vehicles (EV), Electric Vehicle Supply Equipments (EVSE), and the EV Management Servers. ˆ Wide Area Monitoring, Protection, and Control (WAMPAC): includes synchrophasor devices such as Phasor Measurement Units (PMUs) and Phasor Data Concentrators (PDCs). Also this infrastructure needs to be physically accessible to carry on the penetration testing. When the target is set, the guide describes the practical pentesting activity within four building blocks or tasks : Server OS Penetration, Server Application Penetration, Network Communication Penetration, Embedded Device Penetration. In the Server OS Penetration phase, pentesters test control servers operating systems. This is usually performed through standard network pentesting activities and does not require ICS knowledge. The goal of this phase is the identification of known or unknown vulnerabilities that can cause the attacker to be in control of the control servers. The Server OS Penetration follows the schema shown in Figure 2.9. The Information gathering step allows pentesters to acquire information on the system (e.g. looking at DNS records, scanning for active services on network hosts, etc.) 2.10. The Vulnerability analysis will exploit the information found in the previous step to look for vulnerabilities within the system 2.11. Finally the Exploitation will use such vulnerabilities to gain access and control over the system 2.12. In the Server Application Penetration task, pentesters must focus on applications running on control servers. The NESCOR guide allows to use other standard methodologies to perform this task (e.g. Open Web Application Security Project OWASP). The FP7-SEC-285477-CRISALIS 39

2. Standard Figure 2.9.: Server OS Penetration flow (from [10]) Figure 2.10.: Server OS Penetration flow - Information Gathering (from [10]) 40 SEVENTH FRAMEWORK PROGRAMME

2.4. NESCOR Guide to Penetration Testing for Electric Utilities Figure 2.11.: Server OS Penetration flow - Vulnerability Analysis (from [10]) main target of this phase is to gain control of the applications running within the control servers. The Server Application Penetration phase is divided as shown in Figure 2.13. The schema proposed in this phase is similar to the one presented for the Server OS Penetration. The first step concerns application identification and information gathering 2.14. The second one focuses on discovering vulnerabilities within the applications 2.15. Finally, the last step asks pentesters to proceed with the actual exploitation of the targeted applications 2.16. The Network Communications Penetration section focuses on testing communications flowing within the infrastructure (e.g. communication on a field area network of a smart grid). This phase concerns both wireless and wired communications and concentrates both on the network architecture and protocols used. The goal of the Network Communication Penetration is to prove that an attacker can eavesdrop communication, take control of the network and subvert devices trough protocol manipulation. Pentesters perform network communication testing following the schema presented in Figure 2.17. The two subcategories proposed by NESCOR on Network Communications Penetration are: RF Packet Analysis and Network Protocol Analysis. The first concentrates in elements such as: frequency hopping, modulation, multiplexing and data encoding 2.18. The second protocol decoding and exploiting 2.19. The Embedded Device Penetration section checks if industrial devices are exposed FP7-SEC-285477-CRISALIS 41

2. Standard Figure 2.12.: Server OS Penetration flow - Exploitation (from [10]) 42 SEVENTH FRAMEWORK PROGRAMME

2.4. NESCOR Guide to Penetration Testing for Electric Utilities Figure 2.13.: Server Application Penetration flow (from [10]) Figure 2.14.: Server Application Penetration flow - Application Mapping (from [10]) FP7-SEC-285477-CRISALIS 43

2. Standard Figure 2.15.: Server Application Penetration flow - Application Discovery (from [10]) Figure 2.16.: Server Application Penetration flow - Application Exploitation (from [10]) 44 SEVENTH FRAMEWORK PROGRAMME

2.4. NESCOR Guide to Penetration Testing for Electric Utilities Figure 2.17.: Network Communication Penetration flow (from [10]) Figure 2.18.: Network Communication Penetration flow - RF Packet Analysis (from [10]) FP7-SEC-285477-CRISALIS 45

2. Standard Figure 2.19.: Network Communication Penetration flow - Network Protocol Analysis(from [10]) 46 SEVENTH FRAMEWORK PROGRAMME

2.4. NESCOR Guide to Penetration Testing for Electric Utilities to physical attacks. In fact, especially with smart grids, embedded devices are deployed within areas that are not easily accessible (e.g. customer premises, pole-tops, substations, etc.). Moreover, the Embedded Device Penetration advices pentesters to check on data storing components (EEPROM, RAM, Flash memories, etc.), buses (serial and parallel) and ports (serial, parallel and infrared/optical) within the components. The goal of this phase is to extend attacker control on field devices. The Embedded Device Penetration tasks are arranged as shown in Figure 2.20. Figure 2.20.: Embedded Device Penetration flow (from [10]) The Electronic Component Analysis focuses on devices design weaknesses (e.g. unprotected storage or communication within the device) 2.21. The Field Technician Interface Analysis is used to identify vulnerabilities within communication interfaces 2.22. Finally, the Firmware Binary Analysis concentrates on the software running within the device 2.23. It is worth mentioning that NESCOR advices to perform this last test of the guide only as an alternative of source code analysis. NESCOR guide ends its analysis with two final phases: End-to-End Penetration Analysis and Result Interpretation and Reporting. The first concerns the study of those communication flows that allow the infrastructure under analysis to exchange information with other infrastructures that are not in the scope of the penetration testing. This analysis allows to identify potential risks concerning malicious data coming from outside systems and components remotely connected to the infrastructure. Finally, the Result Interpretation and Reporting focuses on documenting the previous steps and achievements and present them to the customers. In this phase pentesters must provide an evaluation on the security of the infrastructure and inform the customers on the likelihood to be successfully attacked. The final report should include: FP7-SEC-285477-CRISALIS 47

2. Standard Figure 2.21.: Embedded Device Penetration flow (from [10]) 48 SEVENTH FRAMEWORK PROGRAMME

2.4. NESCOR Guide to Penetration Testing for Electric Utilities Figure 2.22.: Embedded Device Penetration flow (from [10]) FP7-SEC-285477-CRISALIS 49

2. Standard Figure 2.23.: Embedded Device Penetration flow (from [10]) ˆ Executive Summary: short documents describing vulnerabilities root causes and business strategy to address them ˆ Introduction: description of the goals of the test and the target architecture ˆ Methodology: technical report on tests and tools used during the analysis ˆ Finding and Recommendations: detailed analysis on the results, possible impacts, and proposals for solutions. ˆ Conclusion: summary of the previous To the best of our knowledge the NESCOR methodology is one of the first attempts to provide a schematic approach to pentesting critical infrastructures. 50 SEVENTH FRAMEWORK PROGRAMME

3. Requirements 3.1. Approach Developing a security assessment methodology for critical infrastructure needs the input of different expertises. Within the CRISALIS project we merge the knowledge of three important stakeholders: Academia, SCADA vendors, and SCADA operators. We decided to collect viewpoints and understand industry requirements by interviewing several experts. Over the last year we performed eight interviews mostly among project partners and members of the advisory board. Our choice of the participants was led by the number of years of experience in security and especially in critical infrastructures. The experts were asked about present and future challenges that industries, governments and academia have to faces to keep ICS protected against common and targeted attacks. The core of the interview was a discussion on the value and support a new penetration testing security methodology could give to ICS security. We discuss the outcomes of these interviews in Section 3.3. To further enlarge the information basis that our approach is built upon, we also decided to create a questionnaire about testing security infrastructure. The purposes of interviewing CI and security experts are several. There is the need to understand what are the requirements and the expected feedbacks. Moreover, we also need to validate the benefits of establishing a methodology tuned to CIs and also evaluate its comprehensiveness and accuracy. Using a questionnaire seemed the most efficient way to reach a broader community and inquire about the state of the art of CI testing in general and to identify possible steps forward toward the definition of the more comprehensive concept of CI vulnerability assessment. The questionnaire is then extended by more detailed interviews with some stakeholders. The questionnaire is divided into four sections: General information; Security requirements, design and regulations; Testing and methodologies; and Research. The General Information section focuses on characteristics of the interviewed persons. Information on area of expertise and experience will be used to evaluate the later responses and comments. The security requirements, design and regulations section collects information about 51

3. Requirements deployed security mechanisms and authentication schemata. Moreover, the section investigates on what limitations there are in deploying security mechanisms in Critical Infrastructures. The testing and methodologies section is the core part of the questionnaire. It aims to identify currently used security practices focusing on: kind of tests performed, test frequency, and tools used. Furthermore, the section asks about the expectations the experts have from a CI security methodology. Finally, the Research section concludes the questionnaire asking about CI security research status. We believe that integration of CRISALIS expertise and questionnaire outputs provide the necessary background from which to build a comprehensive security assessment methodology for critical infrastructure. Such a methodology is supposed to take into account already existing best-practice provided by industry and different, but relevant, IT fields. The output has to be compliant with CI experts and operators needs and should allow them to have a common evaluation framework to compare their infrastructures. Section 3.2 presents the questionnaire we used to collect information. The questionnaire is also available online at Critical Infrastructure Security & Penetration Testing [11]. 3.2. Questionnaire Critical Infrastructure Security & Penetration Testing Industry/Researcher Questionnaire University of Twente December 2012 This questionnaire is anonymous unless you want to participate in the follow up mentioned in section E. The purpose of this work is exploring the need for a standard penetration testing methodology for Critical Infrastructure. With the following questions we will collect ideas and requirements useful to realize such methodology. N.B. If you have never worked with Critical Infrastructure please fulfill sections B and C with your opinion on how these systems are supposed to work and be tested. Questions marked with * are intended only for Critical Infrastructures operators/deployers. 52 SEVENTH FRAMEWORK PROGRAMME

A. General information A. General information 1. In which area are you working? SCADA Vendor SCADA End User SCADA Consultant Security Consultant Academia Other: Comments: 2. What is your domain of interest? (Multiple selections are possible) Water Energy Oil & Gas Transportation Other: Comments: 3. Briefly describe your field of work/research and your job. 4. How much experience do you have in working with Critical Infrastructures (CIs)? (little) (a lot) Comments: FP7-SEC-285477-CRISALIS 53

3. Requirements 5. How much experience do you have in security? (little) (a lot) Comments: 6. How important is computer security in your domain? (little) (a lot) Comments: B. Security requirements, design, and regulations 7. What types of security components do you use/manifacture/deploy? (Multiple selections are possible) Firewall Application White-listing Intrusion Detection System Intrusion Prevention System Forensic Analysis Tool Other: Comments: 8. Rank these limitations to security in order of importance from 1 (High importance) to 6 (Low importance). ˆ Costs are a big factor limiting the amount of research and spending on security. ˆ Time constraints. solutions. No time to investigate all potential security issues and ˆ Off-the-shelf security technology are not suitable for CIs and their proprietary software. 54 SEVENTH FRAMEWORK PROGRAMME

C. Testing and methodologies ˆ The underlying hardware used in most CI cannot handle modern security features. ˆ Regulations/laws do not allow all security features. ˆ There is no incentive to secure every aspect of CIs. ˆ Other: Comments: 9. * How important is the correct functioning of security components in your system? (little) (a lot) Comments: 10. * How is the access to the systems information and specification controlled? (Multiple selections are possible) Only those who have a need-to-know can access such information For some specific aspects a very small set of people has access to such information Specific area of the company terrain are only accessible by authorized personel Only the manufacters/suppliers have the full specification of the components Open source software is used Also internally, the security tests can mostly be done as black-box tests Other: Comments: C. Testing and methodologies 11. What kind of security tests are done? (Multiple selections are possible) Stress tests and Denial-of-Service tests Penetration tests FP7-SEC-285477-CRISALIS 55

3. Requirements Security code review Formale code verification Other: Comments: 12. How often are these tests performed? (Multiple selections are possible) At design time During development During the setup Regularly during operation When some changes occur Other: Comments: 13. Are security tests mainly black-box (system s specifications unknown) or white-box (full disclosure of information to pentesters)? Black-box White-Box Unknown Comments: 14. * Is it possible to test the system remotely (passing through public networks)? Yes No Unknown Comments: 15. * Is the security testing outsourced, i.e. done by externals? No, everything is tested in-house. 56 SEVENTH FRAMEWORK PROGRAMME

C. Testing and methodologies No, generally everything is done in-house but in some cases external security testers are included. Yes, but only specific elements are tested by externals. Yes, the whole system is tested for security by externals. Unknown Other: Comments: 16. Is the testing performed within company premises? Yes, always. If outsourced, it can be performed also outside company premises. Unknown Other: Comments: 17. For web servers security/penetration testing methodologies and tools exist. Are there any methods/procedures/standards used for testing CI security? No, I am not aware of any general procedures/standards. No, there are no standards but some methodologies specifically designed for each CI exist. Yes, internally a set of documented procedures exist which are followed for each CI. Yes, published/standardized methods are used for testing wherever possible. Unknown Other: Comments: 18. Can you mention some methods used? FP7-SEC-285477-CRISALIS 57

3. Requirements 19. What are the procedures for security testing CIs? How is the security tested? (Multiple selections are possible) A security test is done on the whole live system. A security test is done when the system is offline. A security test is done on a backup system. Different components are tested separately Unknown Other: Comments: 20. The security tests are done by... (Multiple selections are possible) The team working on the component(s). An internal security team. An external security team. Unknown Other: Comments: 21. Are there any CI-specific tools/software for the testing? (Multiple selections are possible) No, I am not aware of any CI-specific tool. Some simple tools are developed when necessary. Yes, existing software tools are used for testing specific components. Yes, we developed our own tools/software for testing the security of the ICS. 58 SEVENTH FRAMEWORK PROGRAMME

C. Testing and methodologies Unknown Comments: 22. Referring to previous question, can you mention any of such tools/software? 23. Is there a need for a (standard) CI penetration testing methodology? No. There is no incentive to do thorough security testing. A little. The methodologies/documented procedures will not be used. We do a flexible security testing strategy that is decided upon for each individual case. Yes. General guideline methodologies are good, but there is no need for more specific methodologies. Yes. A CI-specific and comprehensive methodology would make security testing more organized/ repeatable/ comparable. Unknown Comments: 24. This CI penetration testing methodology should... (Multiple selections are possible) include a guideline for testing the whole CI. include a guideline for testing specific components. include general guidelines about each testing aspect. include very detailed information about each testing aspect. suggest software/tools to be used. state how to document the results. Other: FP7-SEC-285477-CRISALIS 59

3. Requirements Comments: 25. Are there any areas/aspects/topics that need special attention? D. Research 26. Should more research be done in the area of CI security (either at university or at companies)? No, there is no incentive to do thorough security testing. No, there is already a lot being done (also in non-ci areas that can be applied to CI). Yes, more time should be spent in studying CIs security. Other: Comments: 27. Additional comments: E. Follow up 28. Would you agree to be contacted for a more in depth discussion? Yes No 60 SEVENTH FRAMEWORK PROGRAMME

3.3. Analysis 29. If yes, please provide us your email address: Thanks for your time. In case you provided your email address, we will contact you to organize a 1 hour phone interview. 3.3. Analysis In what follows we provide a summary of the outcomes of the interviews integrated with the trends observed in the answers collected with the questionnaire. 3.3.1. Security Deployment Limitations First of all, all the interviewed confirmed that Critical Infrastructures and Security are two inseparable concepts. Computer security is always considered a priority in implementing and managing Critical Infrastructures. Starting from this confirmation we investigated the major obstacles to address security issues in such infrastructures. There are several limitations in deploying security. Most of the interviewed experts identified costs as the biggest obstacle to properly ensure security. Costs are always a big factor limiting the amount of research and spending on security. This problem has revealed two different yet close aspects. Costs are especially problematics for small companies. Big companies can usually afford to invest in security but such investment is limited by a lack of information sharing between security departments and financial departments. Some security experts reveal several difficulties in explaining security requirements and threats to non-expert people. Time constraints are another crucial limitation to security deployment. Experts rate this limitation almost as important as cost constraints. Our interviews spotted that there are often situations in which the time available to investigate all potential security issues is limited or even insufficient. This regards also the time needed to deploy effective security solutions within the critical infrastructure. Lower in the rank of limitations in deploying security, some interviewed emphasize the non-interoperability of Off-the-Shelf security technologies and industrial control system devices. However, this issue can be solved. On the one hand, Off-the-Shelf tools FP7-SEC-285477-CRISALIS 61

3. Requirements and techniques can be customized and used on critical infrastructures and their proprietary software. On the other hand, critical infrastructure s underlying software, even outdated, can usually handle modern security features. Finally it is worth mentioning that all the interviewed agreed that laws and regulations already in place do not impede operators to deploy new security features to their infrastructures. However, such security deployments should be encouraged and supported by government entities. Summarizing the discussion on issues and limitations to security deployments in critical infrastructures, we believe that a penetration testing technique and, more in general, an assessment methodology for critical infrastructures should focus on reducing cost and being time efficient. Standard methodologies are usually comprehensive sets of tasks that cover all the possible security tests and best practices. We believe that such methodologies should be reduced to a minimum but effective set of activities in order to save money and time. One solution to the issue could be the automation of some testing phases. In general, penetration testers and auditors should cut all disposable processes from their work and maximize the throughput. At the same time, external testers may have a clearer time and cost budget they can plan with. 3.3.2. Security solutions According to our interviews there are no preferences in choosing a specific kind of security solution with respect to the others. Firewalls, Intrusion Detection and Prevention Systems are widely deployed in any kind of network. Moreover, operators use several security schemes to protect accesses (e.g. white or black listing). Also Forensic Analysis tools are integrated within practical security assessments and thus used in Critical Infrastructures. For this reason, we believe that a penetration testing methodology should target different security deployments and evaluate their effectiveness. 3.3.3. Security tests The interviewed were asked about security tests already performed within critical infrastructures. In the questionnaire we focused on four different categories of tests: Stress tests and Denial-of-Service tests, penetration tests, security code reviews, and formal code verification. The experts confirm that such tests are quite known and widely used in critical infrastructures. Vendors usually focus their analysis on all the listed categories. Critical infrastructure operators almost exclusively focus on stress tests and penetration test. 62 SEVENTH FRAMEWORK PROGRAMME

3.3. Analysis We believe that a security assessment tuned on critical infrastructures should take into account all these tests. Moreover, a methodology should arrange such tests within a workflow and merge the results. The interviews revealed that most of the tests are performed during critical infrastructure development and setup. According to our results, only a small percentage of the interviewed considers a live system suitable for a penetration test. However, situations in which the system is off-line for maintenance seem suitable to run specific tests. The reason behind these considerations relies on the concern that a test may affect the proper functioning of critical infrastructures and endanger both physical processes, people, and company businesses. For example, SANDIA (a major United States Department of Energy research and development national laboratory) reported that a ping sweep performed during security testing of an industrial control system caused a robotic arm to dangerously swing [4]. This lead us to the following assumption: security assessments must not have an impact on live systems. Every action must be evaluated beforehand with respect to this risk. In any case, security operators have to be ready to face problematic or dangerous situations that can be caused by a penetration test. This assumption seems to disagree with the main purpose of any activity of security testing. The purpose of the discussed tests is to check the security of the system and show the potential impact of attacks against critical infrastructures. A security assessment methodology tuned on these systems has always to balance between the effectiveness of a test and the overall security that should be maintained in any case. We believe that this is a key point that pentesters must discuss with critical infrastructure operators before starting any activity on the system. 3.3.4. Pentesting Strategies During the interviews, we asked operators about the pentesting strategies already in place within critical infrastructures. Differently from other topics, the outcomes of these discussion show that there is no common agreement on the issue. First of all, critical infrastructures use both White-Box and Black-box penetration testing depending on the purpose of the test. Together, the two techniques contribute to perform a comprehensive assessment that focuses both on vulnerabilities and defense reaction capabilities. From time to time, some operators can exclude one or the other for different reasons. Time constraints is again an obstacle to a comprehensive security assessment. A black box pentesting can take more time as it involves an information gathering phase. However, a reason to choose black box pentesting is to focus on data leaks and operators misbehaviors that allow attackers to gain valuable information to attack the infrastruc- FP7-SEC-285477-CRISALIS 63

3. Requirements ture. Another reason to choose black-box over white box pentesting is the potential impact black-box has on the infrastructure. Without knowing anything about control processes, attackers can try to shut down key components and put the infrastructure in danger. A white-box pentesting could instead convince pentesters to evaluate possible vulnerabilities on such devices without trying to exploit them. Remote access the control network is another main issue during pentesting. From our interviews we know that not all the operators are available to open their systems to Wide Area Networks (WAN) or to the Internet. However, some of the penetration tests on critical infrastructures were performed by passing through public networks. According to the operators that allowed such tests, this situation better simulate real attack scenarios. Working over a public communication channel put in any case the infrastructure under further risk to be exploited by malicious users. Finally, some operators agree on outsourcing security testing and giving access to the system to third parties. These tests are usually performed within company premises but the remote access can be a choice as described in the previous paragraph. In any case, outsourced tests do not substitute internal testing. All the interviewed confirm that a subset of the tests is always lead by the owners of the critical infrastructure and performed by the operators. Outsourced tests are often performed on specific components or subsystems but they are rarely comprehensive and, in any case, related to the overall critical infrastructure. 3.3.5. Penetration Testing Methodologies for Critical Infrastructures One of the main tasks of these interviews was to investigate the presence of any specific penetration testing methodology for critical infrastructures. The majority of the experts is not aware of any specific methodology tuned on critical infrastructure. However, some of the interviewed suggested that (non-public) customized methodologies for specific tests exist. These methodologies are confined to a small set of companies and infrastructures. From that, we can derive that no comprehensive security assessment methodology has been developed and published yet. If no methodology has been developed yet, experts and operators discussed several best practices and tools currently used to perform tests on critical infrastructures. Most of the tools are Off-the-Shelf tools and software adapted to industrial devices. Some of them are: Metasploit [12], Nessus [1], Achilles [13, 14], Mu Dynamics products [15] but also standard IT fuzzers and code checking tools are used in the CI context. We will discuss some of these tools in Section 5. However, in some cases, operators and vendors develop their own tools for specific purposes. At the end of the interview we evaluated how a new assessment methodology specifically tuned on critical infrastructure would be received by the community. Almost all 64 SEVENTH FRAMEWORK PROGRAMME

3.3. Analysis the interviewed agreed on the need for a standard critical infrastructure penetration testing methodology. However, the discussion raised two different directions about what this methodology has to provide to the community. On the one hand, half of the experts say that general guidelines methodologies are good, but there is no need for more specific methodologies. This means that a formal workflow that explains all the steps toward a comprehensive assessment is less valuable than best practices adaptable to different situations. On the other hand, some experts explicitly ask for a critical infrastructure-specific and comprehensive methodology that would make security testing more organized/repeatable/comparable. In this case the result would be a comprehensive documentation that describe and evaluate several different critical infrastructure deployments. Moreover, it specifies tests and mechanisms to exploit on a large set of industrial devices. We deepen this analysis by asking what such a methodology should provide to operators. Requests brought by operators were various and different among each other. Almost all the interviewed operators want the methodology to suggest specific software/- tools to be used. The experts also emphasize the need for a comprehensive evaluation capable to provide both specific tips and general guidelines about each testing aspect. It is worth noting that a lot of importance was given to the definition of a common outcome to the tests. A critical infrastructure methodology must explain how to document test results in order to have a common background useful for test comparisons. Experts stressed the need to assessing specific components with respect to a general evaluation of the infrastructure. Even if a general assessment has a higher value to measure the security of an infrastructure, this is probably too difficult to analyze and compare with respect to single analyses on the components (e.g. with SCADA infrastructures we talk about geographically distributed infrastructures with thousands of devices). It is worth noting that almost no one asks for very detailed information about each testing aspect. This is related to the difficulties that a detailed methodology will have to face with respect to already in place policies and procedures inside specific critical infrastructures. 3.3.6. Final Remarks Thanks to the questionnaire and especially to the feedback of the experts given by the interviews, we have a first insight on what operators and vendors want from a critical infrastructure assessment methodology. The discussed differences among the answers reflect a real world situation in which there is no one-size-fits-all solution. From system to system and from company to company specific needs lead to the development and the customization several different testing methods. In most of the cases we found common purposes and some inter-operable mechanisms that can be a suitable starting point to FP7-SEC-285477-CRISALIS 65

3. Requirements merge such different methodologies. The idea behind this deliverable is to collect all the different information we got and create a general methodology based on common elements among already existing methods. At the same time we want the new CRISALIS penetration testing methodology to be flexible enough and adapt to different scenarios and environments. In section 4 we describe our proposal of critical infrastructure security methodology. 66 SEVENTH FRAMEWORK PROGRAMME

4. CRISALIS CI Security Testing Methodology 4.1. Overview In this section we focus on formalizing and merging all the concepts described in the previous two sections. The output of this analysis will be a new methodology for a security assessment methodology for Critical Infrastructures. Our work had two different starting points. On one hand we have the inputs provided by the questionnaire and the interviews. This is a necessary contribution for different reasons. First of all, experts in the field (e.g. SCADA vendors, utilities, security consultants, etc. ) have provided their expertise that we are building on. Thanks to their knowledge we were able to identify problems and concerns linked to the application domain and constraints in an industrial environment. Moreover, they allow us tailor our methodology to their actual needs. The interviewees explained how assessments and penetration tests are performed today in CIs and what they consider is still missing in these evaluations. Our CRISALIS penetration testing methodology addresses these issues and provides a generalized approach capable of taking the best of specific assessment methods already used, unifying differences and overcoming limitations. On the other hand, we have examined a number of relevant security assessment methodologies adopted from standard IT. OSSTMM, NIST SP800-115 and ISSAF are well known and extensively used. However, we see some shortcomings. E.g., several of the steps may not always be needed in the Critical Infrastructure field. NESCOR methodology is instead focused on industrial systems (e.g. especially devices involved on energy grids) but lacks of a general point of view on critical infrastructure security testing. Our idea of a security assessment methodology for Critical Infrastructure is a process that adopts structure and terminology of IT standard methodologies and integrates NESCOR tests on industrial control system. Furthermore, we aim to answer to stakeholders requirements for a framework of critical infrastructure security assessment and pentesting. CRISALIS CI Security Testing Methodology describes a guide that leads security operators and pentesters through a process of evaluation of critical infrastructure security from requirement identification to practical tests and finally to issue reporting. Our work maps the needs identified by the experts on standard security methodolo- 67

4. CRISALIS CI Security Testing Methodology gies. We do not want to create a new standard from scratch but rethink the existing ones. The result is a lightwheight security assessment methodology involving a penetration testing methodology tuned to industry components and constraints. The use of a common schema to evaluate Critical Infrastructure will allow operators, vendors and security consultants to share information and assessments in an easier way and will make results more meaningful. A uniform approach will allow to easily spot major problems and threats inside Critical Infrastructure and will eventually contribute to a general improvement of the security of such systems. 4.2. Workflow 4.2.1. Main Workflow As introduced in the previous paragraph, we conceptually split our methodology in two major parts: ˆ General Assessment: it covers all the aspects of a security assessment. It involves three sub-phases: a Pre-Assessment in which a company identifies the goals of the assessment and solves any bureaucratic and legal issue regarding the following operations; the actual Practical Assessment in which auditors and pentesters perform practical tests on the assets; and the Post-Assessment phase in which the company evaluates the results and decides about countermeasures and information disclosure. ˆ Practical Assessment: it represents the intermediate part of the General Assessment. It focuses on practical tests performed on Critical Infrastructures. The Practical Assessment is also divided in three different sub-phases: the Preparation in which pentesters discuss technical details with system operators and prepare tools and strategies to be used during the test; the Testing where all the tests are performed; and the Analysis in which the results of the tests are documented and detailed. Figure 4.1 shows the methodology workflow. There are two different contact points between the two methodology s parts. The former links the Pre-Assessment phase to the Preparation one while the latter links the Analysis phase to the Post-Assessment one. In both the cases there is a substantial movement of information flowing from a phase to the other. We will denote these two activities as Information Flows one ( 1 ) and two ( 2 ). 68 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow Figure 4.1.: Critical Infrastructure Methodology Finally, there are two possible loops in the workflow. The first feedback is given by the Analysis phase. At the end of the practical assessment it is possible to restart it without passing to the Post-Assessment. This situation describes a case in which the requirements identified in the General Assessment do not match with the actual testing. The second loop concentrates on the Testing phase and will be described in 4.2.3. Before discussing each working phase in details, it is worth noting that the reason to separate the methodology in two different parts is twofold. The first reason concerns the usability of the methodology. Such schemes allows operators to perform one of the two parts independently. Let us imagine a situation in which the procedures described for the Practical Assessment are not compliant with company policies or they are simply not suitable for company s purposes. In this case, operators can exploit the General Assessment scheme to perform all the accessory operations and substitute the Practical Assessment with a different model in line with their expectations. Thanks to the proposed methodology they will have just to take care about the Information Flows and modify them properly. In the same way, operators can use just the Practical Assessment with the condition to provide the right input and to modify the output by taking into account requests provided by a different security assessment. FP7-SEC-285477-CRISALIS 69

4. CRISALIS CI Security Testing Methodology The second reason behind the proposed scheme regards the maintainability. Critical Infrastructures have changed a lot since the 70s and they are likely going to change even more in the future. Moreover, legislations and company policies also change over time. The possibility to substitute a part of the methodology without altering the rest is a definite advantage in terms of cost and time. In case of such modifications, operators will have to modify the Information Flows to link the new part to the rest of the methodology. 4.2.2. Workflow phases In what follows we describe the five phases and the two Information Flows more in details: ˆ Pre-Assessment: the Pre-Assessment is the starting point of our security evaluation methodology. In this phase the company opens the discussion about the security assessment and explains its motivations and its goals. This phase should involve at least three different actors: the Chief Security Officer (CSO) or whoever is representative for this function and may decide to approve the starting of assessment operations; the Chief Financial Officer (CFO) or whoever is representative for this function and may decide on the budget allocated for the assessment operations and about acceptable risks; and ICS and SCADA operators that will be responsible to contribute to the discussion with the technical point of view. The discussion deals with several topics. First of all, the actors have to decide about security assessment s purposes. The main goal and the motivations behind the following operations have to be stated clearly. What follows is the identification of company assets. The actors have to focus on one or more targets according to the proposed objectives. Critical Infrastructures are complex systems and, for this reason, it is unlikely to have always a comprehensive assessment. Most of the time, specific sections or processes of the Critical Infrastructure will be checked and analyzed. An additional and very important step is a detailed risk analysis that clearly identifies activities endangering the safety of the system and its environment. This needs to be documented as well as an agreement to either skip those steps or perform them with specific safety guards. During the Pre-Assessment, involved actors have to agree on the time plan and decide on how to distribute resources to the five phases of the methodology. Finally they have to investigate and ensure that legal and contract obligations regarding assessment operations are met. This last step overlaps with the Control Assessment proposed by ISSAF and already discussed in 2.3.1. 70 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow ˆ Information Flow 1 : at the end of the Pre-Assessment phase its participants have to prepare a document summarizing all the results of the discussion. The Information Flow describes the work the operators will do for translating such outcomes in practical directives for the pentesters. For example, the assets identified in the previous phase will become systems and components to be tested. In the same way, the main goal will be divided in several sub-goals which will be mapped into specific tests. Such information will be the input of the Preparation phase. ˆ Preparation: the Preparation is the first phase of the Practical Assessment. This phase involves two actors: ICS and SCADA operators that report the outcomes of the Pre-Assessment phase; and the pentesters which are in charge of performing necessary tests. The two actors can can be from the same organization if the purposes or the constraints of the security assessment respectively allow or do not permit the presence of third parties (e.g. external companies, trained pentesters, etc.). Operators and pentesters have to discuss several issues. First of all, they will decide on what kind of pentesting schema to follow. They will decide on the type of penetration testing as proposed by OSSTMM in 2.1.2 (e.g. Blind testing, Grey testing, etc.). Moreover, they will choose on the timing of the attack (e.g. working hours, weekends, etc.). Finally, they will decide whether to attack the live system, or a backup, or performing all the tests when the system is off-line. The attack impact is one of the main concerns that has to be faced during the Preparation. Critical Infrastructures cannot be put in danger by a penetration testing even if the reason to perform the analysis is verifying that possibility. During this phase operators should decide and implement specific safeguards to put in place during tests and prepare an emergency plan with countermeasure to take in case one of the security tests succeed in penetrating and corrupting the system. This plan has to be ready also in case of defensive black box testing. In that situation operators participating to the test will not be aware of the plan unless a problem occurs. In this phase operators have to verify if all the objectives defined during the Pre-Assessment are achievable. Otherwise they should motivate any problem that represents an obstacle to the test. The final step of this phase is the setup of the environment and of all the resources needed to the pentesting activity. ˆ Testing: the Testing phase implements the security tests that have been discussed during the Preparation. Its duration depends on different factors such as: accuracy of the tests (e.g. comprehensive tests vs. vulnerability identification), pentester skills, allocated budget, etc. During the entire phase pentesters have to log all FP7-SEC-285477-CRISALIS 71

4. CRISALIS CI Security Testing Methodology the performed operations and related discoveries. Such report will be the input of the Analysis phase. Security operators responsible for the emergency plan will follow the operations from a different point of view. They will check the presence of penetration side effects on the Critical Infrastructure that the pentesters are not aware off. Such information will be integrated to the pentest log at the end of the Testing phase. It is worth noting that the testing plan arranged in the Preparation phase can change accordingly to the evolution of the tests. Unexpected obstacles can interfere with the penetration test and undermine the feasibility of some operations. Such situations have to be also detailed in the log. In case a test shows severe vulnerabilities that can endanger the Critical Infrastructure, the safeguards and emergency plan should be triggered and prevent real damage. The effect of such an event is the interruption of the methodology workflow and the immediate feedback to the Chief Security Officer. The level of severity needed to trigger the emergency plan has to be identified in the Preparation phase. The testing phase will be extensively discussed in section 4.2.3. ˆ Analysis: the Analysis phase concludes the Practical Assessment. As in the Preparation phase, pentesters and operators are the two actors involved. The former will report about the findings of the Testing phase and will propose a risk rating of the security to the Critical Infrastructure. The latter will critically review the testing, contextualizing the problems and justifying any design or maintenance choices. Moreover, operators have to perform two important tasks: check if the status of the system has returned to normal after the testing session; and apply fixes to minor and easy-to-fix problems and vulnerabilities. At the end of the Testing phase, the pentesters will have to undo permanent modifications to the system, like removing any file or software installed inside the Critical Infrastructure or unpluging any machine used for pentesting. During the Analysis, operators will check the correctness of such activity and will run some integrity checks to verify that the Infrastructure is brought back to a safe state. It is probable that problems such as misconfigurations and other simple activities from the penetration induce some errors into the system. Operators should fix such issues without reporting them to the next stage of the methodology. In this sense, the Analysis phase is also a filter to identify the more severe security problems and what types of problems a later security deployment may create. ˆ Information Flow 2 : at the end of the Analysis phase operators and pentesters will prepare a document that, starting from the report of the Testing phase, summarizes findings and security issues. The format of the documentation can be based 72 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow on the the ones proposed by ISSAF in its Final Reporting 2.3.2 and partially also on the final requirements of the OSSTMM 2.1.2. Some information needed to the document describing the Analysis phase are: Date and time of test Duration of test Auditors and analyst involved Tools that have been used Test type Scope of test Comprehensive list of performed tests Knowledge of which tests have been completed, not completed, or only partially completed, and to what extent Test error margins Vulnerability discovered/exploited with recommendations on how to solve problems found Unknown anomalies Operators will have to motivate and justify such documentation during the Post- Assessment phase. ˆ Post-Assessment: the Post-Assessment closes our security assessment methodology. With respect to the Pre-Assessment phase the presence of the Chief Financial Officer is no longer needed as any decision regarding new operations will be taken afterwards. Therefore, in this phase, the discussion involves ICS and SCADA operators and the Chief Security Officer or whoever represents this role in the company. Operators will explain the steps performed during the Practical Assessment. Pentester can also participate to the meeting. Their task will be to present their findings focusing on major problems encountered. Operators will also propose countermeasures agreed on the basis of the discussion in the Analysis phase with the pentesters. The CSO will evaluate the proposed countermeasures deciding on what actions to take to solve the most pressing issues. It is worth noting that, during the Post-Assessment phase, participants will also discuss the possible impact of vulnerability disclosures. The output of this phase is the final result of the security assessment methodology. Such documentation is both an high-level overview about company s vulnerabilities, possible mitigations, limitations in deploying security and relation to security legislation or policies on the FP7-SEC-285477-CRISALIS 73

4. CRISALIS CI Security Testing Methodology one hand and on the other hand a low-level overview describing specific problems and vulnerabilities. The conclusions of the assessment match the list proposed by the NIST in 2.2.2 (Insufficient patch management, Lack of security baselines, Inadequate incident response procedures, etc.). The output of the Post-Assessment methodology can be used for following security assessments (e.g. to identify solved and still in place problems and vulnerabilities) or as a starting point for different kinds of assessment (e.g. risk assessment introduced in CRISALIS deliverable 2.2). Figure 4.2 shows a summary of the concepts discussed above. We will now discuss the penetration testing in more detail. Figure 4.2.: Critical Infrastructure Methodology Detailed Schema 4.2.3. Penetration Testing Steps The penetration testing process is the core of the assessment methodology. During the Testing phase, pentesters go through five operational steps: Information Gathering, Architecture Analysis, Vulnerability Identification, Penetration, and Maintaining Control. 74 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow Each step groups a set of operations necessary for the next step. Our methodology foresees four subsequent testing processes. These processes follows the four main tasks of the NESCOR 2.4.1 methodology: Server OS Penetration, Server Application Penetration, Network Communication Penetration, Embedded Device Penetration 2.8. Furthermore, we envision a circular representation of the testing steps as each penetration testing process is not necessarily one-time. A successful attack and the consequent control of a part of the system by the pentesters could open new attack vectors. In this case, pentesters can traverse again through the five pentesting steps exploiting new targets or attack vectors. The process ends when all the prepared attack scenarios as well as those discovered during the activity have been tried and evaluated. The specifications of the five penetration testing steps are based on the schema proposed by ISSAF in 2.3.2 and detailed with the sub-tasks listed in the NESCOR methodology. In what follows we describe each step in detail: ˆ Information Gathering: the Information Gathering step focuses on systems for collecting and analyzing information regarding Critical Infrastructure components. This information can concern different elements such as used hardware and software but also more general data about working personnel and company policies. The Information Gathering involves both technical and non-technical methods. Depending on the type of testing (Black Box vs. White Box), pentesters can have some initial information at their disposal that helps them to understand the environment they are going to attack. It is unlikely to find useful information about the Critical Infrastructure internal functioning on the Internet. However, the knowledge of the working staff and their skills and backgrounds can give some ideas on what pentesters will have to face once the attack has started. In case of a White Box penetration testing the Information Gathering is important to quickly identify points that will be most likely vulnerable and focus on them. ˆ Architecture Analysis: the Architecture Analysis step directly follows the Information Gathering and contributes to acquire more information about Critical Infrastructure components. In this case, pentesters concentrate only on hardware and software. The main target is to identify specification of devices working inside the Critical Infrastructure, identify communication patterns, etc.. Different tools can be used during this activities (e.g., sniffers, fingerprinters, disassemblers, etc.). Some of these tools can automate the the work of this step and thus save time and costs. Depending on the four testing processes, during this step pentesters will perform the following operations: Server Applications: FP7-SEC-285477-CRISALIS 75

4. CRISALIS CI Security Testing Methodology * Application and Platform Fingerprinting: to identify devices running services * Functional Analysis: to explore functionalities available to allowed users * Process Flow Modeling: to understand application operation flows * Request/Resource Mapping: to record application requests and responses Server OS * DNS Interrogation: to get information on DNS servers and zones * Port Scanning: to identify hosts on the networks * SNMP Enumeration: to further identify network configurations * Packet Sniffing: to capture communications Network Communication * RF Signal Capture: to capture RF communications * Network Protocol Traffic Capture: to capture communications between IP network devices Embedded Device * Circuit Analysis: to identify electronic main components and functionalities * Datasheet Analysis: to gain information about device roles in the control process * Interface Functional Analysis: to identify possible connections to device interfaces * Firmware Binary Disassembly: to retrieve firmware binary instructions The Architectural Analysis step will allow pentesters to confirm or discard attack hypotheses formulated during the Preparation phase or the Information Gathering step. ˆ Vulnerability Identification: once one or more targets are selected, pentesters will perform a vulnerability scanning. The most commonly used tools for vulnerability identification in Critical Infrastructure are Achilles [13] [14] and Nessus [1]. The list of the activities of this step comprises: Server Applications: 76 SEVENTH FRAMEWORK PROGRAMME

4.2. Workflow * Default Configuration Testing: to test if applications still run default configurations * Authentication, Session and Authorization Testing: to test if applications use default credentials * Code Injection Testing : to test data flaws in XSS, SQL, etc. * Denial of Service Testing: to test for flaws that may cause Denial of Service attacks Server OS * Unauthenticated and Authenticated Vulnerability Scanning: to identify known vulnerabilities on network devices * Vulnerability Validation: to manually validate known vulnerability * Packet Capture Analysis: to look for protocols with known vulnerabilities Network Communication * Spread Spectrum Recovery: to retrieve the information signal * Signal Demodulation: to demodulate the signal * Cryptographic Analysis: to determine if any known vulnerabilities on the involved cryptographic algorithms exist Embedded Device * Dumping Data at Rest: to dump device s internal data * Bus Snooping: to dump information flowing in the bus * String Analysis: to identify byte patterns (e.g. human readable strings) * Entropy Analysis: to differentiate between encrypted and unencrypted data * Field Technician Interface Communications Capture and Analysis: to intercept normal communications on the interface and identify weaknesses (e.g. authentication, integrity checks, etc.) * Firmware Binary Code Analysis: to identify weaknesses in memory use or cryptographic functions ˆ Penetration: during the Penetration step, pentesters try to gain access and obtain control of Critical Infrastructure s components. This is the most dangerous operation of the Testing phase as the attackers try to subvert normal operations FP7-SEC-285477-CRISALIS 77

4. CRISALIS CI Security Testing Methodology of the system or a part of it. The Metasploit Framework [12] is one of the most widely adopted tool to exploit software vulnerabilities. In its professional version, the framework also provides several exploits against Programmable Logic Controllers (PLC). However, most of the times, pentesters will resort to develop customized scripts and tools to exploit ICS-specific vulnerabilities. This is the case of the fuzzers developed within the CRISALIS project and presented in deliverable 5.2. What follows is a list of operations performed by pentesters during the Penetration step: Server Applications: * Application Vulnerability Exploitation: to test proof of concept attacks based on applications analyzed in the previous step Server OS * Device Vulnerability Exploitation: to test proof of concept attacks based on the operating systems identified in the previous step Network Communication * Network Traffic Extraction: to decode and extract communication payloads from RF capture * RF Signal Transmission: to transmit RF signals and wait for responses * Unknown Protocol Decoding: to reverse engineer the network captures and retrieve valuable information * Network Protocol Fuzzing: to send either valid or invalid data to identify device responses * Network Protocol Exploitation: to test feasible attack on the network (e.g. authentication requests) Embedded Device * Systematic Key Search: to identify cryptographic keys * Decoding Retrieved Data: to reverse engineering information to understand its content * Embedded Hardware Exploitation: to determine feasible attacks against embedded components * Field Technician Interface Endpoint Impersonation: to attempt to impersonate device interfaces 78 SEVENTH FRAMEWORK PROGRAMME

4.3. Final Remarks * Field Technician Interface Fuzzing: to send either valid or invalid data to device interfaces * Field Technician Interface Exploitation: to determine feasible attacks against field technician interfaces ˆ Maintaining Control: once in control of a machine or a service, attackers will check every possible information that can be retrieved from the machine and if they can use the compromised computer to find new targets on the network. Pentesters will search for valuable data such as: Passwords Running processes Information regarding the physical process under control Generic contacts (e.g. emails) If the compromised machine is at the edge of the network it may also link to different networks. Pentesters have to evaluate this possibility and, if this is the case, restart the pentesting process against the new target. Attackers have always to verify the possibility to maintain access to the machine for future activities by installing malicious code (e.g., root-kits and backdoors). This guarantees to maintain control even if the exploited vulnerability is patched. Whether this should be conducted needs to be decided in the planing phase. Finally, pentesters have to cover tracks of the intrusion. Such operations are usually performed: Hiding modified files and tools Cleaning logs Restoring integrity checks At the end of this step the Testing restarts on a new target or ends up letting pentesters proceed with the Analysis phase. In this phase all the outcomes will be merged together to have a final and comprehensive overview of found vulnerabilities. Figure 4.3 proposes a comprehensive schema of the assessment methodology plus the chain of the five penetration testing steps. 4.3. Final Remarks The presented CRISALIS penetration testing approach represent a first draft towards a security assessment methodology for Critical Infrastructures. To the best of our knowl- FP7-SEC-285477-CRISALIS 79

4. CRISALIS CI Security Testing Methodology Figure 4.3.: Critical Infrastructure Methodology plus Penetration Testing Steps 80 SEVENTH FRAMEWORK PROGRAMME

4.3. Final Remarks edge, this work is the first attempt to formalize Critical Infrastructure assessments and pentestings. With our proposal we tried to incorporate requests and advices given by Critical Infrastructure experts with existing standard methodologies. The result is a light-weight but comprehensive list of working phases linked by specific information and documentation exchanges that should lead pentesters through a fast, cost-saving and effective evaluation of a system. Our methodology differentiates among two logical parts. The first, called General Assessment, concentrates on the administrative process that a company must follow to organize a security assessment. The second, called Practical Assessment, regards the technical preparation of the evaluation process and the consequent analysis. Finally, the process shown in 4.2.3 describes the actual penetration testing. In this phase we discussed the steps pentesters have to follow to check and evaluate a Critical Infrastructure. As described in the introduction of this chapter, the use of separated but connected logical components improve both usability and maintainability of the methodology. We have conducted some initial penetration testing experiments in one of the CRISALIS testbeds where one new vulnerability was found. These tests also helped to shape and refine this methodology as we were able to identify importance and feasibility of different steps. It also highlighted that there are insufficient structures to report newly found vulnerabilities of incidents to manufacturers and users. FP7-SEC-285477-CRISALIS 81

5. Tools In this section, we discuss various tools available for security testing in critical infrastructures. While CI systems also include generic IT systems like desktop computers or laptops running, e.g., Windows or Linux, we do not discuss tools for generic security testing, as there is a broad body of literature and web resources on this. Instead, our discussion focuses on specific tools for CI like industrial control systems or advanced metering infrastructures. Where generic tools like metasploit also include CI-specific components, they are of course included in this discussion. 5.1. Kali Linux Kali Linux 1 is a Linux distribution specialized for security testing. It brings along myriads of pre-installed tools for general security testing. While no specialized SCADA/IC- S/AMI security tools are contained, it provides a good starting base for installing some of the other tools described herein, as it brings a lot of useful libraries etc. 5.2. Metasploit Framework The Metasploit Framework [12] is one of the most known and widely-used penetration testing tools in the information security community. It is the most famous branch of the Metasploit Project, started in 2003, with the goal to share information and knowledge about information security and to develop penetration testing techniques and supports. The main author of the tool is H. D. Moore who release the first version of the framework in 2003. On 2007, Moore decided to leave Perl and completely rewrite the tool in Ruby. By 2009, the security company Rapid7 had acquired the whole Metasploit Project. The Metasploit Framework is considered a de facto standard for exploit development frameworks. Security experts and developers can use its open source platform to test a wide range of infrastructures and to write new exploits customized to every specific target and test. The extensible model provides to the user a comprehensive database of 1 http://http://www.kali.org/ 82

5.2. Metasploit Framework exploits, payloads, encoders, no-op generators and several other auxiliary codes. Last free release has been published with more that 1500 exploits for different architectures and software. The Express edition of the framework, besides having more features, provides also ICS specific pentesting tools. Pentesters and security experts can use the Metasploit Framework in several ways. However, the exploitation of a known vulnerability on a system is the most common operation performed with the framework. There are four basic steps to follow to achieve this goal: ˆ Exploit choice and configuration: that defines the kind of attack users want to perform on target hardware and software ˆ Payload configuration: that defines which code will be executed on the attacked machine if the exploit works ˆ Encoding configuration: that avoids intrusion prevention and intrusion detection systems to block the attack and the execution of the payload ˆ Exploit execution: that starts the attack process and creates the communication channel with the victim machine The modular approach allows the user to combine exploits and payloads and thus to customize the attack in many different ways. The Metasploit framework provides multiple interfaces: ˆ Command line: light and effective interface provided with the standard edition of the framework ˆ Web-based user interface: provided with the Metasploit Community Edition ˆ Graphical user interface: provided from the Metasploit Express version Moreover, there are some tools providing improved graphical features to the framework. This is the case of Armitage (a management tool that visualize targets on the network and advice users on the use of different exploits) [16] and Cobalt Strike (a collection of threat emulation tools) [17]. FP7-SEC-285477-CRISALIS 83

5. Tools 5.3. Nessus Vulnerability Scanner Nessus is probably the most popular and comprehensive vulnerability scanner on the market. The tool was developed by Renaud Deraison from 1998 to 2005 when Tenable Network Security took charge of the project. In the first years the tool was distributed under the Gnu Public License (GPL) while today is a proprietary software (even if some of its components are still free of charge for standard users). Nessus is a multi-platform tool with the only difference that on UNIX systems the execution is divided by the Nessus daemon (nessusd) which does the scanning, and the client which controls scans and presents the vulnerability results to the user. Tool s operations rely on the description of vulnerability exploits written in NASL (Nessus Attack Scripting Language) and run by the Nessus engine. In a typical operational phase the tool starts with a port scan to identify open ports and running services on devices working on the network. The scan can be performed by one of Nessus s scan services or using well-known port scanners such as Nmap [18]. Once the scan ends the tool starts checking devices for known vulnerabilities. Finally, it provides a report with all the findings and best practice to solve major security issues. Nessus does checks for four different categories of security issues : ˆ Software Vulnerabilities: this group of vulnerabilities include both TCP/IP stack implementation issues and software vulnerabilities. The formers usually allow attackers to perform Denial of Service attacks by using mangled network packets. The latters rely on code bugs and allow attackers to access sensitive data or even remotely control devices. ˆ Misconfigurations: this issue relates to mistakes in the security configuration of a system. Nessus can check parameters operators set up and provide a feedback on their effectiveness. ˆ Default Passwords: Nessus checks the strength of passwords used in the system. The tool tests system accounts with a few common passwords and blank passwords. Moreover, the check can be enlarged by the embedded use of Hydra [19]. ˆ PCI DSS Related Vulnerability: this issue relates to vulnerability tests of compliance for the Payment Card Industry Data Security Standard. For all these tests Nessus can work either in full or safe checks mode. The safe checks mode allows a user to check just for vulnerabilities that does not cause a service or an operating system to crash. We believe, this feature is valuable especially in the contest of critical infrastructures. 84 SEVENTH FRAMEWORK PROGRAMME

5.4. Achilles Test Platform A Nessus vulnerability scan report can contain multiple chapters: ˆ Hosts Summary ˆ Vulnerabilities by Host ˆ Vulnerabilities by Plugin The first chapter shows result of the network scan and list all target hosts on the network, their services and open ports. The other two chapters present a detailed list of vulnerabilities (colored by severity) and the related suggested remediations with the actions to take that address the described vulnerabilities. A Nessus vulnerability scan report can be delivered in these formats: HTML (default), PDF, CSV. The commercial version of nessus distributed by Tenable contains a number of specific SCADA/ICS plugins that test a variety of devices for vulnerabilities 2. Created in cooperation with DigitalBond and their Bandolier project, those plugins cover SCADA/DCS servers from a number of vendors, including ABB and Siemens to check for secure configuration and known vulnerabilities. http://www.tenable.com/plugins/index.php? view=all&family=scada gives an overview on available plugins. 5.4. Achilles Test Platform The Achilles Test Platform and Achilles Test Software are two products of wurldtech 3 for robustness testing, fuzzing, and security testing of IP-based protocols in CI. While the test platform is a complete hardware appliance that also includes aliveness and performance monitoring of the target system, the test software is VM-based and more easy to deploy. Protocols like DNP3, MMS, or Modbus can both be fuzzed using a custom grammar and be subjected to overload attacks. Specific signatures for known attacks are also available. 5.5. SamuraiSTFU SamuraiSTFU is a pen test distribution used to perform penetration testing on ICS environments developed by UtiliSec. This distribution comes by the same authors of 2 http://www.tenable.com/solutions/scada-security 3 http://www.wurldtech.com/ FP7-SEC-285477-CRISALIS 85

5. Tools SamuraiWTF. Unlike this last (and other well-known distributions used within IT environments) SamuraiSTFU focuses on the differences between ICS and standard networks and focuses its tools on industrial devices needs. Some key points of SamuraiSTFU are: ˆ a collection of free and open source tools for all aspects of SCADA and Smart Grid pentesting. This encompasses: a selection of web penteting tools of SamuraiWTF a selection of network pentesting tools of Backtrack a set of hardware pentesting tool not currently included on any other distribution ˆ a comprehensive documentation on SCADA and Smart Grid architectures and protocols. ˆ a collection of pcap samples captured over real environment networks ˆ tools for Smart Grid environments simulations SamuraiSTFU aims to be the pentesting distribution for industrial security experts. Main audiences are utilities and vendors in the energy sector. Secondary audience are utilities from other sectors (e.g. water, gas, etc.). 5.6. SCADA Strangelove Tools SCADA Strangelove 4 is a group of Russian hackers that specializes in SCADA security. Besides running a blog dedicated to SCADA security, they also provide a number of tools for general download. These include network scanners for Profinet, password crackers for Siemens S7 1200 and 1500 systems that can be used for offline password cracking using eavesdropped authentication, a metasploit module called WinCC Harvester, and other similar tools. WinCC Harvester is a Metasploit module for Siemens SIMATIC WinCC forensic/postexploitation that uses WinCC MS SQL access to harvest sensitive information (users, roles, PLCs) from the WinCC database. PLCscan is a tool to scan for PLC devices via s7comm or modbus protocols. The group also provides useful documentation, like a WinCC 7.x hardening guide or a ICS/SCADA/PLC Google/Shodan Cheat Sheet that can be used to find target systems 4 http://www.scadasl.org/ 86 SEVENTH FRAMEWORK PROGRAMME

5.7. Others Ressources for specific protocols using Google or Shodan as well as documentation on a number of security vulnerabilities. Their work is focusing mostly on Siemens systems, although a few other vendors show up on their list. Security testers can mainly use their tools for discovery and to some extend for proof of concept exploitation. 5.7. Others Ressources The website scadahackers.com 5 provides a regularly updated list of SCADA-related security testing tools. Project Robus http://www.automatak.com/robus/ is announcing a fuzzer for DNP3 that should have be provided in March 2014 but is still not openly available. 5 http://scadahacker.com/tools.html FP7-SEC-285477-CRISALIS 87

6. Vulnerability Disclosure Software vulnerabilities are weaknesses in software that can be exploited by attackers. Depending of the type and severity of the vulnerability and on the environment where the affected component is deployed in, attackers might be able to exploit those to compromise either confidentiality, integrity or availability of systems. So, regarded from a high level view, vulnerabilities should be fixed as soon as possible in affected software. Vendors usually call activities involved in fixing software after a vulnerability report as vulnerability handling. This process covers reactive activities such as remediating and mitigating vulnerabilities, as well as informing possible stakeholders about the problem. During this process, incoming vulnerability reports are validated by internal teams, possible fixes for the vulnerability are discussed and implemented in the affected software. As product quality must always be ensured, rigorous testing is performed after development of the software fix. In total, this process may cover from days to months. In the time between vulnerability report to the ready-to-use software fix, customers using the affected product may be vulnerable to the issue. This timeframe where users are at risk should be as short as possible. So in the context of vulnerability handling, a favored practice for vulnerability reporting is called coordinated disclosure. This is the procedure where a vulnerability is reported discreetly to the vendor by the discoverer. Within a certain timeframe, the vendor can fix the vulnerability in the software and publish a software update. After that, both vendor and discoverer may report the vulnerability to the public including documentation of the measures taken. This procedure minimizes the risk for users, as information about the vulnerability will only be publically disclosed when the user has the possibility to apply a fix to the product. Usually multiple parties and stakeholders are involved in vulnerability handling. Figure 6.1 gives an impression about those based on a potential vulnerability handling scenario: A security researcher detects a vulnerability. This researcher may be a penetration tester that performs contracted security assessments and discovers vulnerabilities in products during these assessments. In many cases, also independent security researchers find vulnerabilities. If no contact point at the vendor for reporting vulnerabilities is known to the researcher, he may notify a CERT (Computer Emergency Response Team) within the CERT network about it. CERTs are responsible for the security of IT infrastructures. National CERTs are responsible for whole countries (CERT Bund for Germany, US-CERT for the United States, etc.) or for specific parts like ICS- 88

Security Researcher 1. Report CERT Network 2. Report 3. Confirmation, Coordination Vendor 4. Vulnerability Handling 5. Publication General Public / Media / Press / Internet 5. Publication Customers 5. Publication Figure 6.1.: Vulnerability stakeholders with exemplary vulnerability information flow CERT (responsible for Industrial Control Systems). So the CERT would investigate and inform the affected vendor about the vulnerability report. At the vendor, the process for vulnerability handling will be started then. During this time, the progress of the handling would be coordinated with the involved CERTs. After a software update was successfully produced, both CERTs and the vendor would inform customers, and the general public if necessary. FP7-SEC-285477-CRISALIS 89

7. Conclusion In this deliverable, we have motivated the need and benefits for a methodology for security penetration testing for Critical Infrastructures and Industrial Control Systems. Benefits include time and cost savings for penetration testers, easier comparison of results, and better confidence in completeness and correctness of results. We analyzed a series of existing methodologies regarding their applicability in the CRISALIS context. Based also on the feedback from an online survey and interviews we have conducted, we identified requirements and the need to provide an adapted and revised approach. We propose the CRISALIS security testing methodology that provides a framework for security testers to work in. The discussion also includes pointers to various issues that arise in CI security testing and that need to be considered by testers. Additionally, this deliverable provided links to existing tools for SCADA/ICS/AMI testing and also provides a discussion about vulnerability disclosure. 90

Bibliography [1] R. Deraison, H. Meer, and C. Van Der Walt. Nessus network auditing. Syngress Media Incorporated, 2004. [2] R. Patton. Software testing. Sams, 2005. [3] M. Prandini and M. Ramilli. Towards a practical and effective security testing methodology. In Computers and Communications (ISCC), 2010 IEEE Symposium on, pages 320 325. IEEE, 2010. [4] D.P. Duggan, M. Berg, J. Dillinger, and J. Stamp. Penetration testing of industrial control systems. Sandia National Laboratories, 2005. [5] P Herzog. Osstmm 3 the open source security testing methodology manual. barcelona, españa: Isecom, 2010. [6] Karen Scarfone, Murugiah Souppaya, Amanda Cody, and Angela Orebaugh. Nist special publication 800-115: Technical guide to information security testing and assessment, 2008. [7] Creighton T Hager and Scott F MidKiff. An analysis of bluetooth security vulnerabilities. In Wireless Communications and Networking, 2003. WCNC 2003. 2003 IEEE, volume 3, pages 1825 1831. IEEE, 2003. [8] Creighton T Hager and Scott F Midkiff. Demonstrating vulnerabilities in bluetooth security. In Global Telecommunications Conference, 2003. GLOBECOM 03. IEEE, volume 3, pages 1420 1424. IEEE, 2003. [9] Balwant Rathore, Mark Brunner, Miguel Dilaj, Omar Herrera, Piero Brunati, Rama K Subramaniam, Subash Raman, and Umesh Chavan. Issaf 0.2.1 - information systems security assessment framework, 2006. [10] Justin Searle. Nescor version 3 - guide to penetration testing for electric utilities, 2012. 91

Bibliography [11] Marco Caselli and Frank Kargl. Critical infrastructure security & penetration testing questionnaire. https://docs.google.com/spreadsheet/viewform?formkey= dg15cuv5ck9hwlg1etvuvk51znnyb1e6mq/, 2012. [12] LLC Metasploit. The metasploit framework. www.metasploit.com, 2007. [13] Achilles test platform. http://www.wurldtech.com/product_services/ discover_analyze/achilles_test_platform/. [14] Achilles test software. http://www.wurldtech.com/product_services/ discover_analyze/achilles_test_software/. [15] Mu dynamics software. http://www.mudynamics.com/products/. [16] Armitage - cyber attack management for metasploit. http://www. fastandeasyhacking.com/, 2010. [17] Cobalt strike - advanced threat tactics for penetration testers. http://www. advancedpentest.com/, 2012. [18] Gordon Fyodor Lyon. Nmap Network Scanning: The Official Nmap Project Guide to Network Discovery and Security Scanning. [19] van Houser. Hydra. https://www.thc.org/thc-hydra/. 92 SEVENTH FRAMEWORK PROGRAMME