WHITE PAPER: ENTERPRISE SECURITY

WHITE PAPER: ENTERPRISE SECURITY Security Incidents and Trends in the SCADA and Process Industries A statistical review of the Industrial Security Incident Database (ISID) Prepared by: Eric Byres David Leversage Nate Kube

White Paper: Enterprise Security Security Incidents and Trends in the SCADA and Process Industries Contents Executive summary..................................................................4 Introduction........................................................................5 Background and motivation..........................................................6 The Industrial Security Incident Database..............................................7 The changing landscape..............................................................9 A deceiving trend...................................................................11 Estimating total incidents in the SCADA and control industries.............................12 Summary of incident trend rates......................................................12 A changing threat..................................................................13 Inside to outside....................................................................13 A nastier threat.....................................................................15 The back door into the control system................................................17 The impact of cyber incidents.......................................................20 Financial impact....................................................................20 Operational impact.................................................................21 An alternative Cyber-Threat Impact Index...............................................22 Improving the security of industrial control systems...................................23 The need for comprehensive security programs..........................................24 The need for defense in depth........................................................26 Patch and antivirus management in control systems.....................................27 Layered protection for control devices.................................................28 Summary.........................................................................29

Executive summary Supervisory Control and Data Acquisition (SCADA) and industrial control systems, with their traditional reliance on proprietary networks and hardware, have long been considered immune to the cyberattacks that have wreaked so much havoc on corporate information systems. Unfortunately, both academic research and in-the-field experience indicate that this complacency is misplaced the move to open standards such as Ethernet, TCP/IP, and Web technologies is letting hackers and virus writers take advantage of the control industry s ignorance. The result is a growing number of unpublicized cyber-based security events that are impacting our critical national infrastructures and manufacturing industries. In support of establishing the case for improving control system security, this report summarizes the incident information collected in the Industrial Security Incident Database (ISID). It describes a number of events that directly impacted process control systems and shows that the number of cyber incidents against SCADA and control systems worldwide has increased significantly since 2001. The majority of these incidents are coming from the Internet by way of opportunistic viruses, Trojan horses, and worms, but a surprisingly large number are directed acts of sabotage. In addition, the analysis indicates that many SCADA/process control networks (PCN) have poorly documented points of entry that provide secondary pathways into the system. The analysis highlights several key areas where the average SCADA/PCN system could be better secured. First, the root cause of many of the incidents reported in the ISID is a breakdown in human factors rather than a failure of technology. Thus, it is critical that SCADA owners and operators begin to develop comprehensive control system security management programs that cover all aspects of industrial system security, including cyber and physical security. The Symantec white paper, Effective Practices for Meeting NERC Critical Infrastructure Protection Requirements in the Electric Power Industry, provides guidance on how to create such a security program system by following a simple and holistic five-stage strategy, avoiding the common piecemeal approach to security. Second, the existence of secondary pathways highlights the need for an in-depth defense strategy. This includes better layering of firewall defenses and the hardening of endpoint devices through patch management, antivirus deployment, and distributed security appliances within the SCADA/PCN proper. In summary, the ISID data indicates that organizations that operate SCADA and control systems have good reason to be concerned about cybersecurity. Not only have the number of incidents increased dramatically in the past five years, but the seriousness of these events appears to be growing as well. Furthermore, the cost of an incident can be substantial. Even if 4

there is no direct impact on production or revenue, there is cost associated with expenditure of employee time, upgrading/changing of equipment, and the risk to corporate reputation. Failure to adapt to the changing landscape of security threats and vulnerabilities will leave the industrial controls world exposed to increasing numbers of cyber incidents. The result could easily be loss of reputation, environmental impact, production and financial loss, and even human injury. Introduction This report presents evidence in support of a business case for improving security of Supervisory Control and Data Acquisition (SCADA), 1 process control networks (PCN), and manufacturing and industrial automation systems. It is based on an analysis of statistical trends in security incidents as recorded in the Industrial Security Incident Database (ISID), a collection of known cybersecurity events that have occurred against control systems in the manufacturing and critical infrastructure industries. The report is divided into the following sections: Introduction Background and motivation: Summarizes the general changes in the SCADA/process control industries and technologies that have been seen over the last decade. The Industrial Security Incident Database: Provides a background on the ISID and discusses the organization of the data it contains. The changing landscape: Illuminates current incident trends in incident rates via data gathered from ISID. The changing threat: Looks at the driving forces behind many of the incidents recorded in ISID. The back door into the control system: Discusses the pathways that external attacks are employing to reach the SCADA system and plant floor. The impact of cyber incidents: Discusses the impacts of security incidents on the companies operating SCADA and control systems. Conclusions and recommendations for improving the security of industrial control systems: A discussion of possible means of improving the state of SCADA/PCN security. 1 The term SCADA is used in this paper to represent any industrial control system, including Distributed Control Systems (DCS), Programmable Logic Controllers (PLC), Remote Terminal Units (RTU), Emergency Shutdown (ESD) systems, and safety control systems. 5

Background and motivation Historically, the industrial control and SCADA systems that are responsible for monitoring and controlling our critical infrastructures and manufacturing processes have operated in isolated environments. These control systems and devices communicated with each other almost exclusively and rarely shared information with systems outside their environment. These SCADA systems were typically composed of proprietary hardware and software components designed specifically for control operations. Knowledge of the applications and protocols was limited to those few people directly involved in system design or operation and was not readily available to the general public. Thus the security of critical control systems was thought to be a trivial problem that could be managed through the use of either traditional IT security efforts or internal safety processes. These beliefs were exemplified in articles such as Debunking the Threat to Water Utilities, which made claims such as Most public utilities rely on a highly customized SCADA system. No two are the same, so hacking them requires specific knowledge. 2 Today, with the vast expansion of business communications systems and the move towards having access to real-time business information from any location, these previously stand-alone control systems are now being connected to the outside world. Business efficiency needs are dictating that production information be readily accessible to business managers, sales staff, engineers, maintenance personnel, and others who are not actually on the plant floor. While rarely directly connected to the Internet, studies show that in a typical corporation, 80 to 90 percent of all control networks are now connected to the enterprise network, 3 which in turn is interconnected to the Internet in myriad ways. Furthermore, in an effort to reduce costs and improve performance, both control system vendors and owners have been transitioning from proprietary technologies to the less expensive technologies prevalent in the IT world, such as Ethernet, TCP/IP, Microsoft Windows, and various web technologies. Unfortunately, many of these popular applications, protocols, and operating systems have a significant number of widely known vulnerabilities, with new ones being reported every day. Exploitation tools, worms, and how-to papers are often readily available shortly after the announcement of a new vulnerability, so rather than requiring specific knowledge, hacking many of the components of a modern SCADA system can be done with a few clicks of a mouse. 2 Scott Berinato; Debunking the Threat to Water Utilities, CIO Magazine, CXO Media Inc., March 15, 2002 3 Paul Dorey; Security Management in Process Control: The 3 Waves of Adoption, PSCF Spring 2006 Conference, Process Control Security Forum 6

Even the flaws in SCADA specific technologies have become general knowledge detailed presentations on how to exploit SCADA vulnerabilities have been given at public conferences such as BRUM2600, 4 ToorCon 2005, 5 and Blackhat Federal. 6 As more components of control systems become interconnected with the outside world, the probability and impact of a cyberattack will heighten. In fact, there is increasing concern among both government officials and control systems experts about potential cyberthreats to the control systems that govern our critical infrastructures. What is lacking is good historical data to either back up or dismiss this concern. To help provide guidance on this question, this report will look at the event data collected over the past five years by the Industrial Security Incident Database (ISID). It is hoped that this analysis will provide industry and governments current and relevant statistical data from which intelligent choices regarding industrial cybersecurity can be made. The Industrial Security Incident Database In early 2001 a security research team at the British Columbia Institute of Technology (BCIT) was asked by a major petroleum refining facility to investigate the possibility that their control systems could be impacted by cyber-related events such as hacking or viruses. In the course of this study it became apparent that accurate historical data on cyber impacts was badly lacking in the SCADA or process industries, making accurate risk assessment extremely difficult. To address this shortcoming, two researchers, Eric Byres and David Leversage, founded the Industrial Security Incident Database (ISID) with assistance from Justin Lowe of PA Consulting. Modeled after similar safety-related databases in the process industries, 7 ISID is intended to serve as an industrywide repository for collecting, analyzing, and sharing high-value information regarding cybersecurity incidents that directly affect SCADA, manufacturing, and process control systems. The ISID provides a historical representation of industrial cybersecurity incidents from which industry can gain a realistic understanding of the risks associated with industrial cyberthreats. It also provides ISID members with reliable information support for adapting current security policies to reflect the changing dynamics of industrial cybersecurity. The ISID attempts to addresses questions like: Which industrial cybersecurity incidents are fact and which are urban myth? How urgent is the security risk to control systems? What security vulnerabilities are exploited? 4 We have your water supply, and printers Brumcon report, The Register, October 20, 2003 5 www.toorcon.org/2005/conference.html?id=16 6 www.blackhat.com/presentations/bh-federal-06/bh-fed-06-maynor-graham-up.pdf 7 A good example is the Process Safety Incident Database (PSID), operated by the Center for Chemical Process Safety. Details on PSID can be found at www.aiche.org/ccps/activeprojects/psid/index.aspx. 7

What are the threat sources? How serious are the consequences? Incidents are obtained from either organizations voluntarily submitting an ISID reporting form to ISID investigators or from ISID staff harvesting reports from public sources such as the Internet, discussions at SCADA/industrial cybersecurity conferences, and relevant industrial publications. Examples of the latter include the Slammer Worm infiltration of an Ohio nuclear plant and several power utilities 8, 9 and the wireless attack on a sewage SCADA system in Queensland, Australia. 10 The ISID was designed with the capability to capture as many standard incident characteristics as possible. The database comprises various entry fields that require the submitter to provide a statistical description of the reported incident, including event date, affected industry, location, and affected network or device. The ISID also includes descriptive categories such as incident type (accidental/deliberate), intent (malicious/accidental), as well as type classifications such as external hacks, denial of service (DoS) attacks, and virus/worm infiltrations in order to extract the most information possible from the data. Figure 1 shows some typical data input screens for the ISID application. When an event is either submitted by an ISID member or noted in a public forum, it is first reviewed and verified by the ISID researchers. To protect the confidentiality of the ISID contributors, any information that may identify the source of the incident (such as the contributor s name, event location, or company details) are removed. The ISID researchers then attempt to ascertain the reliability of the report by verifying its details using standard investigative techniques. Each incident is then assigned one of four reliability ratings: 1. Confirmed 2. Likely but Unconfirmed 3. Unknown or Unlikely 4. Known Hoax/Urban Legend 8 NRC Information Notice 2003 14: Potential Vulnerability of Plant Computer Network to Worm Infection, United States Nuclear Regulatory Commission, Washington DC, August 29, 2003 9 SQL Slammer Worm Lessons Learned For Consideration By The Electricity Sector, North American Electric Reliability Council, Princeton NJ, June 20, 2003 10 R vs. Boden [2002] QCA 164 Appeal against Conviction and Sentence, Supreme Court of Queensland, May 10, 2002 8

Figure 1. Typical ISID data entry screens The degree to which the incident details can be verified determines the reliability rating the ISID researcher gives to the incident. For example, those events where the contributor is a reliable and firsthand witness or where there are official documents available (such as U.S. Nuclear Regulatory Commission reports or court documents) are considered Confirmed incidents. In contrast, incidents with secondhand data with limited detail, that have unknown sources, or that appear to the researcher to be improbable are given the rating of Unknown or Unlikely. As of June 30, 2006, there are 116 incidents that have been investigated and logged in the ISID database, with 12 incidents pending investigation and entry. Of these 116 records in the database, the 9 with a reliability of Unknown or Unlikely and the 1 with the reliability of Hoax/Urban Legend were excluded from analysis. An additional incident was also excluded because it had null data in the event date field and could not be used to obtain trend data. This left 105 records that were used for the analysis presented in the remainder of this report. The changing landscape The first question typically asked is whether or not the number of security incidents against SCADA and control systems is increasing or decreasing. To help answer this, Figure 2(a) graphs the frequency distribution of incident event dates. There are 14 categories of years ranging from 1982, the earliest incident event date in the database, to June 2006. 9

1982 to 2001 27% 2002 to 2006 73% 1982 1993 1994 1995 1996 1997 1998 1999 (a) 2000 2001 2002 2003 2004 2005 2006 (b) Figure 2. Incident events by date from 1982 to June 1, 2006: (a) graphed as a frequency distribution; (b) charted as a percentage (105 records) Clearly, cybersecurity incidents impacting control systems is not a new problem as noted above, the earliest incident recorded in ISID occurred in 1982, almost a quarter of a century ago. However, these early incidents were sporadic, and the period of continuous annual incidents (i.e., where there is no year without a reported incident) didn t begin until 1994. The first year to see a significant increase in the frequency of cybersecurity incidents being recorded in the ISID as compared to earlier years was 1998. Notice that there is a striking increase in the annual incident rate starting in late 2001. As figure 2(b) indicates, even though the four and one-half year period from 2002 to June 2006 represents less than 20 percent of the total time scale, it contains almost 75 percent of reported incidents. While it is possible that some of this increase is due to the fact that the database was started in early 2001, we believe that the bulk of the increase is not. We have found the event dates of incidents have a low correlation with the submit date, indicating that companies will report incidents long after they have actually occurred. Thus if more incidents had occurred prior to 2002, we would still expect to see a few of them being submitted as late as 2006. Since this is not happening, it appears that sometime between 2001 and 2002 there was a significant shift in incident occurrence rates. As we have noted in earlier papers on ISID 11 (and will elaborate on in later sections of this report), it appears that the time period between 2001 and 2002 marks a significant watershed for SCADA and controls security and is a natural partition for analyzing trend data in more detail. 11 Eric Byres and Justin Lowe; The Myths and Facts Behind Cyber Security Risks for Industrial Control Systems,VDE Congress, Berlin, October 2004 10

A deceiving trend On first reading of the early indicators for 2006, it might appear that there is a now a marked decrease in the frequency of cyberattacks against the SCADA and Process Control industry as compared to the 2003/2004 period. However, based on our experience in previous years, this is unlikely to be the case the time lag between the occurrence of an incident and when it is logged into the database (a mean delay of 13 months) is likely masking the true incident rates for 2005 and 2006. For example, at this point in 2005 only 10 incidents had been reported for 2004 and 15 for 2003; a year later that number had climbed to 23 and 29 respectively. Thus with eight incidents currently reported for 2005, we can assume that by 2007 the incident numbers for 2005 will be of the same magnitude as 2003 and 2004. Figure 3 shows the predicted incident rates from 1994 to 2005 along with a moving average trend line. Projected Actual 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 Figure 3. Actual and predicted ISID incidents from 1994 to 2005 The good news is that while events have increased significantly since 2001, the rate appears to have leveled off in the past few years and may actually have decreased slightly in 2005/2006. It is likely that the trends in the critical infrastructure industries are following similar trends found in the overall IT world. According to a report written by IBM s Global Security Intelligence team, the global IT threat landscape is going through a fundamental shift, or evolution, in cyber crime from pervasive global outbreaks to smaller, stealthier attacks targeted at specific organizations. 12 As IT networks are becoming increasingly more secure, it is anticipated that many of these attacks will target the most vulnerable access point within a company or organization, which could easily be the SCADA or process control system. 12 Surge in criminal-driven cyber attacks anticipated in 2006, IBM Global Business Security Index Report, December 2005 11

Estimating total incidents in the SCADA and control industries Discussions with operators of traditional business crime reporting databases indicate that the typical incident database collects no better than one in ten of the actual events occurring. Twentynine incidents were collected for 2003 and twenty-three for 2004, so it is likely that industry is experiencing at least 200 incidents per year at the present time. However, this number is probably several orders of magnitude low, due to the fact that of the 197 companies listed in the Fortune 500 with significant manufacturing or critical infrastructure operations, only 14 currently report to ISID and several of these are rather sporadic in their reporting. Thus it is probable that 2,000 to 3,000 industrial cybersecurity incidents are occurring per year to Fortune 500 companies alone. If this estimate is accurate, then it also indicates that even given the increasing acceptance of ISID, companies are still reluctant to provide information about security breaches. Intuitively one can expect that companies do not want to disclose that they have had problems with their network. This is also consistent with research conducted by Katherine Campbell et al 13 that found reports of security breaches can adversely affect a firm s stock price. Finally, the companies that do report to ISID tend to be on the leading edge of industrial cybersecurity preparedness and thus are likely experiencing lower incident rates as compared to the other companies. If nothing else, one conclusion we can draw from these statistics is that there is an ongoing security incident problem, and it may be more widespread than most control systems professionals believe. Summary of incident trend rates The overall trend data collected in the ISID, while limited in scope, appears to indicate two primary developments since the start of the millennium. First, the number of incidents affecting SCADA and control systems started to increase dramatically sometime in late 2001. This jump occurred within a short window of under six months. Second, this overall increase has now reached a plateau and has leveled out somewhere just below 2003 levels, a trend consistent with observations in the IT world. In the next section of this report, we will look at the possibility that while attacks may decrease slightly across the globe in volume, they are likely to increase significantly in severity, malicious intent, and associated negative consequences. 13 Katherine Campbell, Lawrence A. Gordon, Martin P.Loeb and Lei Zhou; The Economic Cost of Publicly Announced Information Security Breaches: Empirical Evidence from the Stock Market, Journal of Computer Security, Vol. 11, No. 3, 2003, pp. 431-448 12

The changing threat Inside to outside As we noted in the previous section, the number of cyber incidents occurring against the SCADA and manufacturing systems took a significant jump in late 2001. This begs the question, Did the nature of these events change as well? To help answer this, the ISID data was analyzed for incident type to get an idea of the threat sources. First, the period up to and including the year 2001 was investigated. Figure 4(a) shows the breakdown of 27 incidents between the years 1982 and 2001. Note that accidents, inappropriate employee activity, and disgruntled employees accounted for 74 percent of the problems, indicating that most of the threat, malicious or otherwise, was coming from within the company boundaries. These statistics correlate well with the numbers being expressed by security researchers in the IT world at the time. For example, this statistic was widely quoted in 2001: A study by the FBI and the Computer Security Institute on Cybercrime, released in 2000, found that 71% of security breaches were carried out by insiders. 14 The ISID study team then produced the same graph for 78 incidents during the period 2002 to 2006, as shown in Figure 4(b). In this time period externally generated incidents account for 60 percent of all events, indicating a surprising and significant change in threat source. Adult or other 7% Accidental 31% External 26% Accidental 52% External 60% Internal 15% Internal 4% Adult or other 5% (a) (b) Figure 4. (a) Incident types charted as a percentage from 1982 to 2001 (27 records); (b) Incident types charted as a percentage from 2002 to June 1, 2006 (78 records) 14 Tony Stephanou; Assessing and Exploiting the Internal Security of an Organization, The SANS Institute, March 13, 2001 13

Interestingly, the IT world appeared to experience the same shift. For example: Deloitte & Touche s 2003 Global Security Survey, examining 80 Fortune 500 financial companies, finds that 90% of security breaches originate from outside the company, rather than from rogue employees. For as many years as I can remember, internal attacks have always been higher than external, said Simon Owen, Deloitte & Touche partner responsible for technology risk in financial services. 60 to 70 per cent used to be internally sourced. But most attacks are now coming from external forces and that's a marked change. 15 Although there is no definite answer as to why this dramatic change took place in late 2001, there are a few possible explanations. First, as we noted earlier in this report, control systems have historically operated in an isolated environment where control devices typically did not communicate with outside systems. The move to integrated business communications systems and the widespread use of commercial off-the-shelf (COTS) technologies like Ethernet and TCP/IP have meant this isolation has broken down, especially since 2000, when the Y2K crisis drove massive upgrading of many systems. Second, the emergence of automated non-email worm attacks starting with Code Red 16 in July 19, 2001, has meant that many of the intrusions have become nondirected and automated, and the control system may have become just a target of opportunity. Since control systems rarely use or allow Simple Mail Transfer Protocol (SMTP) traffic, earlier malware that used email as a vector were unlikely to penetrate the plant floor. On the other hand, protocols such as Remote Procedure Call (RPC) and Structured Query Language (SQL) are ubiquitous in control environments, allowing the worms using these vectors easy access. This second interpretation seems to be supported by a closer look at the external incidents between 2002 and 2006, of which 78 percent (Figure 5) were the result of common viruses, Trojan horses, or worms. Particularly interesting is the fact that of these 36 malware attacks, only one (a Sobig-driven incident) used SMTP as its sole propagation technique. Three worms (Slammer, Blaster, and Sasser) accounted for over 50 percent of the incidents and these utilize the SQL Server Resolution Service (UDP Port 1443), the RPC Service (TCP Port 135) and the Microsoft-DS service (TCP port 445) respectively, to propagate to new victims. On last item worth noting is that the majority of these worm events occurred months or years after the worm was widely known in the IT world and patches were available and proven for control systems. This indicates to us a lapse in security policy rather than technology, a point we will revisit in later sections. 15 Nash, Emma; Hackers bigger threat than rogue staff, VNU Publications, May 15, 2003, www.vnunet.com/news/1140907 16 While Code Red was not the first non-email based automated worm, it appears to be the first to have had significant penetration into industrial systems. 14

Denial of service 4% System penetration 9% Sabotage 9% Virus/worm/trojan 78% Figure 5. The percent total of each external incident type category, 2002 to 2006 A nastier threat If attack trends in the financial and IT sectors are any indication, total attacks in the next few years may decrease slightly in volume, but are likely to increase significantly in severity, malicious intent, and associated negative consequences. As noted security expert Bruce Schneier points out: Hacking has moved from a hobbyist pursuit with a goal of notoriety to a criminal pursuit with a goal of money. Hackers can sell unknown vulnerabilities zero-day exploits on the black market to criminals who use them to break into computers. Hackers with networks of hacked machines can make money by selling them to spammers or phishers. They can use them to attack networks. We have started seeing criminal extortion over the Internet: hackers with networks of hacked machines threatening to launch DoS attacks against companies. Most of these attacks are against fringe industries online gambling, online computer gaming, online pornography and against offshore networks. The more these extortions are successful, the more emboldened the criminals will become. 17 A number of government and industry reports 18,19 have confirmed Schneier s observations. For example, Al Berg of Liquidnet predicts the following patterns for mainstream IT systems over next few years 20 : 1. Cybercriminals will continue using viruses and worms as tools. In the past, attackers released malware primarily to watch their code propagate globally and to wreak havoc on end users. However, members of organized crime have now realized the potential of these technologies as a means to carry out identity theft, espionage, and other cybercrime. 17 Bruce Schneier, Attack Trends, QUEUE Magazine, Association of Computing Machinery, June 2005 18 Ibid, IBM Global Business Security Index Report 19 NISCC Briefing: Targeted Trojan Email Attacks, National Infrastructure Security Coordination Centre, London, UK, June 2005 20 Al Berg, "Seven trends to expect from virus and worm authors in 2006," Threat Monitor, January 4, 2006 (http://searchsecurity.techtarget.com/tip/1,289483,sid14_gci1155150,00.html) 15

2. Malware production pace will increase and quality will improve. As more criminal organizations become malware consumers, writers will be offered additional rewards for developing malicious code. These rewards will lead to the evolution of more sophisticated code, an introduction of new variants, and more attention paid to evading detection. Additionally, 2007 will bring about a trend towards threat customization. 3. Viruses and worms will become increasingly targeted. Rather than sending out mass email to infect large numbers of random targets, authors will begin to craft their code to target specific user populations. 4. Coordinated attacks will reduce the distinctions between these types of threats. As attackers continue to combine virus/worms, Trojan horses, spyware, and phishing methods into increasingly more potent blended threats, the need for a unified antimalware strategy combining antivirus, antispyware, antispam and antiphishing solutions will be a major theme. We believe t hat these trends are likely to apply to the industrial controls world as well. In other words, attacks will be more specifically focused and, when successful against a SCADA or process control network, potentially far more damaging than in previous years. In fact, one statistic that stood out as a surprise to us was that 9 percent of all external incidents from 2002 to 2006 were the result of deliberate sabotage (Figure 5). This would seem to indicate that directed attacks are more prevalent than one might expect. Supporting this possibility is the fact that a number of the reports coming into ISID for 2005 and 2006 indicate a directed and commercial motivation for attackers. For example, in 2005, a confirmed report was provided of a complex and wide-reaching malware-based attack against the manufacturing lab systems of a major electronics manufacturer. The intent of this attack appeared to be industrial espionage. Nine months later, according to Japanese media reports, sensitive security information about a thermoelectric power plant run by the Chubu Electric Power Company was leaked onto the Internet following a virus infection. 21 Probably most ominously, in May 2006 the SANS Institute reported, Two SCADA systems have been penetrated for criminal (extortion) activity. 22 While these last two reports are yet to be confirmed, if true they send a chilling indication of the focused nature of future threats to control systems. 21 John Leyden, Japanese power plant secrets leaked by virus, The Register, May 17, 2006 22 SANS NewsBites, Vol. 8, Num. 42, May 26, 2006 16

The back door into the control system One of the enduring beliefs held in the SCADA and control systems world is that control systems are secure because they are simply never connected to the Internet. But if this is the case, then how are all these viruses getting to the plant floor and infecting SCADA systems? To answer this question, the study team decided to look more closely at the category of events reporting a remote point of entry. The data set was reduced to the 47 incidents that occurred between 2002 and 2006 and had Remote in the point-of-entry field. Figure 6 graphs the frequency distribution of each of the nine remote point-of-entry categories: Internet, Corporate WAN, Corporate Business LAN, Wireless System, Trusted Third Party, Virtual Private Network (VPN) Connection, Public Telecommunications Network, and Dial-Up Modem. Via business network 19% Internet 13% Dial-up modem 11% Corporate WAN 19% Wireless system 9% Other 13% Telco Network 4% VRN connection 6% Trusted third-party 6% Figure 6. Remote points of entry charted as a percentage from 2002 to 2006 (47 records) The results clearly show that while the business network (either LAN or WAN) was a major source, it was certainly not the only source. Secondary pathways such as dial-up connections, wireless systems, public telecommunications networks, VPNs, and third-party connections were all significant contributors. While shocking to some, the large number of and variety of pathways common in automation systems is corroborated both by the keynote presentation at the 2006 Process Control Security Forum (PCSF) and a recent ARC Advisory Group survey. 23 The PCSF paper reported that at one representative large energy company, 80 to 90 percent of all control networks were shown to be connected to the enterprise network, 24 which in turn, is interconnected to the Internet. In the case of the ARC survey, control engineers were asked about the types of connections that their automation networks had to the outside world. The summary results were as follows: 23 Bob Mick; Manufacturing Security Status & Strategies, ARC Advisory Group, October 2005 24 Ibid; Paul Dorey 17

47.5% Company Intranet/Business Network 42.5% Internet Directly 35% Direct Dial-up 20% Wireless Modems 17.5% No Connection 8.0% Other Connections Notice that the percentages in the ARC study do not add up to 100 percent, indicating that many automation networks had multiple connections. Both the research team s experience in conducting site security audits on control systems and the results in Figure 6 indicate that most facilities have not just one pathway, but rather multiple pathways into their control system. For example, one survey in 2004 uncovered 17 different pathways, while site management believed there was only one control system to business network data historian link. The use of older technologies such as dial-up modems for remote support and the integration of new technologies such as VPN access, laptops, and IEEE 802.11wireless present many pathways for attackers to gain access into the SCADA and process control networks. These include: Modems: Both leased-line and dial-up modems have been in use for decades to allow the remote support of control systems and are still widespread, especially on control devices that use serial communications or are located in remote locations. For example, the connection of maintenance modems to protection relays substations is a largely accepted practice throughout the North American power industry. Unfortunately, many of these modem/device pairs have been shown to have either no password or trivial passwords. Some are even so old as to not allow passwords at all. Wireless: There are many ways SCADA control systems companies utilize wireless technology. Traditionally, SCADA networks over large physical areas utilized licensed-band radio systems to allow remote nodes to communicate with a centralized management host. More recently, the large-scale deployment of Wireless Ethernet (IEEE 802.11) has created countless opportunities for intrusion and information theft. Third-party connections: Generally used for remote support by control systems vendors or product transfer by raw materials suppliers, these connections interconnect the control system to an outside network that may not follow the same security policies. Dial-up, long-haul serial, unencrypted wide area network, radio frequency, and VPN style connections are all used. 18

Virtual private networks: Often deployed as part of a third-party connection, these deploy encryption technologies such as SSL and IPsec to tunnel so-called secured communications across insecure networks (such as the Internet) and into the control network. Since the traffic is encrypted, it is commonly believed to be secure, but as noted in ISA-SP-99 Technical Report #1, 25 VPNs do not protect the network and workstations against most data-driven attacks (i.e., viruses) when the end nodes or networks are not also secured. As well, these connections can often bypass firewall rules because data is received in an encrypted format and cannot be checked by the firewall. Mobile devices: Mobile devices such as laptops, Personal Digital Assistants (PDAs), and flash drives are often used in a variety of environments, each with different security policies and practices. This allows the spillover of security issues from one system to the other. For example, if laptops are used both in the plant environment and in a less secure home environment, malware obtained in one setting may be unwittingly transferred to the other. Internet: While commonly denied, both the ARC Study and a number of the incidents in the ISID show that control systems do get connected directly to the Internet. Reasons for this include a desire to download system patches or antivirus updates from vendor Web sites, as well as a misguided desire to conduct typical office activities (such as email) from the plant floor. Figure 7 illustrates a few of the locations of possible pathways into organizations that employ segregated process control/scada networks, and all of them have been points of entry for at least one ISID incident. For example, database records show that the Slammer Worm had at least four different infiltration paths in the control systems it impacted: 1. The Davis-Besse nuclear power plant process computer and safety parameter display systems via a contractor s T1 line 2. A power SCADA system via a VPN 3. A petroleum control system via a laptop 4. A paper machine HMI via a dial-up modem The bottom line is that security designs that assume all traffic into the control system will flow through a single choke point may be making a dangerous assumption. Focusing a single solution (such as the Internet firewall) on a single connection point is likely to miss many possible entry points into the control system and leave the system open to attack. 25 ISA-TR99.00.01-2004, Security Technologies for Manufacturing and Control Systems, Instrumentation, Systems and Automation Society (ISA), 2004 19

Business/corporate network To Internet ISP link or leased line Main firewall VPN Hub/switch Wireless access point Industrial network HMI or server Industrial firewall VPN Hub/switch Laptop Process control nodes Figure 7. Typical entry points in control network structure The impact of cyber incidents In this section we will analyze the empirical data on incident financial costs of cyber events against control systems. As well, we will present a theoretical approach that seeks to overcome the absence of reliable and comprehensive financial statistics. Financial impact Since the inception of the ISID, obtaining reliable financial impact information has proven to be one of the most difficult challenges for the researchers. Even incident contributors that provide detailed data on the technical aspects of an event are often unable to provide even an approximate estimate of the cost impact. Thus for most of the incidents reported in the database, the contributors have been unable (or unwilling) to provide a financial measure of the impact of 20

the industrial cyberattack in fact only 46 of the 116 incidents have such an estimate. Figure 8 (a) shows the impact cost charted as a percentage of incident events from 1982 to 2001, and (b) from 2002 to 2006. $100,000 to $1,000,000 20% $10,000 to $100,000 13% $1,000,000 to $10,000,000 3% $100,000 to $1,000,000 10% $10,000 to $100,000 6% $1,000,000 to $10,000,000 20% Greater than $10,000,000 20% Less than $10,000 46% SO 29% Less than $10,000 27% Greater than $10,000,000 6% (a) (b) Figure 8. Impact Cost: (a) charted 1982 to 2001 (15 records); (b) charted from 2002 to 2006 (31 records) Of the 10 incidents that reported a financial impact greater than $1,000,000, it s interesting to note that four were attributed to sabotage and the remaining six were as a result of accidental equipment failure. Only one of the four incidents involving sabotage was the result of an opportunistic attack. Although the sample data set is not large, it does suggest that a target of choice will incur a significantly larger financial impact compared with a target of opportunity. Operational impact Assessing the consequences of an industrial cyberattack is not simply a case of assigning a financial value to an incident. Although there are obvious direct impacts which may be easily quantifiable financially (e.g., loss of production or damage to plant), other consequences may be less obvious. For most companies the impact on reputation is probably far more significant than the cost of a production outage. The impacts of health, safety, or environmental incidents could be highly detrimental to a company s brand image. Even impacts such as minor regulatory contraventions may affect a company s reputation and threaten their license to operate. Potentially more significant is the nature of the impacts of the attacks. Forty-one percent reported loss of production while 29 percent reported a loss of ability to view or control the plant. Fortunately, human impacts have been small, with only one unconfirmed (and possibly unreliable) report of loss of life. Overall, the reported incidents clearly show that the most likely consequences of industrial cyberattacks are loss of view and the ability to control the process or system. 21

The likely impact of being unable to view or control the process is an increased reliance on emergency and safety systems. Traditionally these systems have been completely independent of the main control system and generally considered bulletproof. However, mirroring the trend in the design of the main control systems, these emergency systems are also becoming based on standard IT technologies (such as TCP/IP). They are increasingly being connected to or combined with the main control system, increasing the potential risk of common mode failure of both the main control system and the safety systems. Consequently, the systemic risks of cyberattacks need to be considered in the design of not just the control systems, but also the safety systems. An alternative Cyber-Threat Impact Index When a precise calculation of costs is not feasible, what are the alternatives? Numerous methodologies that measure risk qualitatively, rather than quantitatively, are in use. A common practice is to rank information assets according to (a) how valuable they are and (b) how vulnerable they are to attack. The results, which can be displayed in a matrix with high-risk, high-value assets in one corner and low-risk, low-value assets in the opposite corner, may guide a firm s allocation of its security spending. The Cyber-Threat Impact Index (CTII) attempts to quantify the total impact of an incident by categorizing it into one of three impact classes: low, moderate, and high. Instead of being wholly dependent on the direct financial impact, other factors such as loss of employee time, loss of hardware, environmental consequences, and health and safety issues are considered as well. Impact is then more accurately defined as the total transaction cost (or consequence) experienced by the organization. The index categories are described by the following: Low: Financial impact of $10,000 or less No environmental or social impact Can be rectified quickly with minimal expenditure and employee time Moderate: Financial impact of $10,000 to $100,000 No environmental or social impact Attack requires upgrading or replacing equipment or software Sizable amount of employee or contractor time to correct the issue Some loss of production or efficiency 22

High: Financial impact of $100,000 and above Impacts to human safety and/or the environment Considerable loss of production and revenue Table 1 summarizes the results of this categorizing, and we can see that the majority of attacks can be classified as serious or moderate as defined by the CTII. The frequency of moderately severe incidents has increased steadily over the last few years. Given this, a future incident has about a 67 percent chance of being moderate or serious based on CTII categories averaged from 2001 to 2004. CTII Percentage Year Serious Moderate Low Serious Moderate Low 2001 4 2 0 66.67% 33.33% 0.00% 2002 6 3 5 42.86% 21.43% 35.71% 2003 8 9 12 27.59% 31.03% 41.38% 2004 5 11 7 21.74% 47.83% 30.43% Table 1. Incidents categorized by Cyber-Threat Impact Index and year Improving the security of industrial control systems This analysis of the ISID data indicates that organizations that operate SCADA and control systems have good reason to be concerned about cybersecurity. Not only have the number of incidents increased dramatically in the past five years, but the seriousness of these events appears to be increasing as well. Furthermore, the cost of each incident can be substantial. Even if there is no direct impact on production or revenue, there is cost associated with expenditure of employee time, the cost of upgrading/changing equipment, and the risk to corporate reputation. Virus and worm-related incidents make up a significant proportion of the total number of incidents impacting control systems. They also account for a significant percentage of the overall costs incurred due to the high volume of such incidents. The high frequency of virus and worm incidents suggests that security methods that are in place in many control systems are insufficient. For example, a perimeter firewall protecting the business network offers little protection against internally released viruses from mobile laptops connected to the control network. 23

The ISID analysis points to two areas where the security of the typical SCADA/PCN system could be improved significantly. First, the large number of incidents involving well known and easily addressed threat vectors indicate that many of the security issues need to be addressed through better policy, practices, and education programs rather than through pure technologybased solutions. For example, incidents involving the Slammer worm continue to be submitted to the ISID, almost five years after the patch for this vulnerability was initially released. Flaws in security policy and employee/contractor awareness are the root cause in nearly all these cases, rather than a failure in technologies such as antivirus or firewall software. Second, the existence of the numerous secondary pathways into the SCADA and control system point to the need for comprehensive, in-depth defense strategies. This includes better layering of firewall defenses and the hardening of endpoint devices through patch management, antivirus deployment, microfirewalls, and host firewalls within the SCADA/PCN proper. The remainder of this section describes both areas for improvement in more detail. The need for comprehensive security programs In any cybersecurity effort it is easy to get caught up in the razzle-dazzle of technological solutions and forget the soft aspects of security such as policy development, responsibilities, and staff training. Yet it is this human part of the equation that is critical to the success of any security program, not the technology. As the introduction to the ISO/IEC 17799:2005 standard notes: Experience has shown that the following factors are critical to the successful implementation of information security within an organization: 1. security policy, objectives and activities that reflect business objectives; 2. an approach and framework to implementing, maintaining, monitoring, and improving information security that is consistent with the organizational culture; 3. visible support and commitment from management; 4. a good understanding of the security requirements, risk assessment and risk management; 5. effective marketing of information security to all managers, employees, and other parties to achieve awareness; 6. provision to fund information security management activities; 7. distribution of guidance on information security policy and standards to all managers, employees and other parties; 24

8. providing appropriate awareness, training and education; 9. establishing an effective information security incident management process; 10. implementation of a measurement system that is used to evaluate performance in information security management and feedback suggestions for improvement.. 26 Reviewing the incidents reported in the ISID, it becomes clear that the root cause of many of the events is a breakdown in these human factors rather than a true failure of technology. Thus it is critical that SCADA/control system owners and operators start by developing a comprehensive control system security management program that covers all aspects of industrial control system security, including cyber and physical security. There are a number of excellent sources that provide guidance on how to create a control system security management system. The ISO/IEC 17799:2005 and ISO/IEC 27001:2005 standards specify a possible process from the IT perspective, while ISA-99.00.02 Part 2: Establishing an Industrial Automation and Control System Security Program 27 defines the key requirements from a process control perspective. Industry-specific requirements for the electric power industry are defined in the North American Electrical Reliability Council (NERC) Standards CIP-002-1 through CIP-009-1. 28 In addition to these formal standards are interpretive guides that help translate the language of standards into everyday terminology. A good example of this is the Symantec white paper, Effective Practices for Meeting NERC Critical Infrastructure Protection Requirements in the Electric Power Industry. 29 This paper summarizes an effective security program into five key steps: Step 1: Critical Asset Identification and Risk Assessment Step 2: Security Policy Creation and Update Step 3: Disaster Recovery Planning Step 4: Deployment of Protective Measures Step 5: Security Monitoring and Management While focusing on the electrical industry, the concepts in this paper are easily applicable to other SCADA and process industries and are easier to understand than most standards documents. By following this staged approach, the user moves from the initial stages of determining the actual security situation on the SCADA system or plant floor and the realistic risks to that system, 26 ISO/IEC 17799 Information technology Code of practice for information security management, ISO/IEC, 2005 27 ISA-99.00.02 Part 2: Establishing an Industrial Automation and Control System Security Program, Instrumentation, Systems and Automation Society (ISA), Fall 2006. 28 http://www.nerc.com/~filez/standards/cyber-security-permanent.html 29 Effective Practices for Meeting NERC Critical Infrastructure Protection Requirements in the Electric Power Industry, Symantec Corporation, 2006 http://www.symantec.com/enterprise/solutions/focus.jsp?solid=electric&solfid=nerc 25

through defining the goals and policies for mitigating that risk, to the creation of processes to ensure that disaster recovery is possible. Only after the program s foundation is laid in Steps 1 to 3 does the actual deployment of security measures come into effect. Finally, the program continues in Step 5 with ongoing monitoring to ensure compliance, and the feedback of measurable results to Steps 1 and 2 for refinement of the overall program. Taking short cuts on any of these steps can be a recipe for disaster. For example, a number of incidents have occurred on sites where control system staff had moved well into Step 4 (Deployment of Protective Measures) before completing the policy creation step. The result was that staff who did not understand the need for the security technologies on their site effectively nullified their security effectiveness by inappropriate actions. For example, during one particular site audit, network cables were discovered that circumvented the SCADA firewalls. The reason later given was that there was no risk analysis showing that the firewalls were important, nor was there a policy stating that bypassing them was unacceptable. Once again, this highlights the need for a control system security management system that is holistic and well designed, rather than a piecemeal approach to security. The need for defense in depth In many of our discussions with controls engineers, we hear the comment: We don t need to worry about securing our control system because the IT department has a firewall between the company and the Internet that will protect us. Yet since nearly 40 percent of all reported incidents were transmitted from the business network to the control system, clearly this strategy isn t working. Modern security practice mandates that effective security requires a defense in depth strategy, where critical systems are protected by layers of security. Depending on a single corporate firewall for control system security violates that strategy by creating a single point of security failure. Once the attacker or worm has either broken through or circumvented the single firewall, the entire control system is left wide open to exploitation. Furthermore, the security needs of the business network are not the same as the security needs of the control network. For example, the business firewall must typically allow users on the inside of the network to browse the Internet using HTTP, while the control system typically requires security policies that explicitly forbid this. Simply put, a single firewall cannot be all things to all departments. A good control system security strategy needs to offer layers of protection, starting with a dedicated control system firewall and progressing to specific protection for key devices and systems on the plant floor or SCADA system. 26

The primary control system firewall defines the security perimeter for the control system and acts as the choke point for all traffic between the outside world and the control system. Proper design and deployment of this firewall is critical ideally, it should be deployed in the appropriate multilayer architecture described in the NISCC Good Practice Guide on Firewall Deployment for SCADA and Process Control Networks. 30 Often this is not the case; as Dorey noted in his PCSF keynote speech, comments like My networks aren t connected; my server uses a separate network card to connect to the PCN and the corporate network do not indicate a secure network design and are simply a great way to infect both networks. Similarly, using routers or switches with access control lists (ACL) is generally not acceptable. Detailed reasons for using proper firewalls and the basics of designing multilayer architectures are described in the NISCC Good Practice Guide. Multifunction firewalls that combine firewall services, antivirus services, VPN services, and intrusion detection services are also recommended. As noted previously, VPNs often bypass firewall rules because data is received in an encrypted format and cannot be validated by the firewall. Combining the firewall function and VPN function in one appliance addresses this issue because the firewall can be given the ability to decrypt (and if necessary reencrypt) the VPN traffic. Similarly, the challenges of deploying A/V in the control network can be partially addressed by multifunction firewalls. Systems that cannot use A/V software (such as PLCs) or systems where signature updating must be delayed can get some level of protection from a firewall that offers A/V services as well. Once the electronic perimeter of the control system is secured, it is necessary to build the secondary layers of defense on the control system itself. This can be achieved using two primary techniques. For those control components (such as HMIs and data historians) that are based on traditional IT operating systems such as Windows and Linux, this can take advantage of the proven IT strategies of patch and antivirus management. For those devices like PLCs, RTUs, and DCS controllers, where patching or antivirus solutions are not readily available, the use of distributed security appliances is recommended. We will discuss both solutions in more detail below. Patch and antivirus management in control systems Hardening the control components that utilize common operating systems is a commonly suggested solution for improving system security. Yet with 78 percent of the reported incidents in the last four years being malware related, the deployment of antivirus (A/V) software and patch management in control systems obviously needs improvement. 30 Eric Byres, John Karsch, and Joel Carter, NISCC Good Practice Guide on Firewall Deployment for SCADA and Process Control Networks, National Infrastructure Security Co-ordination Centre (NISCC), July 8, 2004. 27

The difficulty with both antivirus deployment and patch management in SCADA is that one cannot blindly deploy new A/V signatures or patches into the industrial control environment without risking disruption of operations. In fact, there have been at least two cases recorded in ISID where inappropriate deployment of A/V patches on the control system has caused loss of production (Incidents #39 and #71). This does not mean that the deployment of antivirus software or patches in control systems should be given up as impossible. A number of leading companies have demonstrated that careful A/V and patching policy and practice can be used that balances the need for system reliability with the need for system security. For example, several major petroleum and chemical companies have publicly described how they successfully deployed antivirus technology and patch management on their control systems. 31, 32 The Edison Electric Institute (EEI) has detailed recommendations on a tiered approach to patch management for control systems. 33 Finally, most of the major control equipment vendors now offer guidance on both patch management and A/V deployment for their control products. Thus there is little reason for SCADA system owners/operators not to have good patch and A/V programs in place today. Layered protection for control devices In many cases, the most critical devices in a control system are based on operating systems and architectures that do not allow the addition of security features such as A/V software or permit regular patching. Furthermore, the majority of control devices in use today offer no authentication, integrity, or confidentiality mechanisms, and can be completely controlled by any individual that can ping the device. Thus the most critical devices on the plant floor are also the most vulnerable. A rapidly evolving security solution is the use of low-cost security appliances deployed directly in front of each control device (or group of devices) that needs protection. These appliances provide protection directly at the critical edge device, similar to the way personal firewalls, antivirus software, or intrusion detection systems provide local protection for desktop computers and servers. The result is a true defense in depth strategy, so that even if a hacker or virus manages to get through the main corporate firewall, they will still be faced with an army of SCADA-focused security devices that need to be breached before any damage can be done. Typically, each of these remote security appliances are centrally configured, monitored, and managed from a central management system. Because of their focus on protecting a small number of critical devices rather than a whole network, each appliance can be tuned to meet the security needs of the device it is protecting. 31 Ibid; Paul Dorey 32 Eric Cosman; Patch Management at Dow Chemical, ARC Tenth Annual Forum on Manufacturing, ARC Research, February 20-24, 2006 33 Patch Management Strategies for the Electric Sector, white paper, Edison Electric Institute IT Security Working Group, March 2004 28

Summary The above recommendations outline two of the most important steps that the SCADA and process controls industry needs to take if it is going to effectively protect itself from the threat of cyberattack. Failure to adapt to these changing threats and vulnerabilities will leave the industrial controls world exposed to increasing numbers of cyber incidents. The result could easily be loss of reputation, environmental impact, production and financial loss, and even human injury. 29

About Symantec Symantec is a global leader in infrastructure software, enabling businesses and consumers to have confidence in a connected world. The company helps customers protect their infrastructure, information, and interactions by delivering software and services that address risks to security, availability, compliance, and performance. Headquartered in Cupertino, Calif., Symantec has operations in 40 countries. More information is available at www.symantec.com. For specific country offices and contact numbers, please visit our Web site. For product information in the U.S., call toll-free 1 (800) 745 6054. Symantec Corporation World Headquarters 20330 Stevens Creek Boulevard Cupertino, CA 95014 USA +1 (408) 517 8000 1 (800) 721 3934 www.symantec.com Copyright 2007 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States and other countries. Other names may be trademarks of their respective owners. Printed in the USA. 02/07 11895980