A Strategic Approach to SCADA Cyber Security Water and Wastewater Network Architecture and Segmentation

Transcription

1 A Strategic Approach to SCADA Cyber Security Water and Wastewater Network Architecture and Segmentation Standards Certification Education & Training Publishing Conferences & Exhibits Speakers: Bill Phillips and Norman Anderson 2013 ISA Water / Wastewater and Automatic Controls Symposium August 6-8, 2013 Orlando, Florida, USA

2 Presenter Norman Anderson, PE: Norman has over 6 years experience in the design and commissioning of Process Control Systems for the Water Sector. Norman has provided secure and reliable PLC, SCADA, and Network hardware and software architecture designs and provided control system automation solutions for a range of facilities. Norman has an M.S. in EE from Iowa State University and an M.S. in Physics from the University of Florida. 2

3 Presenter Bill Phillips, PE: Bill specializes in delivery of secure and reliable process control and SCADA network and communications systems, cyber security vulnerability assessment, and facility automation and information system planning and implementation. Bill has over 30 years of process control and SCADA system experience and has focused on control system network and communications cyber security for the last decade. Bill has a BSEE from Clemson University. 3

4 Presentation Outline Securing Networks The Layered Network Architecture Network Organization and Segmentation Configuration Summary 4

5 Importance of Security Why Security is Important at a Water or Wastewater Facility: Critical Infrastructure and Public Safety o Critical resources o Downtime can affect life safety Operational Reliability and Availability o Attacks can lead to significant downtime Financial Impacts o Loss of revenue for utility and its customers o Mitigation and legal costs Media Attention o Loss of public confidence o Staff intimidation 5

6 Securing Networks Securing networks requires proper planning to ensure successful implementation. There are four basic stages of planning and implementation for network security: 1. Assessment Determine Risks and Mitigation techniques Risk impact versus cost of mitigation 2. Design Develop appropriate network architecture and segmentation (NOTE : Tailor to selected HMI suite TCP/UDP port requirements) Choose necessary hardware and software 3. Implementation Qualified and certified installers and designers 4. Operation and Maintenance Develop operational procedures for staff Maintain network, hardware, and software 6

7 Defense-In-Depth Security Risk Security Policies, Procedures, and Maintenance Firewall Rules Vulnerability Awareness Assessment / Design ICS Vulnerabilities Implementation Secure Programming Operational Network Configuration Training and Experience 7

8 Differences Between Corporate IT and Water Sector PCS Networks Process Control System Real Time Mainly used for equipment and processes to function Response time is critical Generally low bandwidth Rebooting must be scheduled or avoided Human safety and process uptime are paramount System uptime is most critical Non-Real Time IT Systems Mainly used by personnel to create and store data Consistent response time desired High bandwidth requirements Frequent rebooting is acceptable Data confidentiality and integrity is highest importance System and data protection is most critical Paraphrase From NIST SP Guide to Industrial Control Systems (ICS) Security Table 3.1., Summary of IT System and ICS Differences 8

9 A Layered Approach A Layered Network is part of the Defense-in-Depth Strategy. Divide the network into zones to provide a hierarchy of control for information flow. Generally most trusted zone is nested inside the other zones with the least trusted on the exterior. Creates a Peel-the-Onion environment for attacks. 9

10 Example Layered Architecture 10

11 Available Guidance Cisco/Rockwell Automation Converged Plantwide Ethernet (CPwE) Design and Implementation Guide odesign and implementation guidelines for industrial control systems oguide provides real network architecture examples and security and implementation methods ANSI/ISA Security for Industrial Automation and Control Systems: Establishing an Industrial Automation and Control Systems Security Program o Builds upon global standards ISO/IEC and ISO/IEC and addresses the difference needed for industrial security o Defines procedures for implementing and assessing secure industrial control systems 11

12 Similarities to other Guides and Standards Cisco/Rockwell Automation CPwE Design and Implementation Guide ANSI/ISA

13 Least Trusted Layer Business networks and large networks such as the Internet or Metropolitan Area Networks (MANs). Use to route between trusted networks using encrypted VPNs. Allowed access to read-only applications for SCADA viewonly and reporting applications. Used for maintenance access by package system vendors but not direct to PLCs. No direct access to the process control network from this layer. Used for access to other services such as software updates and NTP for time synchronization. Not a required layered. Only used when necessary to help operations and provide better service. If possible, external access should be avoided. 13

14 DMZ Layer Location for equipment that accesses the Process Control Network and Outside networks. Domain controllers in this layer should be read-only (slaves) from the Process SCADA network. Equipment located in this layer can access the outside network for alarming, reporting, and updating services but cannot write to the internal network without manual initiation from the Process SCADA network. 14

15 Process SCADA Network Layer SCADA system location with no direct access to the outside untrusted networks. Maintenance access can be provided by hopping through the DMZ. SCADA servers can directly access the Process control PLC network. Terminal services used for SCADA clients, or similar, can access SCADA servers and Operator workstations but not the PLC network. Control should only be allowed from this layer and the PLC network. 15

16 PCS PLC Network Layer Innermost layer requiring the most hoops to jump through for access from the outside. This layer is still segmented on separate networks to minimize broadcast domains and separate dissimilar traffic to allow for implementation of QoS rules. Devices on the same network can communicate in the absence of the firewall or a router to allow the control system to continue operation if the network head end devices were to fail. The PCS Firewall is shown but not required and is mainly used for routing between the SCADA and PCS PLC networks but may be needed for other functions. 16

17 Network Organization Start by defining networks using the logical Class. Generally... Class A is used for the internal networks having the largest number of devices, e.g., the PLC network Class B is used for communications between private networks, e.g., between treatment plants on a Metro-Ethernet network. Class C for public networks, e.g., Webserver or Firewall connected to the Internet and City-Wide network. Networks should be selected where they make sense, but should follow industry standards such as RFC RFC 1918 Name Address Range Network Class 24-bit block Class A (10/8 prefix) 20-bit block Class B (172.16/12 prefix) 16-bit block Class C ( /16 prefix) 17

18 VLAN Approach VLANs accompany subnetworks selected and a 1:1:1 relationship should be maintained between VLANs, Subnets, and Broadcast Domains. Virtual LANs (VLANs) - Useful for SCADA systems because VLANs define broadcast domains that can be widely dispersed (i.e. not on the same network segment) Can reduce costs, by allowing host on different networks to share layer 2 switches. Use 802.1q VLAN encapsulation protocol Layer 3 device required to route between VLANs. Layer 2 devices support VLANs and VLAN Trunking. VLAN Approach: o o o o o Use VLANs in the range of , various restrictions apply to other VLANs Don t Use VLAN 1 (Native or Default VLAN) Verify VLAN capabilities of network switches & routers Use logical approach Incorporate VLAN designations into IP Addresses 18

19 IP Addressing Example Y=0 is the network and Y=255 is the broadcast address Subnet mask can be 9-30 bits. 1st /29 subnet: network address , host range nd /29 subnet: network address , host range

20 VLAN Example VLANs should be selected in a logical order, recommend using Trust Level. In the example below, VLANs are numbered inversely to Trust Level numbers. Aids in network organization and identification of networks, locations, and components. Reduces broadcast domains to reduce network traffic and unnecessary requests to components. Increases network security. 20

21 Providing a coordinated system Approach: Incorporate facility & VLAN numbers into IP addresses Limit broadcast domains to a single facility and to a 254 host max. Primary VLAN Example: 10.VLAN.Facility.Host/X or 10.Facility.VLAN.Host/X X = Subnet Mask bit count X (Generally between 24 &30) based on anticipated host count WAN Example: Y/X X = Subnet Mask bit count (Generally between 24 & 30) based on number of nodes Y = Host Number and depends on Subnet Mask 21

22 Example Network Configuration SFP Gi1/0/21 Gi1/0/22 Gi1/0/23 Gi2/0/21 Gi2/0/22 Gi2/0/23 Gi2/0/24 E0/0 E0/1 E0/2 40-ENS-1 40-ENS-2 E0/2 E0/1 E0/0 SFP Note: Use separate physical media or routers to separate VLANs that have public access to prevent VLAN attacks such as ARP poisoning. 22

23 Example Remote Connections Central Control Room Remote Internet Connected Workstations Remote Control Room 40-FWL-1,2 Firewalls 50-FWL-1,2 Firewalls 15Mbps/3Mbps User VPN User VPN (Disaster Recovery) 15Mbps/3Mbps Internet NOTES: 1. USER VPN CONNECTIONS ARE VPN CONNECTIONS INITIATED BY REMOTE DEVICES. Site-to-Site VPN Site-to-Site VPN User VPN (Disaster Recovery) 2. SITE-TO-SITE VPN CONNECTIONS ARE VPN CONNECTIONS INITIATED BY THE HOST (POLLING PLC). 3. DISASTER RECOVERY CONNECTIONS ARE USED WHEN PRIMARY CONNECTIONS FAIL OR ARE LOST. User VPN Cellular Wireless Network Cellular 3G LINE LEGEND: Disaster Recovery Pathways Mobile Cellular Wireless Workstations Primary Connection Pathways 3G Wireless Digital Cellular Modem Service Provider Connection Pump Station PLC 23

24 VLAN Assignments and Rules Local network in example is broken into multiple VLANs. VLANs incorporated into IP Addresses along with facility and Host numbers. Provides an organized network allowing for internal staff to easily identify devices and networks. Multiple VLANs can reside within Layer 2. Routing accomplished by the Firewall using extended ACLs. 24

25 IP Addressing Table CENTRAL CONTROL ROOM SCADA (VLAN10) DEVICE LINKSTATE (VLAN11) DEVICE MUNICIPAL WAN (VLAN800) DEVICE WEBSERVER(VLAN30) DEVICE / / / /24 1 ENS int Vlan 10 (gateway) 1 FW (virtual, gateway) 1 City gateway 1 FW (virtual, gateway) 2 FW (virtual) 40-FWL-1,2 2 FWa 40-FWL-1 2 FW (virtual, gateway) 2 FWa 40-FWL-1 3 FWa 40-FWL-1 3 FWb 40-FWL-2 3 FWa 40-FWL-1 3 FWb 40-FWL-2 4 FWb 40-FWL-2 4 Primary SCADA (LinkState) 40-SVR FWb 40-FWL-2 11 Primary RODC 40-SVR UPS Secondary RODC 40-SVR-1-2 Do not use Spare for Future Use 6 40-UPS WebServer 40-SVR BROADCAST 7 BROADCAST * 8 Reserved for Future Network Equipment MetroEthernet (Vlan801) Device * Spare for Future Equipment /29 * 10 1 Gateway 255 BROADCAST 11 Primary DC/DNS Server 40-SVR-3-1 PLC (VLAN20) DEVICE 2 FW (virtual, gateway) BUSINESS (VLAN40) DEVICE 12 Primary SCADA Server 40-SVR /24 3 FWa 40-FWL /24 13 Historian Server 40-SVR FW (virtual, gateway) 4 FWb 40-FWL-2 1 FW (virtual, gateway) 14 SCADA Terminal Server 40-SVR FWa 40-FWL FWa 40-FWL-1 Spare for Future Use 15 SCADA NAS 40-SVR FWb 40-FWL FWb 40-FWL-2 16 Alarm Server 40-SVR BROADCAST 11 Primary DC 40-SVR Monitoring Server 40-SVR PUBLIC -(V900) DEVICE 12 Primary MS Exchange Server 40-SVR /29 13 Business Terminal Server 40-SVR-2-3 Reserved for Future Network 19 Reserved for Future Servers 7 Equipment 1 ISP (Gateway) 14 Business NAS 40-SVR FW (virtual, Port Address Translation 17 Printer #1 40-PRT-1 21 SCADA Full Client 40-WKS FWa 40-FWL-1 18 Printer #2 40-PRT-2 22 SCADA T.S. Client 40-WKS FWb 40-FWL-2 21 Business Client 40-WKS SCADA T.S. Client 40-WKS Master PLC#1 (Internet) 40-PLC-1 5 Spare for Future Use 22 Business Client 40-WKS Reserved for future workstation 12 Master PLC#1 (Server) 40-PLC-1 6 Primary MS Exchange Server 40-SVR- 23 Business Client 40-WKS-2-3 * 13 Master PLC#2 (Internet) 40-PLC-2 7 BROADCAST * * Spare for Future SCADA Equipment 14 Master PLC#2 (Server) 40-PLC-2 * Spare for Future Equipment * * Spare for Future PLC Equipment * 255 BROADCAST 255 BROADCAST 255 BROADCAST 25

26 Configuration and Management Configuration and management are simpler. Network expansion is simpler. Subnets are already set with IP Addresses reserved or easy to determine. The appropriate routes between devices are already configured via subnet and VLANs. Router and Firewall rules are simplified using subnets and VLANs instead of individual addresses. Management is simpler since addresses are easily identified with equipment, facility, and VLAN assignments. Identifying an intruder is also more obvious. 26

27 Firewall Trust Level Assignments Security Levels - Implicit Deny Lower-to-Higher level: Each Interface & Sub-interface Inside 100 (Most trusted) Outside 0 (Least trusted) DMZ 50 Interfaces Typically 3-4 separate physical ports on Firewall for small to medium size firewalls. Allows separation of business and control networks. Sub-interfaces allow a single firewall port to be shared by a number of VLAN subnets. Network organization allows for logical assignment of Trust Level with VLANs and Subnets. Use Firewalls with Stateful Inspection Can drop otherwise legitimate packets that are not part of an active connection Holds in memory variables defining the state of each connection State variables include things like source and destination addresses, port numbers, packet sequence numbers 27

28 Firewall Rules Access Control Lists (ACLs) Access Control Lists Used to apply access control rules at interfaces Permit DMZ to-inside SCADA specific traffic such as web server, terminal server and historian traffic. Permit VPN LAN-to-DMZ authenticated remote user traffic such as web server, terminal server and historian traffic. Remote PLC Connections: Consider a Remote PLC DMZ to avoid direct connections between Internet connected PLCs and the SCADA network Consider dual Ethernet DMZ PLC interfaces (i.e. separate VLANs) to increase separation. 28

29 Example Firewall Configuration Define addresses for system components: set address "Trust" " /24" set address "DMZ" "Historian_Svr" HMI- SCADAHIS in DMZ Addresses for the SCADA network through and the Historian server have been set and assigned to the Trust and DMZ trust levels. Set Rule for allowed communication: set policy id 16 from "DMZ" to "Trust" "Historian_Svr" " /24" "_RDP_TCP" permit log count Policy allows service _RDP_TCP from the Historian in the DMZ to the SCADA network in the Trust level. Define the policy: set service "_RDP_TCP" protocol tcp src-port dst-port Policy defines the allowed ports for communication. All other ports are denied. Using an organized and logical network organization allows for simpler and logical configuration. 29

30 Summary Network security is an important aspect of any Water Sector Process Control System. Multi-layered network organization provides a foundation for building a secure Process Control Network. Using logical subnet and VLAN selections provides a usable segmentation framework that allows for easily identifiable components, eases expansion, and makes network configuration and management simpler. A layered network provides additional protection from attacks and allows more time to identify an intruder. VLANs minimize broadcast domains, reduce bandwidth requirements and increase network response and security 30