welcome Lessons Learned from AMI Pioneers Follow the Path to Success Joe Cummins, PCIP UTC TELECOM May 2010
394 Simcoe Street South Oshawa, ON L1H 4J4 (905) 404-2009 2
outline security risks in smart grid tech" energy management" you can not control what you do not measure" understanding energy consumption patterns enables change" energy forecasting empowers consumers as well as utility providers" system design " field equipment: reliability, security, scalability, and maintenance considerations" back end software: device protocol support, thin client, and data archival" case study (military base)" understanding the physical metering landscape is key" performing a radio path study" having multiple wired and wireless options solves local problem areas" installation and commissioning tips" summary" 3
couple of thoughts about Smart Grid tech (1) AMI / AMR systems have similar vulnerabilities as SCADA Systems history is about to repeat itself (2) Old Threats, New Impacts (3) Get the Architecture Right (4) Think Defense-in-Depth (5) Consider a Push Data Flow Model
we ve seen this before Perimeter issues > these systems are interconnected with business applications (billing, work-order, account management systems, etc..), AND also often connected to operational SCADA and Energy Management systems for load shedding and remote tripping Back-end Server/Application issues > similar web and database app vulns as business applications, less secure implementation of protocols, and old versions of application frameworks Too much trust in the Protocol > Most AMI / AMR vendors are simply trusting that the 802.15.4 protocol security implementation will save them, and have not given much thought about scenarios when a communications mote is compromised End Devices have limited resources / weak stacks > The meters themselves do not typically have the resources to handle security features. Basically, the hardware cannot handle more computationally demanding processes, like upgrading their encryption handling capabilities once deployed. Limited tamperdetection capabilities cited, but not found operational in testing.
field life of 15-20 years Déjà Vu Due to high implementation costs, most AMI / AMR projects have long ROI cost recovery models, and are designed to operate for up to 20 years without requiring system upgrades Combine this with patching and firmware upgradability issues, and we are building into place the conditions that created much of the issues with SCADA and Process Control Systems Security Once these devices get deployed, they aren't going to get upgraded due to cost unless there is a major, crippling vulnerability found in them, and people are shamed into fixing it. quote by Jacob Kitchel (security researcher) All it will take is someone to get bored and go shut a city down by telling all the communication motes that everyone didn't pay their bill, then half flash the firmware and brick them all. - quote by Nick DePetrillo (security researcher)
old threats, new impacts Data Enumeration (read-time grid data) Host Enumeration (what systems can we connect to?) Service Enumeration (what services are exposed?) Change Data on the fly (can the data be manipulated in flight?) Steal accounts and passwords (system admin access anyone?) Damage core system components (cause meters to fail ) Denial of Service (PING FLOOD, Malformed Packets, etc )
Man-in-Middle Packet Capture
Write over any data in the stream (real time) MITM Data Injection (change data in meters) Change usage or billing data last24kwh=250; last24kwh=125;
Bricking PLCs and RTUs are relatively easy Smart Meters have similar stack issues PING Flood Often Results in Faulted PLC Processor. PLC Looses Configuration, and Must be Connected Locally with a Serial Cable to Upload Configuration.
denial of access Embedded device has a Login/Write Access password option 16 character limit Vendor specific Modbus/TCP function code Password stored in the Flash of the controller 11
denial of access Quick script to sweep the network, find controllers supporting this function code, and configure a password. 12
denial of access Locked Out. We just turned the device into some blinking bricks. 13
denial of control 14
let s open our eyes before we repeat the security issues we had with SCADA 15
get the architecture right Smart Grid could change current system connectivity models Currently each power company has control of their system access, and only EXPORTS their data to the local ISO through defined protocols (i.e. ICCP) This limits risk while allowing near real-time view of system loads Protects critical cyber assets at control systems, substations, and generation facilities from being exposed to 3rd parties Easier to design security into the system up front We need to think about some basic architecture and security standards now ahead of the implementation curve (think ISA99 and NERC)
US Military AMR. why? you can not control what you do not measure" Monitor the following devices:" electric power meters! gas meters" water meters" steam meters" older analog meters may need to be upgraded or replaced" Must support digital output capabilities" Modbus, DNP, or known protocols" If not, then add analog-to-digital devices that convert to Modbus RTU" 4-to-20 ma analog input module" KYZ pulse input module " understanding energy consumption patterns enables change" shifting load or heavy industrial equipment times can reduce peak demand" peak demand has the biggest impact on energy cost" energy forecasting empowers consumers as well as utility providers"
Smart Meter Configuration mesh radio network options" 900 MHz Serial > AES 256" 900 MHz Wireless Ethernet (802.11 TCP/IP over RF) > AES 256" 2.4 GHz Serial (802.15.4) > AES 256" 2.4 GHz Wireless Ethernet (802.11 b/g/n) > WPA2 Enterprise" data concentrators" each can poll up to 100 meters (25 per COM port)" data is polled and logged in field controller" if backhaul connection is lost, controller will store data, then backfill on reconnect" multiple backhaul options (serial or Ethernet in wired and wireless options)" back-end software" energyware" MS SQL, Oracle, CSV file transfer, and API options for enterprise integration" 18
Mesh Networks - Zoned 20
Web enabled HMI interface 22
contracted for several military base installations" sample Radio Path Survey" SAMPLE 23
lessons learned understanding the physical metering landscape is key" having over 4,000 device drivers allowed re-use of some legacy meters" performing a radio path study is essential for wireless solutions" all RF communications must be encrypted, and never allow Ethernet direct to the meter! all field equipment must meet strict environmental requirements for industrial applications some sites can have climates as low as -40 deg F and as hot as 120 def F outdoors. Office-grade or residential/commercial grade equipment failed." having multiple wired and wireless options solves local problem areas" planning and testing system at the vendor office prior to implementation exposes potential problems prior to commissioning" software that supports multiple forms of data integration (i.e. SQL, Oracle, CSV, and custom APIs) allows flexibility during integration with existing business or enterprise applications " understanding the use of data by enterprise applications, and documenting the data transfer methods upfront, makes integration on site smooth" design of solution should address data retention requirements both in the field (in case of communications loss) and in the back office for data archival" 24
Considerations 25
get the architecture right Smart Grid could change current system connectivity models Currently each power company has control of their system access, and only EXPORTS their data to the local ISO through defined protocols (i.e. ICCP) This limits risk while allowing near real-time view of system loads Protects critical cyber assets at control systems, substations, and generation facilities from being exposed to 3rd parties Easier to design security into the system up front We need to think about some basic architecture and security standards now ahead of the implementation curve (think ISA99 and NERC) NIST recently published a Smart Grid Security Model
defense-in-depth A truly secure Smart Grid should defend itself at multiple points throughout the system Should use active defense systems like firewalls or UTM devices to actively stop attacks at the touch points IDS/IPS should backup the firewalls and UTM devices to add another layer of protection All devices and system components should create security events and logs All events and logs should be centrally collected for event correlation, incident response, forensics, and audit trail Core key system components should have redundancy so that system continues to work, even while under attack System should use strong authentication methodology
consider a Push data flow model Rethink the open model where all meters can be read by everyone, data shared openly Do we really need real time data for true system demand / response? Can we achieve this in a secure way? Walk before we run Who owns the data? If the utilities own the system, they own the risk Before opening up the system, consider a model where participants push data out on a prescribed basis using a secure protocol
contact info Joe Cummins, PCIP President, Principal Consultant Red Tiger Security, Canada office: +1.877.387.7733 mobile: +1.613.878.6007 fax: +1.800.864.6249 jcummins@redtigersecurity.ca www.redtigersecurity.com User Forum www.redtigersecurity.com/community 29