Change and Configuration Management

Change and Configuration Management for CIP Compliance OCTOBER 21, 2009 Developed with:

Presenters Bart Thielbar, CISA Senior Research hanalyst Sierra Energy Group, a Division of Energy Central CIP-003, R6 Primer and Related Requirements Kim Morris Director, Architecture and Information Security Public Service of New Mexico (PNM) Change and Configuration Strategies for CIP 2

Change and Configuration Management CIP-003, R6 Primer and Related Requirements BART THIELBAR, CISA SENIOR RESEARCH ANALYST

Disclaimer The information from this webcast is provided for informational purposes only. An entity's adherence to the examples contained within this presentation does not constitute compliance with the NERC Compliance Monitoring and Enforcement Program ("CMEP") requirements, NERC Critical Infrastructure Protection ("CIP") CIP) Reliability Standards, or any other NERC Reliability Standards or rules. While the information included in this material may provide some of the methodology that NERC has elected to use to assess compliance with the requirements of the Reliability Standard, this material should not be treated as a substitute for the Reliability Standard or viewed as additional Reliability Standard requirements. In all cases, the entity should rely on the language contained in the Reliability Standard itself, and not on the language contained in this presentation, to determine compliance with the CIP Reliability Standards. 4

Agenda Purpose, Applicable CIP Standards Change and Configuration Management New Developments related to V2 and TFE s Audit Trail Requirements 5

Why Good Change and Configuration Management? Differing practices and polices among departments and/or units within departments t Differing documentation practices Managerial visibility and organizational control Risk Management NERC s Interests = Best Interests of Reliability 6

Applicable CIP Standards and Rules CIP 003, R6: Change Control and Configuration Management CIP 007: Within Context of Change and Configuration Management Test Procedures Ports and Services Security Patch Management Malicious Software Prevention Security Status t Monitoring i Disposal or Redeployment Documentation Review and Maintenance 7

Schedule for Table 3 Entities Requirement Begin Work Substantially Compliant Compliant Auditably Compliant CIP 003, R6 12/31/06 12/31/08 12/31/09 12/31/10 CIP 007 12/31/06 12/31/08 12/31/09 12/31/10 8

Change Management Documented process of change control Adding, modifying, replacing, or removing Critical Cyber Assets Applies to hardware and software Easy to develop tunnel vision i and focus only on software Links with Configuration Management 9

Configuration Management Documented process of change control Identify, control, and document All entity or vendor related changes to Critical Cyber Assets Links with Change Management 10

Overview/Explanation Documentation that tracks changes to Critical Cyber Asset hardware or software to include: Adding: adding hardware/software to existing system Modifying: making a change to existing hardware/software Replacing: adding new hardware/software Removing: retire/redeploy hardware/software Configuration management activities to: Identify: when a change needs to be made or has been made Control: approval for changes Document: documentation for the whole process 11

Process Summary Items requiring documentation when changed: Process documentation (use good version control) Critical Asset & Critical Cyber Asset List Test environment Patch management Changes to Electronic Security Perimeter (ESP) Changes to Physical Security Perimeter (PSP) Process for identifying and changing g ports and services settings Criteria in the change process: Types of changes Who initiates the changes Who approves the changes Approvals and dates for all auditable items Testing records 12

Change Management vs. Configuration Management (One View) Change Management Process and activities undertaken to make changes to CCA s Ex: Applying a software upgrade or adding additional memory to laptop Configuration Management Process and activities iti undertaken to establish and/or make changes to configuration of CCA s Ex: Setting up a new CCA and/or changing configuration of existing CCA such as port activation The documented process must address both 13

Testing Why: Ensure that new cyber assets and/or changes to cyber assets (not just CCA) do not compromise CCA serving Bulk Electric System Significant Change: Security patches, service packs, vendor releases/upgrades (including operating systems, applications, databases, etc.) Environment: Test Environment is very important must reflect the production environment Documentation: Test results must be documented (good, bad or neutral) All consistent with generally accepted best practices for change management 14

Speaking of that Good change and configuration management practices are just good business Should be viewed as a part of overall control and governance framework CIP Standards have specific requirements, but individual policy and practice may go beyond Manage the risks or they may manage you! 15

A Simplified View Change Control and Configuration Management (CIP 003, R6) Test Procedures Asset Management Documentation Review and Maintenance Security Issues and dimpacts (e.g., patch management, virus protection, etc.) Access Review and Maintenance 16

The Audit Trail Recall -> Once again, all associated measures, measure documentation ti Documentation emphasizing: Process for change control and configuration management Test procedures, test environment, test results Security & Access issues (Ports & Services, Patch Mgmt, Malicious Software Protection, Security Status Monitoring, Cyber Vulnerability Assessment & Account Management) Disposal or Redeployment Annual review & update of CIP 007 Documentation If changes to systems or controls, documentation needs to be updated within 90 days* 17

New Developments 9/30/09 FERC Order changes documentation requirement from 90 days to 30 days for CIP 007, R9. This is effective in April, 2010. Also applicable to CIP 006, R1.7, CIP 008, R1.4; and CIP 009, R3 10/12/09 NERC Compliance Bulletin #2009-007007 Addresses Interim Approach to Technical Feasibility Exceptions (TFE s) May be requested for CIP 007, R2.3, R4, R5.3, R5.3.1, R5.3.2, R5.3.3, R6 and R6.3 18

Possible Penalties and Sanctions Up to $1 M per day, per violation Violation Severity (level of non compliance) Violation Risk Factors Mitigating factors may reduce penalties and sanctions Quality of compliance program, self-reporting, voluntary corrective actions, etc. Aggravating factors may increase penalties and sanctions Repeat violations, evasion, inaction, unwarranted intentional violations based on economic choice, etc. May potentially impact reputation, rate cases, etc. 19

Final Thoughts Always remember the importance of tone at the top and how it influences a culture of compliance Change and Configuration Management Practices are about Risk Management and impact many areas of CIP compliance efforts Compliance is a process, not an event Documentation, documentation, documentation NERC s Interests = Best Interests of Reliability 20

Change and Configuration Management Change and Configuration Strategies for CIP KIM MORRIS DIRECTOR, ARCHITECTURE AND INFORMATION SECURITY

Agenda NERC Guidance CIP Interdependencies with Change Control and Configuration Management Questions & Answers 22

Based in Albuquerque, N.M., PNM Resources is an energy holding company with 2008 consolidated operating revenues from continuing and discontinued operations of $2.5 billion. Through its utilities - PNM and TNMP - and energy subsidiary - First Choice Power - PNM Resources serves electricity to 859,000 homes and businesses in New Mexico and Texas. Current Capacity 2717 MW 23

CIP Standards Applicability 24

CIP Component Breakdown Citi Critical lasset tinventory Selection Criteria Security Perimeter Asset Selection Perimeter Definition Access Controls Cyber Assets Change Configuration Management Management 25

CC & CM in a Nutshell Applies to Hardware and Software in the Security Perimeter Industrial control systems Example: Control Center SCADA Physical Security Management Systems Example: CCTV, Badge reader systems Communications within ESP IT Management Services Examples: Monitoring and Management systems 26

Change Control Definition Summary Establish and Document a process for managing Change for Critical Cyber Assets Applicability Change Management Process Requestors Approval Authority Testers Implementers Supporting Documentation 27

Configuration Management Definition Summary Establish and Document configuration management process for adding, modifying, replacing or removing critical cyber asset hardware or software Applicability Document Management 90 Day window (change to 30-days in V2) Version Control Classification and Protection Testing Training Asset Inventory Ports and Services 28

Interdependencies 29 Function Organization ntrol Industrial Co Systems Control Cente er Operations IT Manageme ent Services Access Control X X X X X X X Change Management X X X X X X Document Control X X X X X X X X X Testing and Quality Assurance X X X X X X Network Management X X X X X Incident Response X X X X X X X X Systems Management X X X X X Training X X X X X X X X X Recovery Operations X X X X X X Governance X X X X X X X X X Exception Management X X X X X X Physical Secu urity General Coun nsel Human Resou urces Communicati ions Generation Substations

Documentation, Documentation Governance Documents: Policy Procedures Controls Asset Configuration Examples: Cyber Security Policy Test Plans Recovery Plans Monitoring 30

Document Maintenance Change Management Ensure ongoing document review via change management process Asset Configuration Ports and Services Hardware/Software Release & Patch Level Recovery Plans Training Plans Testing and Q/A Procedures and Testing Results Asset Inventory 31

Document Governance Scheduled Periodic Reviews Annual Review Internal Governance Team Vulnerability Assessment Validate Documentation 32

Leverage Existing Processes Governance Methodologies Incident Management Vulnerability Management Risk Management Change Management Configuration Management Corporate IT Security Existing Policies and Procedures Existing Governance Processes 33

Sample Reference Approach 34

Organization Roles for Compliance Senior Manager Compliance Manager 3 rd Party Security Network Training Corporate Support Engineer Engineer Engineer IT Security 35

Leverage Existing Programs and Standards Organizations Financial Sarbanes Oxley (SOX) Reliability NERC Information Technology Infrastructure Library ITIL National Institute of Standards NIST International Standards Institute ISO 36

Test Procedures Change Management Applicability Security Patches Application and OS Updates Database Updates Firmware/Hardware Documentation Considerations Testing and Q/A Back out Plans Contingency Operations i.e. Illness, weather, disaster Recovery Operations Ports and Services Training TFE s 37

Training, Training, Training Change and Configuration Management Training Asset Additions, Changes, Disposals Incident Response Governance Polices and Procedures 38

Change and Configuration Guidance for Malicious Software Prevention Change Control and Configuration Management Antivirus and Malware Engines and Management Software Dat and Signature files Intrusion Prevention Host-Based versus Appliance-Based Signature updates TFE s (per current NERC guidance) 39

Technical Feasibility Exceptions (TFE s) Recommendations per current NERC guidance: Establish standardized policy and process for TFE s Capture forms for TFE s in Security Policy Utilize standard exception process for TFE s 40

A Look into the Future.. AMI and Smart Grid Impacts IP enabled networks Integrated Utility Electronic Security Perimeter Smart Controls Network visibility ibilit to the home 41

Final Thoughts Additional Cyber Risks will continue to be identified Ensure compliance program can adapt to meet the changing demands of the organization and reliability Streamline Processes and Controls Six Sigma Tools and automated processes Find opportunities to use existing processes and controls Think like an auditor 42

Questions & Answers 9/30/09 FERC Order: http://www.ferc.gov/eventcalendar/files/20090930165448-rd09-7-000.pdf NERC Compliance Bulletin: http://www.nerc.com/files/2009-007 007_Public_Notice Notice-V1.pdf Contact Information: webcastquestions@energycentral.com The magazine for building a smart grid and delivering information-enabled energy. FREE subscriptions available at www.intelligentutility.com. 43

Questions & Answers 9/30/09 FERC Order: http://www.ferc.gov/eventcalendar/files/20090930165448-rd09-7-000.pdf NERC Compliance Bulletin: http://www.nerc.com/files/2009-007 007_Public_Notice Notice-V1.pdf Contact Information: webcastquestions@energycentral.com Your source for IT and smart grid research, analysis, and consulting. Visit www.sierraenergygroup.net. 44

Questions & Answers 9/30/09 FERC Order: http://www.ferc.gov/eventcalendar/files/20090930165448-rd09-7-000.pdf NERC Compliance Bulletin: http://www.nerc.com/files/2009-007 007_Public_Notice Notice-V1.pdf Contact Information: webcastquestions@energycentral.com Go to where the power industry gathers for news, information, and analysis, visit www.energycentral.com. com 45

Questions & Answers 9/30/09 FERC Order: http://www.ferc.gov/eventcalendar/files/20090930165448-rd09-7-000.pdf NERC Compliance Bulletin: http://www.nerc.com/files/2009-007 007_Public_Notice Notice-V1.pdf Contact Information: webcastquestions@energycentral.com 46 Get the inside scoop with Energy Central Professional News Service. Start your FREE trial at http://pro.energycentral.com/professional.

Questions & Answers 9/30/09 FERC Order: http://www.ferc.gov/eventcalendar/files/20090930165448-rd09-7-000.pdf NERC Compliance Bulletin: http://www.nerc.com/files/2009-007 007_Public_Notice Notice-V1.pdf Contact Information: webcastquestions@energycentral.com Join the discussion, raise your question, and voice your opinion at www.energyblogs.com. 47

Questions & Answers 9/30/09 FERC Order: http://www.ferc.gov/eventcalendar/files/20090930165448-rd09-7-000.pdf NERC Compliance Bulletin: http://www.nerc.com/files/2009-007 007_Public_Notice Notice-V1.pdf Contact Information: webcastquestions@energycentral.com The magazine for C-level executives about the business of energy. FREE subscriptions available at www.energybizmag.com. 48

CIP Compliance Series Webcasts For comprehensive preparation for the implementation, compliance, and auditing phases of the CIP standards program, attend all six. Upgrade and save 10%. Apply your single event purchase to the cost of the entire series. Call 800-459-2233 or e-mail orders@energycentral.com for information. Date Topic 9/23/09 Identifying Critical Assets (On Demand) 10/6/09 Program Governance Issues (On Demand) 10/21/09 Change Management Systems (On Demand) 11/11/09 Personnel Issues & Training 12/2/09 Physical & Electronic Access Controls 12/16/09 Testing Procedures & Recovery Plans 49

Thank You for Joining Us For the latest news, articles and blogs, please visit... www.energycentral.com 50