ENERGY TRANSFER PARTNERS, L.P. PHMSA CRM and SCADA. Joe Schmid & Anil Pinto. Tuesday, April 29th, 2014

Transcription

1 ENERGY TRANSFER PARTNERS, L.P. PHMSA CRM and SCADA Joe Schmid & Anil Pinto Tuesday, April 29th, 2014

2 INTRODUCTION Energy Transfer is a Texas-based company that began as a small intrastate natural gas pipeline operator and is now one of the largest and most diversified pipeline companies in the US. Growing from roughly 200 miles of natural gas pipelines in 2002 to approximately 56,000 miles of natural gas, natural gas liquids (NGLs), refined products, and crude oil pipelines today. CygNet has been our SCADA platform since Interstates; Fayetteville Express Florida Gas Transmission Panhandle Eastern Sea Robin Pipeline Transwestern Pipeline Tiger Pipeline Trunkline Gas Liquids; Lone Star West Texas Gateway Liberty Freedom Justice Intrastates; Allegheny Houston Pipeline Oasis SETPS (Aquila) ET Fuels Regency; RIGS MidCon West Texas South Texas SUGS 2

3 ENERGY TRANSFER PIPELINES 3

4 ETP SCADA TEAM The ETP SCADA team is an IT shared services organization. The 14 member SCADA department consist of Houston and San Antonio SCADA teams. The total combined years of pipeline automation experience is over 162 years. We support two main Control Centers located in Houston and San Antonio with a total of 20 24/7 control desks that fall under PHMSA CRM requirements. In addition to Control Center SCADA, the teams also support real-time data needs for operations and commercial, along with EFM collection for measurement accounting. 4

5 AGENDA Section 1 PHMSA and Control Room Management Section 2 Controller Information Section 3 Alarm Management 5

6 PHMSA Pipeline Hazardous and Material Safety Administration PHMSA is a U.S DOT Agency. The PHMSA agency was created within the U.S. DOT under the Norman Y. Mineta Research and Special Programs Improvement Act of 2004 The agency is responsible for developing and enforcing regulations for the safe, reliable, and environmentally sound operations of; The US' 2.6 million mile pipeline transportation infrastructure which accounts for 64 percent of the energy commodities consumed in the US. That s roughly 182,000 miles of hazardous liquid pipelines, 304,000 miles of gas transmission pipelines, and 1.9 million miles of gas distribution pipelines in the United States. PHMSA's Mission Statement Protect the people and the environment from the inherent risks associations with the transportation of hazardous materials, whether it be by pipeline or other modes of transport. 6

7 US PIPELINE MAP Source: 7

8 DOT BACKGROUND INTO CONTROL ROOM MANAGEMENT The DOT had been examining Control Room operations for over 10 years. (1999, 2003, 2005) Three Advisory Notices on SCADA & Fatigue were issued. (1998) SCADA System Check List was initiated (2001) SCADA Training Class Developed (2002) Pipeline Safety Improvement Act ( ) Study of control room operations to enhance pipeline safety (2006) Two CRM Workshops (2006) Report to Congress 8

9 PIPES ACT 2006 The PIPES Act was introduced by the Senate Commerce Committee, approved by Congress, then signed into legislation by the President. Pipeline Inspection, Protection, Enforcement, and Safety The Act broadened the scope of the systems-based approach to assessing and managing safety related risks. The additional initiatives included: (1) Increasing enforcement activity, transparency, and data quality. (2) Implementing an integrity management program for distribution pipelines. (3) Require a human factors management plan to reduce risks associated with human factors, including operator fatigue in pipeline control centers. (4) Implement NTSB recommendations on the Supervisory Control and Data Acquisitions (SCADA) systems in pipelines. (5) Mandated that the Secretary of Transportation issue regulations on control room management (CRM) requiring a human factors plan that assures control systems are matched to human capabilities and limitations, including fatigue management. 9

10 NATIONAL TRANSPORTATION SAFETY BOARD (NTSB) The NTSB is an independent Federal agency charged by Congress with investigating every civil aviation accident in the U.S. and significant accidents in other modes of transportation-railroad, highway, marine and pipeline. (2005) in a Pipeline Safety Study, the NTSB identified 10 of 18 Hazardous Liquid accidents from 1998 to 2005 as having potential CRM involvement. In the process of this study, the NTSB identified five areas for potential improvement: display graphics, alarm management, controller training, controller fatigue, and leak detection systems. As a result, the NTSB issued five safety recommendations to PHMSA: 10

11 BELLINGHAM, WASHINGTON 11

12 INDUSTRY SUPPORT 12

13 INDUSTRY SUPPORT Letters to the Department of Transportation (DOT) from Industry Associations supporting DOT efforts to improve pipeline safety and reliability. Associations included; American Gas Association (AGA) American Petroleum Institute (API) American Public Gas Association (APGA) Association of Oil Pipe Lines (AOPL) Interstate Natural Gas Association of America (INGAA) 13

14 PHMSA PARTS 192 (GAS) & 195 (LIQUIDS) Final Rule Amended February 1 st, PHMSA amended Federal pipeline safety regulations with the final rule on 49 CFR Parts 192 and 195 to address human factors and other aspects of control room management for pipelines where controllers use SCADA. Compliance Deadlines August 1 st, 2011 An Operator must have developed CRM plans and procedures and implemented all except for Controller Information and Alarm Management by this date. August 1 st, 2012 An Operator must have implemented Controller Information SCADA Displays and Alarm Management. 14

15 PHMSA PARTS 192 (GAS) & 195 (LIQUIDS) a)general Under the final rule, affected pipeline operators must; Define the roles and responsibilities of controllers and provide controllers with the necessary information, training, and processes to fulfill those responsibilities. Operators must implement methods to prevent controller fatigue. The final rule further requires Operators to manage SCADA alarms, assure control room considerations are taken into account when changing pipeline equipment or configurations, and review reportable incidents or accidents to determine whether control room actions contributed to the event. b)roles and Responsibilities Controllers and their supervisors/managers have certain responsibilities, accountabilities and authorizations during normal, abnormal, and emergency operations. 15

16 PARTS 192 (GAS) & 195 (LIQUIDS) c)adequate Information Controllers are to be provided adequate information, tools, processes, and procedures necessary to carry out their roles and responsibilities. d)fatigue Mitigation A policy to mitigate the safety risks associated with controller fatigue, which includes provisions for shift lengths, shift rotations, fatigue education, and fatigue recognition training. e)alarm Management SCADA alarms are a key tool for managing pipeline operations, but excessive numbers of alarms can overwhelm controllers. Implement a policy to provide for effective controller response to safety-related alarms including provisions to review SCADA safety-related alarms, identify points of safety off-scan, verify alarm set-point values, review the plan, monitor controller activity, and address deficiencies. 16

17 SECTION 1 PHMSA PARTS 192 (GAS) & 195 (LIQUIDS) f)change Management Policy and procedures that verify changes affecting control room operations are coordinated with control room personnel. g)operating Experience A policy with procedures that verify lessons learned from operating experience are incorporated, as appropriate, into CRM procedures. h)training A Controller training program that trains each controller to carry out his or her roles and responsibilities, including recognition of and response to abnormal operating conditions, emergency communication responsibilities, and a working knowledge of the pipeline system, including infrequently used pipeline operating setups. 17

18 SECTION 2 - CONTROLLER INFORMATION c)adequate Controller Information Pipeline SCADA Displays Required API sections 4, 8, 9, & 11 required 195 all sections required Point-to-Point Verification Internal Communications Plan Backup SCADA System Testing Shift Briefings 18

19 API-1165 RP - SCADA DISPLAYS Section 1 Scope Section 2 References Section 3 - Definitions Section 4 - Human Factors Engineering Consideration in Display Design Short Term Memory STM limitations pose a strong argument for well grouped data and should always be considered when building or modifying displays. Signal to Noise Ratio Displays should not have elements without meaningful purpose. Eye Scan Pattern Information critical to the controller should be picked up on the initial scan. Consistency Consistency is essential to display design, especially if the controllers are managing multiple monitors. Coding Coding with colors allows more information to be conveyed in a smaller space and helps with STM limitation because color is processed prior to conscious processing. Section 5 - Display Hardware General Considerations Display Device Types Number of Monitors per Controller Console External or Remote Display Devices Display Response Controller Input Devices 19

20 PIPELINE SCADA DISPLAYS AND API RP-1165 Section 6 Display Layout and Organization General Consideration Display Hierarchy Window Management Issues Section 7 Display Navigation General Considerations Navigation Techniques Zoom, Pan, and Overlays Section 8 - Object Characteristics Color Number of Colors Color Selection Optimal contrast occurs when using the three primary colors (red, green, blue) three secondary colors (yellow, cyan, and magenta) and black, and white. Low contrast colors such as cyan on top of blue should be avoided. Meaning of Colors Redundancy Symbols and Shapes Animation Text Format Content Labels Messages Abbreviations 20

21 PIPELINE SCADA DISPLAYS AND API RP-1165 Section 9 - Object Dynamics Coding Data Values Discrete State Database Points Numeric Database Points Data Attributes Section 10 Control and Selection Techniques Object Selection Command Execution Error Management Section 11 Administration Consistency within a Company Documentation Consistency between Control Centers and Remote Locations MOC 21

22 PIPELINE SCADA DISPLAYS AND API RP-1165 Data Attributes - Each database point within the system has many possible data attributes above and beyond its actual value. A data attribute refers to the quality or characteristics of the point. It is important that a controller can easily see both the data value and the data attribute, especially the data attributes that indicate the point value is abnormal or unreliable. 22

23 DISPLAY EXAMPLE LAYOUT AND ORGANIZATION

24 POINT TO POINT VERIFICATION Point to point verification is required of SCADA when new field equipment is installed or when changes are made to field equipment or displays that could affect pipeline safety. 24

25 POINT VERIFICATION 25

26 BACKUP SCADA SYSTEM TESTING Pipeline operators will be required to test any backup SCADA systems and to test and verify a means to manually operate the pipeline (in the event of a SCADA failure) at least annually not to exceed 15 months. 26

27 SCADA BACKUP TEST 27

28 SHIFT BRIEFINGS Pipeline operators must establish a means of recording shift changes and other situations in which responsibility for pipeline operations is handed over from one controller to another. 28

29 SHIFT BRIEFING Excel template based reporting CygNet Realtime points using VBA and Excel automation. Formatted for Printer ready PDF format. PDF auto generated and stored on FileSystem / Blob for accessibility. 29

30 SHIFT BRIEFING Color coded point status for discrete values Operator interactive checklists. Operator log notes and electronic signing integrated. 30

31 SHIFT BRIEFING Excel automation to query point verification records from AUD. Automation uses CygNet s EventIf component to query AUD on specific filters and date and time ranges. 31

32 SHIFT BRIEFING PDF REPORT 32

33 SECTION 3 ALARM MANAGEMENT Alarm Management - The PHMSA CRM rule requires an Alarm Management plan, but does not require following API 1167 (Pipeline SCADA Alarm Management). As a best practice, Energy Transfer follows API PHMSA s Alarm Definition - Alarm means an audible or visible means of indication to the controller that equipment or processes are outside operator-defined, safety related parameters. Examples of Safety Related Alarm Conditions Fire (Fire Detection) Leak (Gas Detection, Pressure or Volume Rate of Change, Liquid Leak Detection Application) H2S (H2S Detection) ESD Over Pressure (exceeding MAOP) Mainline Valve closure Wide Area Communication failure SCADA System Failure Physical Security Breach PHMSA requires corrective action procedures. Alert - A non-safety related notification of a pipeline event that does not require immediate action because it does not have immediate significant consequences. Alerts are always of lower priority than alarms, they are not safety-related. 33

34 SECTION 3 ALARM MANAGEMENT Review safety related alarm operation to ensure alarms are accurate and support safe pipeline operations. Monthly Documentation Identify points affecting safety that have; been taken off scan had the alarm inhibited generated false alarms been forced to manual values Annual Documentation Verify correct safety related alarm set points values and alarm descriptions. Monitor the content and volume of general activity being directed to and required of each controller annually. Review the plan annually and address deficiencies. 34

35 ALARM MANAGER 35

36 ALARM MANAGER 36

37 CRM POINTS MONTHLY REVIEW 37

38 CRM POINTS MONTHLY REVIEW VERIFICATION 38

39 ALARM MANAGER 39

40 ALARM MANAGER 40

41 ALARM MANAGER 41

42 ALARM MANAGER 42

43 ALARM MANAGER 43