Fire Risk Mitigation in Mission Critical Facilities

Fire Risk Mitigation in Mission Critical Facilities Mark L. Robin, PhD DuPont Chemicals & Fluoroproducts mark.l.robin@chemours.com 2015 NFPA Conference & Expo McCormick Place in Chicago, IL June 23, 2015 1

Air Traffic Control Towers: Mission Critical Typical Monday Morning Air Traffic Snapshot http://blogs.mprnews.org/newscut/2012/10/hurricane_impact_on_air_traffi/ 2

Airline traffic at 10:20 a.m. EDT over the United States Sept. 26, 2014, following a fire at a suburban Chicago ATC Facility http://www.cbsnews.com/news/chicago-air-traffic-halted-over-fire-at-faa-facility/ 3

Consequences of Fire at a Mission Critical Facility: Aurora ATC Facility More than 2,000 flights in and out of Chicago cancelled by evening of fire Flights as far away as Dallas cancelled Ultimate cancellation of 3900 flights in the four days following the fire a Cost to American, United and Southwest estimated in the hundreds of millions of dollars for each airline b $123 million cost in passengers lost economic activity a Some passengers diverted to Detroit a US Travel Association estimate b Boyd Group Int l estimate 4

Fires in Mission Critical Facilities: Recent Examples Facility Location Aurora ATC Tower Chicago, IL State Police Data Center Maryland, USA Iowa Legislative Building Samsung SDS Iron Mountain Cowboyminers NSA Spy Center Shaw Communications Iowa, USA Fires DO occur in Gwacheon, South Korea mission critical facilities Buenos Aires, Argentina Bangkok, Thailand Utah, USA Calgary, Canada Macomb County IT facility Michigan, USA Cowboyminers Fire Samsung SDS Facility Fire 5

Fire Risk in IT Facilities Fire History Typically involve small fires Potentially large impact Equipment, Data and Business Continuity at Risk Equipment and data loss High cost of downtime Leading cause: electrical distribution equipment Fire Hazard Wires, cables, cord, plugs, outlets Fuel load primarily electronic equipment and power cables: plastics 6

Electronic Equipment: Fire Damage Thermal Damage - Due to fire itself (heat) Non-thermal Damage: Combustion Products - Smoke, soot, water, acids Non-thermal Damage: Suppression Agent - e.g., water, foam, dry chemical Electronic equipment is very susceptible to damage from fire 7

Electronics: Thermal Damage Component Onset of Damage Storage media (magnetic tape, floppies, etc.) 125 o F (52 o C) Hard drives 150 o F (66 o C) Electronic components 174 o F (79 o C) Paper 350 o F (177 o C) Polystyrene cases, reels 650 o F (310 o C) Microfilm 225 o F (107 o C) Damage results in loss of equipment and data 8

Electronics: Non-Thermal Damage from Combustion Products Steam, Smoke, Soot, Various Combustion Products Hydrogen Chloride - From combustion of PVC - Reacts with galvanized zinc components forming ZnCl 2 - ZnCl 2 layer formed, reacts with humidity to produce a corrosive ZnCl 2 solution Corrosive Combustion Gases - HF, HBr, SO 2, CH 3 COOH, NO 2, HCN 9

Cost of Business Interruption Cost of datacenter downtime is high and getting higher Average cost per minute increased 41% from 2010 2010: $5,617 per minute 2013: $7,908 per minute Source: Ponemon Institute 2013 10

Fire Protection Options for Mission Critical Facilities FOAM WATER CO 2 POWDER CLEAN AGENTS 11

Carbon Dioxide (CO 2 ) Extinguishment via oxygen depletion and heat absorption Is a clean agent Toxicity concerns CO 2 lethal at its required extinguishing concentrations 12

Powder-based Agents Efficient fire extinguishment via chemical interaction with flame All incorporate solid particles Corrosion concerns Major cleanup required Significant downtime 13

Foam Efficient extinguishment via separation of fuel from air Aqueous = Corrosion concerns Major cleanup required Significant downtime 14

Fire Protection Options for Mission Critical Facilities FOAM WATER Corrosive Extensive Clean-up CO 2 Corrosive Extensive Clean-up Toxic POWDER CLEAN AGENTS 15

Fire Protection Options for Mission Critical Facilities WATER- BASED SYSTEMS Required by code for STRUCTURE protection CLEAN AGENTS Protection of Structure s CONTENTS 16

Automatic Sprinkler Standards NFPA 13 Standard for the Installation of Sprinkler Systems Principle document addressing design and installation of sprinkler systems NFPA 72 National Fire Alarm Code NFPA 75 Standard for the Protection of Electronic Computer/Data Processing Equipment NFPA 76 Recommended Practice for the Fire Protection of Telecommunications Facilities 17

Automatic Sprinkler Systems: Design Objectives - In general terms of property protection, sprinkler systems are typically designed to achieve fire control... - Fire control can be described as limiting the fire size by decreasing the rate of heat release and pre-wetting adjacent combustibles, while maintaining ceiling gas temperatures so as to avoid structural damage NFPA Fire Protection Handbook, 19th Edition, p. 10-193. 18

Fire Control Heat Release Rate sprinkler activation Time 19

Automatic Sprinkler Systems Thermal Response Heat detection serves as basis of sprinkler system response fusible link glass bulb Activation at ceiling temperatures 135 o F Fire size at activation 100s of kw 20

Fire Protection Basics: Clean Agents Clean Agents NFPA 2001 (2015) 3.3.6 Electrically nonconducting, volatile, or gaseous fire extinguishant that does not leave a residue upon evaporation. ISO 14520 2006 Edition Minimum cleanup required No to minimum downtime 21

Clean Agent Fire Suppression Systems NFPA 2001 Standard for Clean Agent Fire Extinguishing Systems (2015) Total flooding agents Uniform distribution of agent throughout enclosure Ability to extinguish hidden and obstructed fires Primary Design Objective = Fire Extinguishment Rapid detection Rapid extinguishment Fire size at activation 0.1 to several kw 22

Clean Agent System vs. Sprinkler System (Fire Extinguishment vs. Fire Control) sprinkler activation FIRE CONTROL Heat Release Rate detection clean agent system activation FIRE EXTINGUISHMENT Time 23

Comparison Testing of Preaction Sprinkler and Clean Systems a Clean Agent System: FM-200 System designed and installed in accordance with NFPA 2001 Preaction Sprinkler System Designed and installed in accordance with NFPA 13 Detection/Alarm Systems Designed and installed in accordance with NFPA 72 In-cabinet fire: ABS sheets a M.L. Robin and E.W. Forssell, "Comparison Testing in a Simulated Data Processing/ Telecommunications Facility: FM-200 and Automatic Sprinkler Systems, 2004 Halon Options Technical Working Conference, May 4-6, 2004, Albuquerque, NM. 24

Fire Location 25

ABS Sheets 26

Preaction Sprinkler System Designed in accordance with NFPA 13 Based on Ordinary Hazard Class I classification Nine sprinkler heads in main space Nine sprinkler heads above suspended ceiling 11 ft spacing for area coverage of 121 ft 2 Maximum spacing allowed under NFPA 13 is 15 ft Recessed pendant standard response glass bulb sprinklers Temperature rating 135 o F Application density of 0.15 gpm/ft 2 required 27

FM-200 Suppression System Designed in accordance with NFPA 2001 7% by volume FM-200 Discharge time: 9.5 seconds Hygood Ltd cylinder Hygood Ltd 8-port aluminum nozzle orifice area 1.57 in 2 System design via Hygood Ltd s design software 30 s delay employed from detection to system activation Maximum delay time allowed under recommendations of FM Global Property Loss Prevention Sheet 5-14 on Telecommunication Facilities 28

Results: Preaction Sprinkler System Photoelectric detector in NE corner in full alarm at 94 seconds from ignition Ionization detector in NE corner in full alarm at 112 seconds from ignition Complete obscuration due to smoke at approximately 240 seconds from ignition Sprinkler head in NE corner actuated at 273 seconds from ignition Sprinkler head in N corner actuated at 347 seconds from ignition 29

Results: Preaction Sprinkler System Fire not extinguished by sprinkler system IR camera shows fire burning through entirety of test Fire contained to source cabinet Max ceiling temperature of 560 o F observed at thermocouple tree nearest fire Fire Damage Entire cabinet and contents destroyed Non-Fire Damage Extensive water, smoke and soot damage 30

Results: Preaction Sprinkler System Non-Fire Damage Black ring around entire enclosure Ceiling tiles discolored Soot particles scrubbed from smoke layer cover floor, horizontal surfaces Walls discolored from smoke damage Water damage to paper goods 31

Pre-action System 32

Pre-action System 33

Pre-action System 34

Pre-action System 35

Pre-action System 36

Pre-action System 37

Pre-Action Sprinkler System Performance Summary Design objective attained: Fire was controlled System performed exactly as expected Fire contained to origin Ceiling temperatures managed such that structural damage and/or collapse did not occur Structure saved 38

Results: FM-200 System AnaLASER II alarmed at 78 seconds from ignition FM-200 system activated at 108 seconds from ignition Fire extinguished at 125 seconds from ignition 7 seconds from end of system discharge No change in ceiling temperatures Fire damage Slight scorching of inside of test cabinet Non-fire damage Several ceiling tiles displaced 39

FM-200 System: Before.. 40

FM-200 System: After.. 41

FM-200 System Performance Summary Design objective attained: Fire extinguished System performed exactly as expected Contents of structure saved 42

Fire Risk Management 101 Fire risk analysis is a process to characterize the risk associated with fire that addresses the fire scenario or fire scenarios of concern, their probability, and their potential consequences. Risk factors to be considered include life safety and (direct and indirect) economic losses from the loss of function (capacity) or data, loss of professional reputation, and the costs of redundant systems R.W. Bukowski, Fire Protection Engineering Emerging Trends, Issue 76 (09/2013) Risk Considerations for Data /Center Fire Protection 43

Fire Risk Management 101 KEY ELEMENTS 1. Risk 2. Scenario 3. Probability 4. Consequences Direct Indirect Mission Critical Facilities: Equipment-related fire Cables, electronic equipment NOT zero Due to fire and extinguishant Ex: Business interruption 44

Fire Risk Management Standards & Guides SFPE Engineering Guide: Fire Risk Assessment (SFPE G.04.2006) NFPA 551 Guide for the Evaluation of Fire Risk Assessments NFPA Handbook of Fire Protection, Chapter 8, Section 3, Fire Risk Analysis ASTM E1546-09a Standard Guide for Development of Fire-Hazard-Assessment Standards 45

Mission Critical Facility Fire Protection Sprinkler and Clean Agent systems vastly different Design objectives different: Control vs Extinguishment Preaction systems best suited to protection of structure Clean agent systems best suited to protection of contents of structure Sprinkler systems alone inappropriate for protection of high value assets Clean agents not ideally suited for structural protection Loss during a fire NOT limited to fire damage Potential loss differs for clean agent vs sprinkler systems 46

Quantifying Loss: Direct vs. Indirect Costs of Fire Losses and costs associated with a fire are not limited to the equipment involved in the fire Direct: Equipment involved in the fire Data Historical and Intellectual Property Financial records Cost of Clean up Indirect: Business interruption & downtime Insurance Lawsuits Collateral Damage Loss of customer confidence 47

Shaw Datacenter Fire: Minimum Fire Protection and Its Consequences The potential consequences of adopting a minimum protection approach to fire protection can be clearly seen in the results of the recent Shaw datacenter outage in Calgary 48

Shaw Datacenter Fire: Transformer fire sets off sprinkler system Sprinklers take out backup systems housed on site Knockout out of primary and backup systems supporting key public services Crippled city services, including 311 services Delay of 100s of surgeries at local hospitals IBM Canada forced to fly backup tapes holding vehicle and property registration data to a backup facility in Markham, Ontario 49

Shaw Datacenter Fire: Extensive water damage to furniture, walls and sensitive electronic equipment on the floors below the top story fire location Temporary relocation of over 900 Shaw employees while damage is repaired Six days of service outage 50

Shaw Datacenter Fire: More than 20,000 Shaw business and household clients watched their cable, telephone and Internet services disappear almost immediately City and provincial clients watched their own networks go dark as the water trickled down, affecting other data companies below the fire. Some cellular carriers were also said to be affected, as well as ATB banking services, ATMs and debit terminals throughout the city 51

Shaw Datacenter Fire: Competitor Q9 Networks Inc., which operates three data centres in the city, has early detection systems that can detect smoke and use gases that don t harm people to prevent fires from starting. Water needs to be there by law, said Q9 chief executive Osama Arafat, but you want to try and deal with a fire using less destructive means first. Source: http://m.controlfiresystems.com/news/data_fire/ 52

Case Study: Iowa State Legislature Building Datacenter Fire: February 18, 2014 Mission Critical Facility Payroll processing for state employees was scheduled for that evening Dept of Revenue needed to process tax collections. Justice Dept needed to process claims and fee payments. 53

Iowa Legislature Fire Timeline 3pm Fire ignites: FM-200 System deploys 9pm Data Center Cleared of burnt equipment 2am Major websites and agency systems restored 3am Remaining major agency applications restored http://www.govtech.com/state/fire-in-your-data-center-no-power-no-access-now-what.html 54

Fire Protection for Mission Critical Facilities Fires DO Occur in Mission Critical Facilities Risk, Consequences, Losses dependent on facility & fire protection system Sprinklers : fire control/structure protection Clean agents: fire extinguishment/content protection Substantial risk reduction at very high benefit/cost ratios may be realized by protecting sensitive, valuable and mission-critical assets, such as those found in IT and telco facilities, with both a clean agent system and a sprinkler system. 55

Thank You 56