Repair strategies for a survivable Water-Treatment Facility

Size: px

Start display at page:

Download "Repair strategies for a survivable Water-Treatment Facility"

Posy Ford
8 years ago
Views:

1 Repair strategies for a survivable Water-Treatment Facility B.R. Haverkort A. Remke S. Roolvink Oktober 4-6, 2010

2 Background Dependability of critical infrastructures is essential Stable society and healthy economy Recovery costly and takes time Increased attention by governments [Dutch Ministry of Internal Affairs, 2005] [U.S. Department of Energy, 2002] Legal demand: 24x7 availability Repair crew labour cost Using Arcade-PRISM [Haverkort et al., 2010]

3 Water-Treatment Facility

4 Water-Treatment Facility

5 Approach Survivability [Cloth and Haverkort, 2005] Survivability is the ability of a system to recover to a predefined service level in a timely manner after the occurrence of a disaster. Quantitative survivability [Haverkort et al., 2010] Smarter repair strategies Engineering solution for state-space reduction

6 Least Redundancy First Phase with fewer redundancies can form bottlenecks Internal non determinism: Phase with most failed components Using other strategies (FFF, FRF, FRL, FFL) (3) Softeners (2) (1) Sand filters Reservoir (2+1) Pumps MTTF: 2000 h MTTR: 5 h MTTF: 1000 h MTTR: 100 h MTTF: 6000 h MTTR: 12 h MTTF: 500 h MTTR: 1 h

7 Drawback Least Redundancy First (1) AA (2) C C (3) B B Service level computed logic gates No awareness of system architecture

8 Largest Service Increase First Repair the bottleneck in the system Quantitative survivability Haverkort et al. [2010] Use Quantitative survivability logic gates Internal non determinism: Using other strategies (FFF, FRF, FRL, FFL) (3) Softeners (2) (1) Sand filters Reservoir (2+1) Pumps MTTF: 2000 h MTTR: 5 h MTTF: 1000 h MTTR: 100 h MTTF: 6000 h MTTR: 12 h MTTF: 500 h MTTR: 1 h

9 State-space reduction (3) (3) Line 1 Softeners Sand filters (3) Line 2 Softeners MTTF: 2000 h MTTR: 5 h MTTF: 1000 h MTTR: 100 h (1) Reservoir (2) (1) Sand filters Reservoir MTTF: 6000 h MTTR: 12 h (3+1) Pumps (2+1) Pumps MTTF: 500 h MTTR: 1 h States Trans. Dedicated line one line two FRF line one line two FCFS line one line two Weak bi-simulation Symmetry reduction Property driven state-space reduction

10 Bare Disaster Recovery State-space reduction: Engineering solution Remove operational components Remove remaining failures Leads to computational error Reduces the CSL formulae State-space: Lowerbound: N + 1 Upperbound: 2 N up up up up up up up up down down down down down down down down

11 Computational error (1/2) n (A(i) A(i)) (1) i=1 A(a) = (Pr{rec; t i } + Pr{rec; t i 1}) i 1 2 (2) A(a) = (Pr{rec; t i } + Pr{rec; t i 1}) i 1 2 (3)

12 Computational error (2/2) Computed error for 4 and 8 component model Separate error removal components and failures Fixed MTTR = 500hours Using factor = MTTF MTTR Three scenarios MTTF variable: All, operational, failed

13 Computational error 4 component model Factor All - only failed All - no failures All variable Failed variable Operational variable

14 Computational error 8 component model Factor All - only failed All - no failures All variable e Failed variable e Operational variable e

15 Water-Treatment Facility (2) Intake Intake pumps pumps (2) Coagulation Coagulation (6) Fast Fast Sand Sand filters filters (2) Ozonation Ozonation MTTF: 800 h MTTR: 2 h MTTF: h MTTR: 6 h MTTF: 7000 h MTTR: 24 h MTTF: 5000 h MTTR: 3 h (3) (3) Slow Softeners Slow Softeners Sand Sand filters filters (1) Reservoir Reservoir (3+1) Pumps Pumps MTTF: 2000 h MTTR: 5 h MTTF: 8000 h MTTR: 48 h MTTF: 6000 h MTTR: 12 h MTTF: 500 h MTTR: 1 h

16 Facility phases (1/2) Intake pumps Water intake from lakes/rivers Coagulation Chemicals bond contaminants Removal of phosphates, organic materials, viruses, bacteria and heavy metals Fast Sand filters Gravel and Sand filters Removal of organic materials, manganese and iron Ozonation Oxidation using Ozon Removal of organic materials, viruses and bacteria

17 Facility phases (2/2) Softeners Removal of calcium Slow Sand filters Slow moving water Remove last remaining bacteria Reservoir Temporary storage Pumps Distribute to customers

18 Disaster 1 (2) Intake Intake pumps pumps (2) Coagulation Coagulation (6) Fast Fast Sand Sand filters filters (2) Ozonation Ozonation MTTF: 800 h MTTR: 2 h MTTF: h MTTR: 6 h MTTF: 7000 h MTTR: 24 h MTTF: 5000 h MTTR: 3 h (3) (3) Slow Softeners Slow Softeners Sand Sand filters filters (1) Reservoir Reservoir (3+1) Pumps Pumps MTTF: 2000 h MTTR: 5 h MTTF: 8000 h MTTR: 48 h MTTF: 6000 h MTTR: 12 h MTTF: 500 h MTTR: 1 h

19 Results disaster 1 service level: Bronze Silver Gold

20 Results disaster 1 service level: Bronze Silver Gold

21 Results disaster 1 service level: Bronze Silver Gold

22 Disaster 2 (2) Intake Intake pumps pumps (2) Coagulation Coagulation (6) Fast Fast Sand Sand filters filters (2) Ozonation Ozonation MTTF: 800 h MTTR: 2 h MTTF: h MTTR: 6 h MTTF: 7000 h MTTR: 24 h MTTF: 5000 h MTTR: 3 h (3) (3) Slow Softeners Slow Softeners Sand Sand filters filters (1) Reservoir Reservoir (3+1) Pumps Pumps MTTF: 2000 h MTTR: 5 h MTTF: 8000 h MTTR: 48 h MTTF: 6000 h MTTR: 12 h MTTF: 500 h MTTR: 1 h

23 Results disaster 2 service level: Bronze Silver Gold

24 Results disaster 2 service level: Bronze Silver Gold

25 Results disaster 2 service level: Bronze Silver Gold

26 Conclusions and Future work Conclusions: LRF and LSIF perform equally well LSIF smart strategy it takes architecture into account BDR useful approximation technique Future work: Analyse influence of more repair crews Use/create CTMPD model checking Automate BDR and CSL reduction

27 The end... Questions?

28 L. Cloth and B.R. Haverkort. Model Checking for Survivability! In Proc. of QEST 2005, pages , Dutch Ministry of Internal Affairs. Raport bescherming vitale infrastructuur. Technical report, B.R. Haverkort, M. Kuntz, A. Remke, S. Roolvink, and M. I. A. Stoelinga. Evaluating repair strategies for a water-treatment facility using arcade. In The 40th Annual IEEE/IFIP International Conference on Dependable Systems & Networks (DSN 2010), Chicago, IL, USA, U.S. Department of Energy. 21 steps to improve cyber security of scada networks. Technical report,

Access Health CT Small Business

Access Health CT Small Business STEP 2 Horizontal Plan Selection: Bronze Metal Tier Your employer has chosen to offer the Horizontal Plan Selection. This option gives you access to all carriers and plans available in the Bronze Metal