Techneau, 10. December Risk Evaluation and Decision Support for Drinking Water Systems

Transcription

1 Techneau, 10. December 2010 Risk Evaluation and Decision Support for Drinking Water Systems

2 TECHNEAU Risk Evaluation and Decision Support for Drinking Water Systems 2010 TECHNEAU TECHNEAU is an Integrated Project Funded by the European Commission under the Sixth Framework Programme, Sustainable Development, Global Change and Ecosystems Thematic Priority Area (contractnumber ). All rights reserved. No part of this book may be reproduced, stored in a database or retrieval system, or published, in any form or in any way, electronically, mechanically, by print, photoprint, microfilm or any other means without prior written permission from the publisher

3 Colofon Title Risk Evaluation and Decision Support for Drinking Water Systems Authors Andreas Lindhe 1, Lars Rosén 1 and Per Hokstad 2 1 Chalmers University of Technology 2 SINTEF Quality Assurance By Thomas Pettersson, Chalmers University of Technology Deliverables number D4.2.5 and D4.4.2 This report is: PU = Public

4

5 Summary The vital importance of a reliable and safe drinking water supply makes efficient risk management necessary for water utilities. Risks must be assessed and possible risk-reduction measures evaluated to provide relevant decision support. The World Health Organization (WHO) emphasises the use of an integrated approach where the entire drinking water system, from source to tap, is considered when assessing and managing risks. This report provides a background to risk evaluation and decision support for managing risks in water utilities. A special focus is put on cost optimisation, and methods for cost-benefit analysis (CBA), cost-effectiveness analysis (CEA) and multi-criteria decision analysis (MCDA) of risk reduction alternatives are presented. A dynamic fault tree method is presented that enables quantitative, integrated risk assessment of drinking water systems. It is shown how the method can be used to evaluate uncertainties and provide information on risk levels, failure probabilities, failure rates and downtimes of the entire system and its subsystems. The fault tree method identifies where risk-reduction measures are needed most and different risk-reduction alternatives can be modelled, evaluated and compared. The method is combined with economic analysis to identify the most cost-effective risk-reduction alternative. Integrated risk assessments of drinking water systems are commonly performed using risk ranking, where the probability and consequence of undesired events are assessed using discretised scales. There is, however, no common, structured way of using risk ranking to prioritise risk-reduction measures. Two alternative models for risk-based, multi-criteria decision analysis (MCDA) for evaluating and comparing risk-reduction measures have therefore been developed. The MCDA models are based on risk ranking, they can consider uncertainty in estimates and include criteria related to, for example, different risk types and economic aspects. This report provides methods for integrated risk assessment that make it possible to evaluate risks and prioritise risk-reduction measures in an efficient way. This study also provides good examples of applications of these methods in Gothenburg, Sweden, Bergen, Norway and Březnice, Czech Republic. Based on the practical applications of these methods, it is concluded that the methods provide relevant decision support for efficient risk management in water utilities. v

6 vi

7 Contents Summary Contents v vii 1 Introduction Objective The TECHNEAU Risk Management Framework Notation Abbreviations 4 2 Risk evaluation Introduction Risk measures Qualitative measures of risk risk matrix and risk ranking Dimensions of risk Risk acceptance Risk Acceptance criteria (RAC) The two limit approach to risk acceptance: ALARP Target values The decision-making process Principles and criteria for risk evaluation How to establish a RAC Normative issues 12 3 Decision Support Methods developed in TECHNEAU A generic framework for decision support The dynamic fault tree method (DFT) Method development Failure types and conceptual model Logic gates and dynamic calculations Generic fault tree structure Risk and measure of risk Input data and uncertainties Case study Göteborg Key aspects when applying the fault tree method Quantitative risk assessment and economic analysis using DFT Modelling risk reduction Economic analysis of risk-reduction measures Case study Göteborg Key aspects when modelling and evaluating risk reduction Multi-criteria decision models General approach The discrete model The beta model 43 vii

8 3.4.4 Performance score and matrix Case study Bergen Case study Březnice Key aspects when applying the MCDA models 53 4 Dicussion and Conclusions Introduction Advantages and limitations of the methods developed The dynamic fault tree (DFT) method The MCDA models Communication and organisation Conclusions 58 5 References 61 Appendix 1: Equations for logic gates in dynamic fault tree analysis 65 Appendix 2: Target values and f-n curves an example from Trondheim 67 viii

9 1 Introduction 1.1 Objective The main objective of Work Area 4 (WA4 Risk Assessment and Risk Management) in TECHNEAU is to integrate risk assessments of the separate parts into a comprehensive decision support framework for cost-efficient risk management in safe and sustainable drinking water supply. Specific goals of WA4 are to provide tools and guiding documents to support the water utilities in their risk assessment and risk management work. A schematic illustration of the framework, guides and tools that is to be produced in WA4 is presented in Figure 1.1. Figure 1.1. A schematic illustration of the framework, guides and tools that will be produced in WA4. The report Generic Framework and Methods for Risk Management in Water Safety Plans (Rosén et al., 2007) forms the basis for WA4. It describes risk management on a general level and also provides an overview of risk analyses methods for water utilities. The report Methods for risk analysis of drinking water systems from source to tap (Hokstad et al., 2009) provides a guide on the use of risk analysis for drinking water systems. Lindhe et al. (2010c) present summarises the application of developed risk assessment methods in case studies. The report Decision support for risk management in drinking water supply (Rosén et al., 2010b) constitutes a literature review providing the background 1

10 to several topics relevant for decision-making and risk management of drinking water supplies. The present report treats risk evaluation and decision support methods for identifying the best alternative for risk reduction and control. A few case studies are also included. A crucial part of risk evaluation is for stakeholders to define limits for acceptable/tolerable risk. When a risk analysis has been carried out, the estimated risk should be evaluated and compared to the risk acceptance criteria (RAC) to decide whether the risk is tolerable. Further, risk evaluation includes the process of identification of risk-reduction measures and controlling risk during operation. The objective of this report is to provide a guide on these tasks. The main target group is management and personnel of water utilities with some basic knowledge/competence of risk management. 1.2 The TECHNEAU Risk Management Framework The TECHNEAU framework for integrated risk management is presented in Figure 1.2 (Rosén et al., 2007). The framework includes the following main components: Risk Analysis Risk Evaluation Risk Reduction/Control Risk Management Risk Assessment Risk Analysis Define scope Identify hazards Estimate risks Qualitative Quantitative Risk Evaluation Define tolerability criteria Water quality Water quantity Analyse risk-reduction options Ranking Cost-efficiency Cost-benefit Risk Reduction/ Control Make decisions Treat risks Monitor Acquire new information Update Analyse sensitivity Develop supporting programmes Document and assure quality Report and communicate Review, approve and audit Figure 1.2. The main components of the TECHNEAU generic framework for integrated risk management in WSP (after Rosén et al., 2007). 2

11 Various activities required for carrying out a risk analysis, risk evaluation and risk reduction/control are indicated in the rightmost box of Figure 1.2. The risk evaluation requires that a risk acceptance/tolerability criterion is defined (by the water utility). The estimated risk is then compared with this acceptance criterion in order to decide whether the risk is acceptable (tolerable) or not. Furthermore, various risk-reduction measures are considered to evaluate their cost-effectiveness, and to prioritise amongst various alternatives. Principles for this decision process, including normative issues, are the topic of the present report. 1.3 Notation The following notation and definitions of terms are applied in the TECHNEAU project: Hazard is a source of potential harm or a situation with a potential of harm. Hazardous agent is for example a biological, chemical, physical or radiological agent that has the potential to cause harm. Hazardous event is an event which can cause harm. Hazard identification is the process of recognizing that a hazard exists and defining its characteristics. Risk is a combination of the frequency, or probability, of occurrence and the consequence of a specified hazardous event. The total risk is given by aggregating the risk of the various hazardous events. Risk analysis is the systematic use of available information to identify hazards and to estimate the risk to individuals or populations, property or the environment. Risk estimation is the process used to produce a measure of the level of risk being analysed. Risk estimation consists of the following steps; frequency analysis, consequence analysis, and their integration. Risk evaluation is the process in which judgements are made on the tolerability of the risk on the basis of risk analysis and taking into account factors such as socio-economic and environmental aspects. Risk assessment is the overall process of risk analysis and risk evaluation. Risk management is the systematic application of management policies, procedures and practices to the tasks of analysing, evaluating and controlling risk. Risk measure is a quantitative measure of a specified risk, i.e. a quantified measure of the combination of probabilities/frequencies and consequences of a hazardous event. Risk-reduction measure is a preventing/detecting/controlling/ mitigating measure which has the effect of reducing (or eliminating) the probability and/or the consequences of an hazardous event, i.e. reducing the risk. 3

12 Risk reduction is the process where decisions are made regarding risk reducing measures; what needs to be done, by whom, when and at what cost. 1.4 Abbreviations ALARP CBA CCP CEA CER CML CRA DFT HACCP HAZID MCDA RAC RPN SSM THDB WHO WSP As Low As Reasonable Practicable Cost-Benefit Analysis Critical Control Points Cost-Effectiveness Analysis Cost-Effectiveness Ratio Customer Minutes Lost Coarse Risk Analysis Dynamic Fault Tree Hazard Analysis and Critical Control Points Hazard Identification Multi-Criteria Decision Analysis Risk Acceptance Criteria Risk Priority Number Substandard Supply Minutes TECHNEAU Hazard Database World Health Organization Water Safety Plan 4

13 2 Risk evaluation 2.1 Introduction When the risk analysis of a drinking water system has been carried out, a risk evaluation is performed to decide whether the identified risks are acceptable (tolerable) and what to do if there are unacceptable risks. The risks that are identified during a risk analysis can be measured (quantified) in various ways, and the risk acceptance criteria (RAC) will usually be based on the chosen risk measures. If the risk is found to be acceptable, it may be sufficient to control the risk rather than reducing it. However, if the risk is found to be unacceptable, various risk-reduction measures/options have to be analysed and compared to identify the best alternative. Thus, both the tolerability of the various risks and the costs, as well as additional criteria, of the risk reducing options will be considered during the risk evaluation. 2.2 Risk measures Due to the different aspects of risk that exist there are many ways to measure risk. However, the measures are usually based on the unwanted consequences and/or their probability. Examples on application of risk measures in risk evaluation are given in chapters and Qualitative measures of risk risk matrix and risk ranking In a Coarse Risk analysis (CRA) (Hokstad et al., 2009) we analyse various hazardous events, and the risk is given by the likelihood (probability/frequency), P, and the consequence, C, of the event. It is a very common approach if we do not have sufficient resources or information to carry out a full quantitative analysis of P and C to apply a classification of risks. Then probabilities and consequences are divided into categories. For the likelihood we use categories such as rare and frequent, and various consequences could be categorised e.g. as small, medium and catastrophic. These categories may represent a simple ranking of likelihood and consequences, or the categories can be properly defined. For instance, the probability category, rare could be defined as meaning less than once a month. Similarly, the consequence category small with respect to health effects could be defined as at most 10 consumers with minor health effects. This set (P, C), will be inserted in a risk matrix, see Figure 2.1. In this example there are four probability categories, ranging from P1 (rare) to P4 (very frequent), and four consequence categories, ranging from C1 (low) to C4 (catastrophic). Seven categories of risk are introduced, ranging from 1 ( very low ) to 7 ( very high ). Such a risk matrix is a common way to present risk. 5

14 In particular it is used to present the risk of the hazardous events identified in a CRA (Hokstad et al., 2009). C1 C2 C3 C4 P P P P Figure 2.1. Risk matrix with four categories of probability (P) and of consequence (C). Another example of a risk matrix adapted from (Hokstad et al., 2009) is illustrated in Figure 2.2. Again the two axes represent likelihood and severity of consequences, and definitions of the five likelihood and consequence categories are given. There are also defined four risk categories, L, M, H and E; ranging from L (low risk) to E (extreme risk). Severity of consequences Likelihood Insignificant Minor Moderate Major Catastrophic Almost certain M H H E E Likely M M H E E Moderate L M H H E Unlikely L L M H H Rare L L M H H Note: The number of categories should reflect the need of the study. E Extreme risk, immediate action required; H High risk, management attention needed; M Moderate risk, management responsibility must be specified; L Low risk, management by routine procedures. Examples of definitions of likelihood and severity categories that can be used in risk scoring Item Definition Likelihood categories Almost certain Likely Moderate Unlikely Rare Severity categories Catastrophic Major Moderate Minor Insignificant Once a day Once per week Once per month Once per year Once every 5 years Mortality expected from consuming water Morbidity expected from consuming water Major aesthetic impact possibly resulting in use of alternative but unsafe water sources Minor aesthetic impact causing dissatisfaction but not likely to lead to use of alternative less safe sources No detectable impact Figure 2.2. Example of a risk matrix and definitions of likelihood and severity categories to be used in risk scoring in WSP (adapted from Davison et al., 2005). Four classes of risk are shown. 6

15 This qualitative approach will provide a ranking of hazardous events, according to the corresponding risk category. Ranking can also be carried out based on the results of, for example, a Fault Tree Analysis (FTA). The various contributing causes ( basic events ) can be ranked according to their contribution to the top event (unwanted event) of the fault tree (Lindhe et al., 2009) Dimensions of risk The risk will have various dimensions (aspects), and for a water supply system there are two main dimensions: loss of quality (giving health effects), and loss of quantity (resulting in water unavailability/supply interruptions), If risk is measured by the use of a risk matrix, there should be one risk matrix for each dimension. In general, there are several ways to quantify the various dimensions of risk (see Chapter 4, Hokstad et al., 2009). A couple of examples related to loss of quality are Probability (mean number) of consumers receiving infected water during a year. Probability (mean number) of consumers being infected by drinking water during a year. DALY = Disability Adjusted Life Years, (cf. Appendix B of Hokstad et al., 2010). A couple of examples related to loss of quantity are Customer Minutes Lost (CML); the average number of minutes that drinking water is not delivered to an average consumer. Mean number of consumers affected by shortage of drinking water during a year. 2.3 Risk acceptance The main purpose of the risk evaluations is to decide whether or not the identified risks are acceptable (tolerable) or not. The risks are therefore compared to predefined risk acceptance criteria (RAC). A detailed description of risk tolerability in the context of drinking water is presented by Rosén et al. (2010b). In this section some of the most important concepts of RAC are presented Risk Acceptance criteria (RAC) The purpose of using RAC is to support the decision makers, and we note that RAC can be applied at different levels. 7

16 Various RAC at a "lower level could be related to risk tolerability regarding the use of specific equipment or processes. For instance, we could have a RAC giving an upper limit for the frequency of raw water contamination. Similarly there could be acceptance criteria related to the probability of failure of safety functions or treatment systems, or probability hazardous events of the utility. However, the risks of hazardous events can also be merged in order to give a measure of the overall total risk of the drinking water system. Then we may define a top level RAC for the overall risk. Such an RAC can be given as an upper limits for a risk measure like Mean number of consumers being infected by drinking water during a year or DALY = Disability Adjusted Life Year. In order to calculate such overall measures of safety (to check RAC) will require detailed analyses and they are often hard to estimate. The rationale behind using RAC in the decision process of risk evaluation could be to improve: 1. Risk control: Use of RAC should help to evaluate and control the undesired consequences of the planned activity to a level that is acceptable to the affected parties. 2. Efficiency of the decision process: Use of RAC should be an efficient way to structure the tasks of the decision process. Even if the RAC is tailored to the specific situation, it may not be necessary to repeat all arguments every time a risk is evaluated. The use of RAC should contribute to more focus and involvement regarding safety issues for the affected parties. Unfortunately the use of RAC could also lead to somewhat "automatic" decisions. So a possible problem with the use of RAC is that setting a target does not give drive for improvements beyond this level. That is, the creative process of finding even better solutions and measures is in practice limited to meeting the criteria. If that is the case, the use of RAC does not play an active role in the risk management process. This has caused some authors to advice against the use of RAC (Aven and Vinnem, 2005). However, applied in a proper way, the use of RAC in combination with other incentives would often prove useful for the decision process. This should be the case if use of RAC is combined with involvement and a drive for risk reduction The two limit approach to risk acceptance: ALARP When the risks related to a drinking water system (or a subsystem of it) are evaluated, we can apply the ALARP (As Low As Reasonably Practicable) principle to decide on risk acceptance, see Figure 2.3. The ALARP principle applies two acceptance limits. An upper acceptance limit is specified, meaning that the solution being analysed is definitely unacceptable if the risk 8

17 is above this limit, (see red area in figure), and in this case the risk must be reduced or eliminated. But a lower limit is also specified. Risks below this limit are considered acceptable and do not need to be further investigated (see green area). However, risks in between these two limits, in the so-called ALARP region, (see yellow area), should be investigated further and be reduced as far as reasonably practicable. This means that risk reducing measures should be investigated and their cost-effectiveness be evaluated. Unless a risk reducing measures is unreasonably expensive relative to its effect on the risk, it should be implemented. Thus, a systematic discussion to reduce risk should be carried out for any risk in the ALARP region. Unacceptable Risk The risk cannot be accepted under any circumstances ALARP Region The risk can be accepted if it is economically and technically unreasonable to reduce it Acceptable Risk Figure 2.3. The ALARP (As Low As Reasonably Practicable) Principle (Melchers, 2001). The borders (limits) between the three regions (red, yellow green) should be decided in a process prior to the actual risk analysis; cf. the decision-making process to define RAC, as discussed in the next section. Note that if a risk matrix is used to measure the risk, (Figure 2.2 and Figure 2.1), we can define three areas in the matrix, green, yellow and red; cf. Figure 2.4, and this can then represent an application of the ALARP principle. A risk falling in the red region is absolutely unacceptable, and a risk in the yellow region, must be investigated further to identify possible risk reducing measures of reasonable cost. Severity of consequences Likelihood Insignificant Minor Moderate Major Catastrophic Almost certain Likely Moderately likely Unlikely Rare Figure 2.4. Risk matrix with three regions according to the ALARP principle. 9

18 A principle, closely related to ALARP, and with the same meaning, is ALARA (As Low As Reasonably Achievable), see Davidsson et al. (2002) Target values The water Safety Plan refers to health-based targets and there are often politically established safety targets that can be regarded as acceptable levels of risk. For instance the City of Göteborg defined a safety target as: the duration of interruption in delivery to the average consumer shall, irrespective of the reason, be less than a total of 10 days in 100 years (Göteborg Vatten, 2006). Whether the risk is unacceptable or not was given by the probability of exceeding the target value. Similarly, Appendix 2 presents safety targets applied by the City of Trondheim (Norway). 2.4 The decision-making process The discussion on use of RAC cannot be separated from the relevant decision process, involving various stakeholders. Note that there are some principles that can assist in the formulation of suitable RAC; discussed below. Different stakeholders are in various ways and to a different degree involved in the risk management process. Note that stakeholders exposed to the risks are not always those benefiting from the risk generating activities. For example, industries in a catchment area of a water supply may benefit from their production, but they will also contribute to water safety risks to consumers, which do not benefit from the industrial activities. This is one of the normative issues in the risk evaluation decision process, cf. section Principles and criteria for risk evaluation Due to the multi-dimensional character of the decision-making for risk issues, it is of primary importance that the evaluation of risks and the decisionmaking are made with respect to criteria and principles that are agreed upon among the affected stakeholders. There are different principles for evaluation of risks, and these principles form the basis for defining risk tolerability, and they should be openly communicated and accepted by the involved stakeholders. Davidsson et al. (2002) present the following four general approaches that can be used when evaluating risk: - Principle of reasonableness If it is reasonable with respect to economical and technical means, the risk shall be reduced regardless the level of risk. - Principle of proportionality The overall risk resulting from an activity should not be unreasonably large compared to the benefits. - Principle of allocation The allocation of risk in society should be reasonable/fair compared to how the benefits are allocated. 10

19 - Principle of avoidance of disasters Risks with disastrous consequences should be avoided so that the consequences can be managed with accessible resources. Actual RAC should be formulated taking these principles into account. Renn ( 2008) mentioned that technical analyses of risk have drawn much criticism from the social science. One reason to this is that the technical analyses not are considered to include people s perception of risk and social constructions. Klinke and Renn (2002) present nine criteria to be used for evaluating risk. These criteria are meant to include more than just the extent of damage and probability of occurrence when evaluating risks. The nine criteria are: - Extent of damage - Probability of occurrence - Incertitude - Ubiquity - Persistency - Reversibility - Delay effect - Violation of equity - Potential of mobilization How to establish a RAC In the outset it may not be obvious which risks are tolerable, and the decision process can benefit from a documented line of arguments, e.g. by comparisons to existing risk levels. This could promote consistency in various decisions. Various principles exist to decide on the actual limit between "acceptable and non-acceptable risk. The following are two general principles to assist in attaining a numerical limit for acceptance. 1. "The Comparison criteria (e.g. NORSOK Z-13 (NORSOK, 2001)) is essentially the same as the French GAMAB ( Globalement Au Moins Aussi Bon ) principle. This is primarily used when non-standard solutions, e.g. new technology, are to be implemented. Then the acceptance will require that the solution shall give at least as low risk as the presently accepted practice/solution. In general the Comparison Criteria seem the most helpful approach by modifications of systems, e.g. by introduction of new technology, and when new utilities shall be built. 2. "The Additional risk" criteria, which can be seen as a version of the (German) MEM (Minimum Endogenous Mortality) principle. Roughly speaking, this principle starts from an existing basic risk. Then a new activity shall not significantly increase this. By specifying such an underlying 11

20 basic risk, we are assisted in also specifying a RAC. We can require that the increase in risk due to an (increase in a specific) activity shall be less than a certain percentage of the basic risk. In general, the following are useful input, when the actual limit of a RAC shall be specified: - Historical risk data and acceptability of risk in similar activities; (i.e. we utilise accumulated knowledge) - Assessment of perceived risk of stakeholders, - Willingness to accept the risks by involved parties, see below. The risk tolerability levels must be defined taking people s perception and aversion of risks into consideration. The public perception has for example been found to have an important affect on the priorities and legislative agendas of regulatory bodies (Slovic, 2001). Examples on factors affecting peoples risk aversion are: - Catastrophic potential - Familiarity - Uncertainty - Individual or societal - Controllability - Voluntariness Finally, when a RAC shall be decided, one should have the ambition to achieve continuous risk reduction, and one must be aware of the various ethical challenges, see Section Normative issues The risk evaluation has obvious ethical aspects, and the decision process will benefit from including normative discussions, (e.g. regarding the choice of the RAC): Which risks can we actually tolerate? The following are main normative issues in the decision process of a water utility, cf. Hokstad et al. (2009): - Which dimensions (aspects) of risk shall be evaluated? Shall decision makers restrict to consider water quality and water quantity? Should special/additional attention be given e.g. to major accidents or environmental issues? - What are the preferences and trade-offs between the various dimensions of risk (as water quality and water quantity)? That is, when we know the costs of two risk reducing measures, which of them should be given priority? And could different stakeholders/consumers be treated differently, etc.? - How shall we arrive at a RAC (the actual acceptance limits) for various risks? 12

21 It is obvious that a discussion is needed to define the dimensions of risk to evaluate. For example, shall we only deal with average risk values for the total population, or do we focus on high risk groups (Stallen et al., 1996). Another topic is the question of public s perceived risks (fears). To what extend shall that be taken into consideration? Sometimes it is also an issue to achieve a fair distribution of risk amongst various parties affected (Hokstad and Vatn, 2008). So a RAC-approach should have an "ethical foundation", securing that safety is not compromised. Use of RAC should be seen as a means to reduce risk, and management commitment is essential in the process. 13