Risk-informed decision-making processes

Transcription

1 Risk-informed decision-making processes An overview Enrico Zio and Nicola Pedroni n THEME Risk analysis

2

3 The Foundation for an Industrial Safety Culture (FonCSI) is a french public-interest research foundation created in It aims to: undertake and fund research activities that contribute to improving safety in hazardous organizations (industrial firms of all sizes, in all industrial sectors); work towards better mutual understanding between high-risk industries and civil society, aiming for a durable compromise and an open debate that covers all the dimensions of risk; foster the acculturation of all stakeholders to the questions, tradeoffs and problems related to risk and safety. In order to attain these objectives, the FonCSI works to bring together researchers from different scientific disciplines with other stakeholders concerned by industrial safety and the management of technological risk: companies, local government, trade unions, NGOs. We also attempt to build bridges between disciplines and to promote interaction and cross-pollination between engineering, sciences and the humanities. The work presented in this document is the result of research funded by the FonCSI. The opinions presented are those of the authors. Foundation for an Industrial Safety Culture A public-interest research foundation 6 allée Émile Monso BP Toulouse cedex 4 France contact@foncsi.org iii

4

5 Résumé Titre Panorama des processus décisionnels tenant compte du risque Mots-clefs incertitude, analyse de risque, prise de décision, arbitrage, RIDM Auteurs Enrico Zio et Nicola Pedroni Date de publication Décembre 2012 Les auteurs présentent les concepts généraux, définitions et enjeux de la mise en œuvre de processus décisionnels tenant compte du risque (mieux connus sous leur dénomination en anglais, Risk-informed decision-making ou RIDM). Il s agit de démarches structurées qui visent à aider les décisionnaires confrontés à des décisions complexes à fort enjeux, impliquant des objectifs multiples, en présence d incertitude. Elles visent à s assurer que le choix entre les alternatives soit fait en étant informé des risques de chaque option, et que l ensemble des attributs d une décision soient considérés dans un cadre intégré. Des motivations de l utilisation de ces techniques en complément d approches déterministes traditionnelles d analyse de risque sont fournies. Les processus de RIDM adoptés par la NASA et par la Nuclear Regulatory Commission, autorité de tutelle du nucléaire aux États Unis d Amérique, sont décrits en détail. Le document se termine par une analyse des similitudes et des différences d approche de mise en œuvre de ces démarches entre ces deux organismes. About the authors Enrico Zio est Professeur de Fiabilité et Analyse de Risque au Politecnico di Milano, et Directeur de la Chaire Systèmes complexes et défis énergétiques de l École Centrale Paris & Supelec. Il est également chairman du European Safety and Reliability Association (ESRA). Nicola Pedroni est maître de conférences au département Énergie du Politecnico di Milano. Sa recherche concerne les méthodes calculatoires avancées pour l évaluation de la sécurité des systèmes industriels, en présence d incertitude. To cite this document Zio et Pedroni (2012). Panorama des processus décisionnels tenant compte du risque. Numéro des Cahiers de la Sécurité Industrielle, Fondation pour une Culture de Sécurité Industrielle, Toulouse, France (ISSN ). Disponible à l adresse v

6 Abstract Title Overview of risk-informed decision-making processes Keywords risk, uncertainty, decision-making, RIDM, arbitration, PRA Authors Enrico Zio and Nicola Pedroni Publication date December 2012 The authors introduce the general concepts, definitions and issues related to the use of Risk-informed decisionmaking (RIDM). These are structured processes which assist decision-makers when faced with high impact, complex decisions involving multiple objectives and the presence of uncertainty. They aim to ensure that decisions between competing alternatives are taken with an awareness of the risks associated with each option, and that all attributes of a decision are considered in an integrated manner. Motivations for the use of these techniques as a complement to more traditional deterministic approaches to risk assessment are provided. The RIDM processes adopted by NASA and by the US Nuclear Regulatory Commission are described in detail, with an analysis of commonalities and differences in approach. About the authors Enrico Zio is Professor of Reliability, Safety and Risk Analysis at Politecnico di Milano and Director of the Chair in Complex Systems and the Energetic Challenge of École Centrale Paris & Supelec. He is also chairman of the European Safety and Reliability Association (ESRA). Nicola Pedroni is an associate professor in the Energy Department of the Politecnico di Milano. His research concerns advanced computational methods for the reliability assessment of industrial systems in presence of uncertainties. To cite this document Zio and Pedroni (2012). Overview of risk-informed decision-making processes. Number of the Cahiers de la Sécurité Industrielle, Foundation for an Industrial Safety Culture, Toulouse, France (ISSN ). Available at vi

7 Foreword The past two decades have seen an evolution from risk-based to risk-informed safety management approaches, in which quantitative outcomes of risk assessment are only one component of the decision-making process, being combined with other criteria (such as social preferences, political concerns and budgetary constraints). This change in the relationship between risk assessment and decision-making has been driven by several factors: Better awareness that real decisions (in the fields of industrial safety, environmental management, urban development) must integrate multiple concerns, and that outputs from risk assessment procedures often comprise significant uncertainty, and thus cannot be used mechanically to derive a well founded decision. Ongoing debate on the role of technical knowledge and expertise in public decisionmaking, in which the technical rationality of scientists and engineers is seen by some critics as disregarding the social context and citizens concerns (a cultural rationality ) in their analyses [Plough and Krimsky 1987]. This has led to greater stakeholder participation in decision-making, in which technocratic decision processes, driven purely by rational technical considerations, are modified to integrate the concerns and the perceptions of stakeholders. From a more technical viewpoint, a recognition that both deterministic approaches to risk assessment (which focus on the ability of engineering principles such as safety margins, redundancy and diversity to prevent and reduce the impact of catastrophic failures) and probabilistic methods (which integrate estimations of the likelihood of accident scenarios and which allow operators and regulators to determine which barriers would provide the most benefit in risk reduction) provide useful insights into safety management, and that a framework for combining their inputs is needed. The present document provides an introduction to risk-informed decision-making, along with a description of the manner in which the technique has been implemented by the nasa for management of risk in space programmes, and by the us nrc for the regulation of nuclear activities. The authors also document the respective advantages and disadvantages of deterministic and probabilistic approaches to risk assessment. Eric Marsden, FonCSI November 20, 2012 We welcome your feedback! Please send your comments on suggestions for improvement of this document by to cahiers@foncsi.org. vii

8

9 Contents 1 Introduction 1 2 Risk-informed decision-making Risk and risk assessment The Risk-Informed Decision-Making (RIDM) process NASA Risk-Informed Decision-Making process Part 1 Identification of decision alternatives Part 2 Risk analysis of decision alternatives Part 3 Risk-informed alternative selection usnrc Risk-Informed Decision-Making process Step 1 Define the plant safety issue to be addressed Step 2 Identify the applicable requirements and criteria Step 3 Evaluate how the plant safety issue affects the requirements Step 4 Weight the inputs from the assessments carried out Step 5 Make the decision Step 6 Implement the decision Step 7 Monitor the effect of the decision Conclusions 37 A Deriving performance measures in the NASA RIDM process 41 B Ordering the performance measures in the NASA RIDM process 43 C Establishing risk tolerances on the performance measures in the NASA RIDM process 45 Bibliography 47 ix

10

11 1 Introduction Context Although the use of risk assessment and uncertainty analysis for decision-making may take different perspectives, there is a shared and common understanding that these tools provide useful decision support in the sense that their outcomes inform the decision-makers insofar as the technical risk side of the problem is relevant for the decision [Aven 2010]. Further, the actual decision outcome for a critical situation involving a potential for large consequences typically derives from a thorough process which combines i) an analytic evaluation of the situation (i.e., the risk assessment) by rigorous, replicable methods evaluated under agreed protocols of an expert community and peer-reviewed to verify the assumptions underpinning the analysis, and ii) a deliberative group exercise in which all involved stakeholders and decision-makers collectively consider the decision issues, look into the arguments for their support, scrutinize the outcomes of the technical analysis and introduce all other values (e.g. social and political) not explicitly included in the technical analysis. This way of proceeding makes it possible to keep the technical analysis manageable by complementation with deliberation for ensuring coverage of the non-modelled issues. In this way, the analytic evaluation (i.e., the risk assessment) supports the deliberation by providing numerical outputs (point estimates and distributions of the relevant safety parameters, possibly to be compared with predefined numerical safety criteria for further guidance to the decision) and also all the argumentations behind the analysis itself, including the assumptions, hypotheses, parameters and their uncertainties [Nilsen and Aven 2003]. With respect to the latter issue, the key point is to guarantee that uncertainties are taken into account in each step of the risk assessment procedure whilst ensuring that the information and knowledge relevant for the problem are represented in the most faithful manner. In particular, uncertainties have to be 1. systematically identified and classified; 2. represented and described by rigorous mathematical approaches; 3. propagated through the steps of the risk assessment procedure onto the risk measures until the decisions. The bottom line concern with respect to uncertainty in decision-making is to provide the decision-makers with a clearly informed picture of the problem upon which they can confidently reason and deliberate [Zio 2009; Aven and Zio 2011]. For more than 30 years, probabilistic analysis has been used as the basis for the analytic process of risk assessment and the treatment of associated uncertainties. The common term used is Probabilistic Risk Assessment (pra, also referred to as Quantitative Risk Assessment, qra). Its first application to large technological systems (specifically nuclear power plants) dates back to the early 1970s [USNRC 1975], but the basic analysis principles have not changed significantly since that period. However, the purely probability-based approaches to risk and uncertainty analysis can be challenged under the common conditions of limited or poor knowledge on the high-consequence risk problem, for which the information available does not provide a strong basis for a specific probability assignment: in such a decision-making context, many stakeholders may not be satisfied with a probability assessment based on subjective judgments made by a group of analysts. In this view, a broader risk description is sought where all the uncertainties are 1

12 Overview of risk-informed decision-making processes laid out plain and flat with no additional information inserted in the analytic evaluation in the form of assumptions and hypotheses which cannot be proven right or wrong [Ferson and Ginzburg 1996; Walley 1991; Dempster 1967; Shafer 1976; Dubois and Prade 1988; Dubois 2006]. Notice that in the implementation of the decision it is common for decision-makers to seek for further protection by adding conservatisms and performing traditional engineering frameworks of defense-in-depth (typical of a deterministic approach to risk assessment) to bound the uncertainties and in particular the unknown unknowns (completeness uncertainty). In general, the insights provided by the probabilistic approach complement those provided by the deterministic approach. In view of this, the trend is to move towards a much more risk informed approach in which the insights from the risk information provided by the PRA is used formally as part of an integrated decision-making process. When this integrated process is applied to making decisions about safety issues, this is sometimes referred to as Risk Informed Decision-Making (ridm). Objectives of this document In this wide framework of decision-making in risk assessment practice in presence of uncertainties, the present document aims to: 1. introduce the general concepts, definitions and issues related to Risk-Informed Decision- Making (ridm) processes in presence of uncertainties; 2. describe in detail the ridm processes adopted by two organizations in the complex, safety-critical fields of aerospace and nuclear engineering, i.e., the National Aeronautics and Space Administration (nasa) and the United States Nuclear Regulatory Commission (usnrc). Document structure The document is structured as follows: In chapter 2, the definitions of risk and risk analysis are briefly recalled, after which the general concepts, motivations and issues related to Risk-Informed Decision-Making (ridm) processes are outlined; Chapters 3 and 4 present in detail how the ridm process is implemented respectively at nasa and by the usnrc; Annexes A to C provide detail on the derivation of performance measures, the ordering of performance measures and the establishment of risk tolerances on the performance measures in the nasa ridm process. 2

13 2 Risk-informed decision-making 2.1 Risk and risk assessment In the past, a deterministic approach was chosen as the basis for making decisions on safety issues. In particular, the approach was to: 1. identify a group of failure event sequences leading to credible worst-case accident scenarios {S i } called design-basis worst-case scenarios accidents; 2. predict their consequences {x Si }; 3. design appropriate safety barriers which prevent such scenarios and protect from, and mitigate, their associated consequences [Zio 2009]. Within this approach (often referred to as a structuralist defense-in-depth approach), safety margins against these scenarios are enforced through conservative regulations of system design and operation. These regulations operate under the assumption that the challenges and stresses caused to the system and its protections by any credible accident, are less than those caused by the worst-case, credible accidents. The underlying principle has been that if a system is designed to withstand all the worst-case credible accidents, then it is by definition protected against any credible accident [Apostolakis 2006]. However, in recent years, a probabilistic approach to risk analysis (PRA) has arisen as an effective way for analyzing system safety, not limited only to the consideration of worstcase accident scenarios but extended to examining all feasible scenarios and their related consequences, with the probability of occurrence of such scenarios becoming an additional key aspect to be quantified in order rationally and quantitatively to handle uncertainty. In particular, the move has been towards an integrated approach that combines the insights provided by the deterministic approach and those from the probabilistic approach with any other requirements in making decisions on a safety issue [USNRC 1975; NASA 2002; Aven 2003; Bedford and Cooke 2001; Henley and Kumamoto 1992; Kaplan and Garrick 1981; McCormick 1981; USNRC 1983]. safety margins Risk in PRA DEFINITION Within the Probabilistic Risk Assessment framework, risk is operationally defined as a set of triplets [USNRC 1983]: the scenario(s) leading to degraded performance with respect to one or more performance measures (e.g., scenarios leading to injury, fatality, destruction of key assets); the likelihood(s) (qualitative or quantitative) of those scenarios; the consequence(s) (qualitative or quantitative severity of the performance degradation) that would result if those scenarios were to occur. Uncertainties are included in the evaluation of likelihoods and consequences. Defining risk in this way supports risk management, because [NASA 2010]: the definition distinguishes high-probability, low-consequence outcomes from lowprobability, high-consequence outcomes; 3

14 Overview of risk-informed decision-making processes it points the way to proactive risk management controls, for example by supporting identification of risk drivers and the screening of low-probability, low-consequence outcomes; it can point the way to areas where investment is warranted to reduce uncertainty. The document [Zio and Pedroni 2012], available in the same collection as the present document, provides more information on the way in which uncertainty arises and can be managed in a PRA. 2.2 The Risk-Informed Decision-Making (RIDM) process In this section, general concepts related to Risk-Informed Decision-Making (ridm) processes are presented. In particular, the following questions are answered: What is ridm? ( 2.2.1) When is ridm invoked? ( 2.2.2) How does ridm help? ( 2.2.3) What is RIDM? A risk-based decision-making process provides a defensible basis for making decisions and helps to identify the greatest risks and prioritize efforts to minimize or eliminate them. It is based primarily on a narrow set of model-based risk metrics, and generally does not lead much space for interpretation. Considerations of cost, feasibility and stakeholder concerns are generally not a part of risk-based decision-making, which is typically conducted by technical experts, without public consultation or stakeholder involvement. In contrast, risk-informed decision-making (ridm) is a deliberative process that uses a set of performance measures, together with other considerations, to inform decision-making. The ridm process acknowledges that human judgment has a relevant role in decisions, and that technical information cannot be the unique basis for decision-making. This is because of inevitable gaps in the technical information, and also because decision-making is an intrinsically subjective, value-based task. In tackling complex decision-making problems involving multiple, competing objectives, the cumulative knowledge provided by experienced personnel is essential for integrating technical and nontechnical elements to produce dependable decisions [NASA 2008, 2010] When is RIDM invoked? ridm is invoked for key decisions (e.g., design decisions, make-buy decisions, budget reallocation, ), which typically require setting or rebaselining of requirements [NASA 2010]. It is invoked in many different venues, including boards and panels, safety review boards, risk reviews, engineering design decision forums and configuration management processes, among others [NASA 2010]. ridm is applicable for decisions that typically have one or more of the following characteristics [NASA 2010]: high financial stakes: significant costs and significant potential safety impacts are involved in the decision; complexity: the actual ramifications of alternatives are difficult to understand without detailed analysis; presence of uncertainty: uncertainty in key inputs creates substantial uncertainty in the outcome of the decision alternatives and points to risks that may need to be managed; multiple objectives: large numbers of objectives require detailed formal analyses; diversity of stakeholders: high accuracy is needed to define objectives and derive the corresponding performance measures when the set of stakeholders represents a wide variety of preferences and perspectives. 4

15 2.2. The Risk-Informed Decision-Making (RIDM) process ridm typically requires detailed, quantitative analyses, which are expensive to undertake, and is thus not suited to small, low budget projects which are in their operational phase How does RIDM help? ridm is a structured process that [NASA 2008]: aims at achieving project success by risk-informing the selection of decision alternatives; ensures that decisions between competing alternatives are taken with an awareness of the risks associated with each, thus helping to avoid late design changes, which can be relevant sources of risk, cost overshoot, schedule delays, and cancellation; tackles some of the following issues: namely, 1. the possible incongruence between stakeholder expectations and the resources required to address the risks to achieve those expectations; 2. possible misunderstanding of the risk that a decision-maker is accepting when making commitments to stakeholders; 3. the miscommunication in considering the respective risks associated with competing alternatives. tries to foster development of a robust technical basis for decision-making by: coupling the attributes of the proposed decision alternatives to the objectives that define project success; considering all attributes (that are important to the stakeholders) in an integrated manner; helping ensure that a broad spectrum of decision alternatives are considered; performing quantitative assessment of the advantages and drawbacks of each decision alternative relative to the identified objectives; taking into account the uncertainties related to each proposed decision alternative to quantify their impact on the achievement of the identified objectives; communicating the quantitative assessment of the proposed decision alternatives into the decision environment, where it is deliberated along with other considerations to form a comprehensive, risk-informed basis for alternative selection. The next chapter illustrates the manner in which ridm is used at nasa. 5

16

17 3 NASA Risk-Informed Decision-Making process Within the nasa organizational hierarchy [NASA 2010], top-level objectives (nasa Strategic Goals, such as mission success), flow down in the form of progressively more detailed performance requirements, whose achievement guarantees that the top-level objectives are met. Each operational/organizational unit within nasa discusses with the unit(s) at the next lower level in the hierarchy a series of objectives, performance measures and requirements, resources, and schedules that characterize the tasks to be performed by the unit(s). At each step, the lower level operational/organizational unit manages its own risks and reports risks and elevates decisions for managing risks to the next higher level. Employing the Risk-Informed Decision-Making (ridm) process in support of key decisions as requirements flow down through the operational/organizational hierarchy ensures that objectives remain intertwined to nasa Strategic Goals [NASA 2010]. Throughout this process, interactions take place between the following actors [NASA 2008]: 1. the stakeholders (i.e., individuals or organizations that are affected by the outcome of a decision but are outside the organization doing the work or making the decision); 2. the risk analysts (i.e., individuals or organizations that apply probabilistic methods to the quantification of risks and performances); 3. the subject matter experts (i.e., individuals or organizations with expertise in one or more topics within the decision domain of interest); 4. the Technical Authorities; 5. the decision-maker. Figure 3.1 illustrates these roles and interfaces within the nasa ridm process. It can be seen that it is fundamental that the analysts conducting the risk analysis of alternatives take into account the objectives of the various stakeholders in their analyses. These analyses are performed by subject matter experts in the domains spanned by the objectives. The completed risk analyses are deliberated and the decision-maker selects a decision alternative for implementation (with the agreement of the Technical Authorities). The nasa ridm process involving the above mentioned actors consists of three parts, namely Part 1, 2 and 3, briefly summarized below and described in detail in the following Sections 3.1, 3.2 and 3.3, respectively [Dezfuli et al. 2010; NASA 2008, 2010]: Part 1 Identification of decision alternatives Objectives are decomposed into their constituent objectives, each of which reflects an individual issue that is significant to the stakeholders. At the lowest level of decomposition are performance objectives, each of which is associated with a performance measure that quantifies the degree to which the performance objective is addressed by a given decision alternative. In general, a performance measure has a direction of goodness that indicates the direction of increasingly good performance measure values. A complete set of performance measures is considered for decision-making, that reflects stakeholder interests and spans the mission execution domains of i) safety, ii) technical, iii) cost and iv) schedule [NASA 2010]. Objectives whose performance measure values are required to lie within predefined limits (for all decision alternatives) give rise to imposed constraints performance measures 7

18 Overview of risk-informed decision-making processes stakeholders objectives risk analysts performance models subject matter experts (safety, technical, cost, schedule) objective values analysis results deliberation consultation contending alternatives Technical Authorities risk concurrence decision maker decision Figure 3.1 Roles and interfaces in the nasa ridm process reflecting those limits. Objectives and imposed constraints constitute the basis on which decision alternatives are compiled, and performance measures are the means by which their ability to meet imposed constraints and achieve objectives is quantified ( 3.1). Part 2 Risk analysis of decision alternatives The performance measures of each alternative are quantified; given the presence of uncertainty, the actual outcome of a particular decision alternative will be only one of a (possibly) broad spectrum. Therefore, it is incumbent on risk analysts to model all possible outcomes of interest, accounting for their probabilities of occurrence, in terms of the scenarios that produce it: this produces a probability distribution of outcomes for each alternative. If the uncertainty in one or more performance measures prevents the decision-maker from assessing important differences between alternatives, then the risk analysis may be iterated in order to reduce uncertainty. The iterative analysis stops when the level of uncertainty does not preclude a robust decision from being taken. Robust decision DEFINITION A robust decision is based on sufficient technical evidence and characterization of uncertainties to determine that the selected alternative best reflects the decision-maker s preferences and values given the state of knowledge at the time of the decision, and is considered insensitive to credible modeling perturbations and realistically foreseeable new information [NASA 2010]. TBfD The principal product of the risk analysis is the Technical Basis for Deliberation (TBfD), a document that lists the set of candidate alternatives, summarizes the analysis methodologies used to quantify the performance measures, and presents the results. The TBfD is the input that risk-informs the deliberations that support decision-making ( 3.2). Part 3 Risk-informed alternative selection Deliberation takes place among the stakeholders and the decision-maker who either i) prunes the set of alternatives and asks for further analysis of the remaining alternatives or ii) selects an alternative for implementation or iii) asks for new alternatives. To facilitate deliberation, a set of performance commitments is associated with each alternative. Performance commitments identify the performance that an alternative is capable of, at a given probability of exceedance, or risk tolerance. By establishing a risk tolerance for each performance measure independent of the alternative, comparisons of performance among the alternatives can be made on a risk-normalized basis. In this way, stakeholders and decision-makers can deliberate the performance differences between alternatives at common levels of risk, instead of having to choose between complex combinations of performance and risk ( 3.3). The nasa ridm process has just been described as a linear sequence of steps: however, 8

19 3.1. Part 1 Identification of decision alternatives this representation is used only for the sake of simplicity. Actually, in reality, some steps of the processes may be conducted in parallel, and others may be iterated multiple times before moving to subsequent steps. In particular, Part 2 (namely, Risk analysis of decision alternatives) is internally iterative as analyses are refined to meet decision needs; in addition, Part 2 is iterative with Part 3 (namely, Risk-informed alternative selection), as stakeholders and decision-makers iterate with the risk analysts in order to produce sufficient technical basis for taking a robust decision. The following sections provide details concerning Parts 1, 2 and 3 of this decision process. 3.1 Part 1 Identification of decision alternatives Decision alternatives have to be identified in the context of the objectives that have to be achieved. Thus, the identification of the alternatives starts with the process of understanding stakeholders expectations (Step 1.A, described in 3.1.1); then, stakeholders expectations are decomposed into quantifiable objectives and performance measures in order to allow comparison among the candidates (Step 1.B in 3.1.2); finally, a set of feasible alternatives is compiled that addresses the quantified objectives (Step 1.C in 3.1.3) Step 1.A Understand stakeholders expectations Stakeholder expectations result when they i) specify what is desired as a final outcome or as a thing to be produced and ii) establish bounds on the achievements of goals (these bounds may for example include costs, time to delivery, performance objectives, organizational needs). In other words, the stakeholder expectations that are the outputs of this step consist of i) top-level objectives and ii) imposed constraints. Top-level objectives state what the stakeholders want to achieve from the activity: these are frequently qualitative and multifaceted, reflecting competing sub-objectives (e.g., increase reliability vs. decrease cost). Imposed constraints represent the top-level success criteria outside of which the top-level objectives are not achieved. Planetary Science Mission example The hypothetical Planetary Science Mission example described in [NASA 2010], supposes that the objective of the ridm process is to place a scientific platform in orbit around a given planet in order to collect data and send it back to earth. Stakeholders expectations may include: the launch date must be within the next 55 months due to the launch window; the scientific platform should provide at least 6 months of data collection; the mission should be as inexpensive as possible, with a cost upper limit of $500 M; the probability of radiological contamination of the planet should be minimized, with a goal of no greater than 0.1% Step 1.B Derive performance measures Although the top-level objectives state that the goal has to be accomplished, they may be too complex and/or vague for operational purposes; thus, in general, decision alternatives cannot be directly compared only on the basis of the (multifaceted and/or qualitative) top-level objectives. To overcome this issue, top-level objectives are decomposed, using an Objective Hierarchy (OH), into a set of different lower-level objectives describing (in more detail) the complete set of necessary and/or desirable characteristics that any feasible alternative should have. Each of these lower-level objectives is then associated to a performance measure, that quantifies the extent to which a decision alternative meets the corresponding objective and, thus, provides a mathematical basis for comparing the different alternatives. 9

20 Overview of risk-informed decision-making processes Construct an Objective Hierarchy An Objective Hierarchy (OH) is built by subdividing an objective into lower-level objectives of more detail. Figure 3.2 shows an example OH. At the first level of decomposition the top-level objective of interest is declined into the general execution domains of Safety, Technical, Cost and Schedule. Below each of these general domains the objectives are further decomposed into sub-objectives, which themselves are iteratively decomposed until appropriate quantifiable performance objectives are generated. The decomposition of objectives stops when the set of performance objectives is operationally useful and quantifiable. Top-level objectives Top-Level O bjectives decomposition Mission execution domains Safety O bjective 1 Safety O bjectives Safety O bjective n 1 Technical O bjective 1 Technical O bjectives Technical O bjective n 2 C ost O bjective 1 C ost O bjectives C ost O bjective n 3 Schedule O bjective 1 Schedule O bjectives Schedule O bjective n 4 Quanti able performance objectives Safety S afety O bjective S afety O bjective O1,1 bjective 1,1 1,1 Safety Safety O bjective Safety O bjective no bjective 1 n,1 1,1 n 1,1 Technical Technical O bjective Technical O bjective O1,1 bjective 1,1 1,1 Technical Technical O bjective Technical O bjective no bjective 2 N,1 N 2,1 2,1 C ost C ost O bjective C ost O bjective O1,1 bjective 1,1 1,1 C ost Cost O bjective Cost O bjective no bjective 3 N,1 N 3,1 3,1 Schedule Schedule O bjective Schedule O bjective O1,1 bjective 1,1 1,1 Schedule Schedule O bjective Schedule O bjective no bjective 4 N,1 N 4,1 4,1 Figure 3.2 An example objective hierarchy Planetary Science Mission example OH By way of example, the figure below shows the objective hierarchy built from the top-level objective Project success for the Planetary Science Mission example introduced in [NASA 2010]. It can be seen that the top-level objective of interest, i.e., Project success, is successively decomposed through the general execution domains of Safety, Technical, Cost and Schedule, producing a set of performance objectives at the leaves. project success safety technical cost schedule maximize ground safety maximize the data collection duration minimize the cost meet the launch window minimize the probability of planet contamination maximize the types of data collected Figure 3.3 Example objective hierarchy for a nasa planetary science mission 10