Flood Risk Analysis considering 2 types of uncertainty

Transcription

1 US Army Corps of Engineers Institute for Water Resources Hydrologic Engineering Center Flood Risk Analysis considering 2 types of uncertainty Beth Faber, PhD, PE Hydrologic Engineering Center (HEC) US Army Corps of Engineers

2 Flood Risk Management The US Army Corps of Engineers has a mission in flood control, now flood risk management or reduction change in terminology either follows or attempts to instigate a change in thinking traditionally, attempted to reduce flooding, but now focus on reducing flood risk risk = likelihood & consequence For considering new projects, analysis is economic. invest Federal dollars to decrease flood damages by greater amount For existing projects, analysis is about safety, and includes potential life loss

3 Risk Analysis In deterministic analysis, we look in detail at the damages caused by a some large flood events. To do a stochastic RISK ANALYSIS, we must also consider the likelihood of those events happening. Corps guidelines for risk analysis require both likelihood of occurrence of damaging flood events, and uncertainty in our estimates and modeling. RISK & UNCERTAINTY ANALYSIS We describe both likelihood and uncertainty with probability distributions.

4 Concepts What is the difference between natural variability (aleatory) and knowledge uncertainty (epistemic)? Do the differences matter in decision making? If yes How can we separately consider them in risk analysis computations and decision metrics? What happens if we do not consider them separately How can we estimate and describe them? How can decisions best consider them?

5 Two Types of Uncertainty Natural Variability (Aleatory) = some variables are random and unpredictable by nature, and their values differ event to event or place to place Knowledge Uncertainty (Epistemic) = some variables are more or less constant, but we do not know their values accurately Both variability and uncertainty are described by probability distributions weir coefficient

6 Outline Expected Annual Damage (EAD) what it is, how it s been computed other decision-making metrics Monte Carlo Simulation event sampling / modeling Natural Variability and Knowledge Uncertainty definitions, how they affect EAD how we sample them and when performance indices reducing compute time

7 Outline Expected Annual Damage (EAD) what it is, how it s been computed other decision-making metrics Monte Carlo Simulation event sampling / modeling Natural Variability and Knowledge Uncertainty definitions, how they affect EAD how we sample them and when performance indices reducing compute time

8 Decision Making, Metrics For new investment, Cost / Benefit analysis is primary Cost is the expense of building and maintaining a structure, or of changes to the damage potential of the flood plain Benefit is the reduction in flood damages over time Spending Federal dollars, so need investment to have positive expected cost/benefit ratio, and for portfolio needs to be positive on average use mean values Local decision-making is different

9 Expected Annual flood Damage (EAD) The metric we evaluate is an average annual damage expected value is the mean or average of a probability distribution Expected Annual Damage can be interpreted as the average damage over a very long period of time. This annualized value can be compared to an equivalent annual cost in cost/benefit analysis. benefits of project = reduction in EAD The old way of computing EAD was to condense the flood frequency information, the hydraulics, and the economics into summary relationships, and combine them. Corps of Engineers IWR-HEC

10 Expected Annual Flood Damage Flood Damage expected annual damage years Corps of Engineers IWR-HEC

11 Summary Curves for Frequency, Hydraulics and Economics Peak Flow (cfs) Hydrology Flow-Frequency Hydraulics Stage-Flow CDF % Economics Stage-Damage 1 Exceedance Prob 0 Probability distribution of annual peak flow Flow (cfs) 1 Probability per unit PDF Area = 1 Variable Value Cumulative Probability CDF Variable Value Corps of Engineers IWR-HEC

12 Summary Curves for Frequency, Hydraulics and Economics Peak Flow (cfs) Hydrology Flow-Frequency Hydraulics Stage-Flow CDF Economics Stage-Damage 1 Exceedance Prob 0 Probability distribution of annual peak flow Flow (cfs) Variable Value Area = 1 PDF Variable Value CDF Probability per Unit Exceedance Probability 12 Corps of Engineers IWR-HEC

13 Computing EAD with summary curves The mean (expected value) of annual flood damage is computed by combining summary curves: flow-frequency curve to obtain a: stage-flow function damage-frequency curve stage-damage function The mean of the damage-frequency function is the expected value of annual damage, or EAD. Corps of Engineers IWR-HEC

14 Computing EAD with Summary Curves Peak Flow (cfs) 2 1 CDF Flow-Frequency Curve captures year-to-year variability in flow Stage (ft) 1 p 0 Exceedance Probability 3 Corps of Engineers IWR-HEC CDF 1 p 0 Exceedance Probability captures year-to-year variability in damage AREA = mean = expected annual damage, EAD E[D] N i 1 D i p

15 Other Decision Metrics Annual Exceedance Probability the likelihood of flood impact in any year we re familiar with the National Flood Insurance Program s 100-year (1% chance) base flood based primarily on natural variability Assurance of 1% protection of interest to local sponsor used for levee certification chance that have AEP 1%, given uncertainty based primarily on knowledge uncertainty Dollars per statistical life saved, etc currently, willingness to pay is 9.1 million$ per DOT, once below tolerable risk guidelines

16 Outline Expected Annual Damage (EAD) how it s been computed other decision-making metrics Monte Carlo Simulation Event sampling / modeling Natural Variability and Knowledge Uncertainty definitions, how they affect EAD how we sample them and when performance indices reducing compute time

17 Monte Carlo Simulation We re interested in variable Y=damage, which is a complex function of X=flow, ie, Y = g(x) X is a random variable, described by a probability distribution So, Y is also a random variable with a probability distribution How do we determine the distribution of Y=damage? variable X If distribution of X is known, can develop the distribution of Y analytically, or can use Monte Carlo Simulation probability / X PDF

18 Monte Carlo Analysis Replace the probability distribution of variable X=flow with a very large sample of values Relative Frequency PDF of X Value of X histogram Then, for each member of the sample, compute Y=g(X) This process creates a large sample of the variable Y (damage)

19 Monte Carlo Analysis Relative Frequency From the generated sample of Y (damage), infer its probability distribution with statistical analysis In the case of Y=damage, we distribution of Y have been mostly interested in the mean of the distribution, Value of Y or EAD

20 Why Monte Carlo? One value of Monte Carlo simulation is the ability to use complex deterministic models It is easier to do math on a member of a sample than on the probability distribution itself Can operate on (or evaluate functions of) members of the sample, then recombine the resulting sample into a new distribution

21 Slightly More Complex When variable Y is a function of 2 random variables, X and Z Create a sample of variable X Create a sample of variable Z (if X and Z are correlated, need a correlated sample) Compute Y = h(x, Z) for every pair of X and Z

22 Generating the Sample How do we generate a sample of values from a particular probability distribution? First, switch from a PDF to a CDF, ie cumulative probability probability / X f(x) probability that less than X CDF: F(x) = P[X<x] 1 0 F(x) variable X variable X

23 Generating the Sample Generate pseudo-random values, uniform U i ~ U[0,1] random number generators usually produce U[0,1] Use U i as the cumulative probability, and compute x i as the inverse of the CDF of X F(X) 1 U i = F(x i ), x i = F -1 (U i ) U i CDF, F(X) A frequency analysis on the sample x i provides the original probability distribution, ie P[X x] 0 x. i X

24 How large a sample? sam ple size = 100 sam ple size = sam ple size = 1000 the input sample is large enough when its statistics reproduce the parameters of the distribution the output sample is large enough when the statistics of interest stabilize

25 Computing EAD with Summary Curves Peak Flow (cfs) CDF Flow-Frequency Curve captures year-to-year variability in flow Stage (ft) 1 p 0 Exceedance Probability Corps of Engineers IWR-HEC CDF 1 p 0 Exceedance Probability captures year-to-year variability in damage AREA = expected annual damage, EAD

26 Monte Carlo Simulation for Flood Risk The peak flow frequency curve is the primary source of natural variability in annual flood damages In Monte Carlo Simulation (Analysis), we replace a probability distribution with a very large sample of values from that distribution we can then deterministically model each member of the sample to compute damage creates a large sample of damages Can consider many distributions at the same time, but we ll start by looking at just peak flow variability

27 Computing EAD by Event Sampling Simple Monte Carlo Simulation Peak Discharge (cfs) CDF replace flowfrequency curve with a sample Stage (ft) One Event (sample member) 1 0 Exceedance Probability end with a sample of damages EAD 1 N N i 1 Damage(i) Corps of Engineers IWR-HEC

28 Replacing Flow-Frequency Curve with a Large Sample of Peak Flows 1000 events (annual peak flows, or annual max X-duration flows) PDF form (histogram) CDF form (frequency curve) peak annual flow (cfs) Peak Annual Flow (cfs) each point is an event events are ranked and plotted against relative frequency of exceedance (plotting positions) count Exceedance Probability

29 From each flow, compute a damage, create a sample of damages 1000 event damages -- the average or mean of these is the EAD Event Damage (1000$) PDF form (histogram) Annual Flood Event Damage (1000$) CDF form (frequency curve) each point is an event count Exceedance Probability

30 How many is enough? Convergence We continue creating and evaluating new events until the statistic of interest stabilizes average damage 1% exceedance damage, This is convergence variable X probability / X avg 1%

31 How many is enough? Convergence We continue creating and evaluating new events until the statistic of interest stabilizes average damage 1% exceedance damage, This is convergence probability / X avg 1% variable X Average Damage average of damage % exceedance damage events 100 events 0 1% exceedance damage

32 How many is enough? Convergence We continue creating and evaluating new events until the statistic of interest stabilizes average damage probability / X 1% exceedance damage, This is convergence avg 1% variable X % exceedance damage average of damage 1% exceedance damage 1000 events 1000 events Average Damage

33 Using models rather than summary curves Simple Monte Carlo Simulation One Event (sample member) Peak Discharge (cfs) CDF 1 0 Exceedance Probability replace flowfrequency curve with a sample EAD 1 N N i 1 Damage(i) end with a sample of damages Corps of Engineers IWR-HEC

34 Outline Expected Annual Damage (EAD) how it s been computed other decision-making metrics Monte Carlo Simulation Event sampling / modeling Natural Variability and Knowledge Uncertainty definitions, how they affect EAD how we sample them and when performance indices reducing compute time

35 Variability and Uncertainty Natural Variability (Aleatory) = some variables are random and unpredictable by nature, and their values differ event to event or place to place Knowledge Uncertainty (Epistemic) = some variables are more or less constant, but we do not know those values accurately Both variability and uncertainty are described by probability distributions PDF weir coefficient

36 How do these affect EAD? We estimate average damage (EAD) because the natural variability in flooding prevents us from knowing what future damages will be Natural Variability: All random variables that vary event-to-event or vary spatially are captured within the distribution of damage, and so in the mean damage flood magnitude, forecasts, channel roughness mean = EAD annual damage PDF annual damage

37 How do these affect EAD? Knowledge Uncertainty: Watershed parameters that we do not know exactly introduce uncertainty into the damage distribution and so into the mean damage flood likelihood, hydraulic coefficients, channel capacities This uncertainty creates a probability distribution of EAD EAD distribution annual damage

38 Including Uncertainty in the EAD computation So far, the Monte Carlo simulation we looked at sampled only natural variability from the flood frequency relationship We need to include uncertainty in the sampling and modeling to include it in the evaluation of EAD In the flood frequency relationship, the uncertainty stems from sampling error, which is the error from estimating probabilities from a small sample

39 Computing EAD with Summary Curves no uncertainty considered Peak Flow (cfs) CDF only capture natural variability Stage (ft) 1 0 Exceedance Probability Corps of Engineers IWR-HEC 1 0 Exceedance Probability AREA = expected annual damage, EAD

40 How do we capture knowledge uncertainty in MC event modeling? Nested Monte Carlo: A. Sample instances of natural variabilities as flood events, with enough events to capture the distribution of damage. B. Sample instances of knowledge uncertainties in model parameters for each realization of the damage distribution. 1 outer loop B = a realization A B inner loop A varies natural variability, computes EAD outer loop B varies knowledge uncertainty, computes EAD distribution

41 Sampling Variability and Uncertainty Nested Monte Carlo Simulation Peak Flow (cfs) CDF Sample new frequency curve (uncertainty) and then sample events (variability) sample uncertain model parameters sample variabilities Corps of Engineers IWR-HEC One Event (sample member) 1 0 Exceedance Probability For each realization, get an EAD estimate: N 1 EAD Damage(i) N i 1 still end with One a sample of Realization damages After repeating for many realizations:

42 Relative Frequency 0 250, , ,000 1,000,000 1,250,000 1,500,000 1,750,000 2,000,000 2,250,000 2,500,000 2,750,000 3,000,000 3,250,000 3,500,000 3,750,000 4,000,000 4,250,000 4,500,000 4,750,000 5,000,000 5,250,000 5,500,000 5,750,000 6,000, Annual Damage ($) Relative Frequency 0 250, , ,000 1,000,000 1,250,000 1,500,000 1,750,000 2,000,000 2,250,000 2,500,000 2,750,000 3,000,000 3,250,000 3,500,000 3,750,000 4,000,000 4,250,000 4,500,000 4,750,000 5,000,000 5,250,000 5,500,000 5,750,000 6,000,000 sample of annual damage from one realization (spans natural variability) provides 1 estimate of EAD sample of mean damage (EAD) from all realizations (spans knowledge uncertainty) provides distribution of EAD mean = average = EAD Annual Damages 100 realizations EAD estimates Average Damage (EAD) $

43 Model Parameters Variability and Uncertainty Hydrologic Frequency Natural Variability Annual Maximum Flow (flood frequency curve) Snowmelt forecasting Knowledge Uncertainty Flood frequency curve parameters

44 Model Parameters Variability and Uncertainty Reservoir Modeling Natural Variability Starting Storage/Elevation Demands (water, power) Current Power Capacity (outages) Sedimentation changes Knowledge Uncertainty Stream routing coefficients Reservoir physical data: storage/elevation, release capacity, etc

45 Model Parameters Variability and Uncertainty Channel Backward Routing Natural Variability Knowledge Uncertainty Manning s n Weir Coefficients Bridge Debris Gate Coefficients Ice thickness Bridge/culvert coefficients Dam/levee breeching Manning s n parameters Contraction/Expansion coefficients Boundary Conditions Terrain Data

46 Model Parameters Variability and Uncertainty Floodplain Damage Natural Variability Structure value Content Value Car Value Other Value Depth/Damage functions Fatality Rates Mobilization Curve Knowledge Uncertainty Foundation Height Ground Elevation Note, many of these are captured as spatial variability rather than uncertainty

47 How many realizations? Optimally, until convergence The number of realizations needed to define the resulting distribution depends on its use 100 realizations 10,000 realizations average EAD Relative Frequency 300, , , , , , ,000 1,000,000 1,100,000 1,200,000 1,300,000 1,400,000 1,500,000 1,600,000 1,700,000 1,800,000 1,900,000 2,000,000 2,100,000 2,200,000 2,300,000 Average Damage (EAD) $ 100 realizations , , , , , , ,000 1,000,000 1,100,000 1,200,000 1,300,000 1,400,000 1,500,000 1,600,000 1,700,000 1,800,000 1,900,000 2,000,000 2,100,000 2,200,000 2,300,000 Relative Frequency EAD distribution average EAD Average Damage (EAD) $ 10,000 realizations EAD distribution

48 Using the Sample of EAD Probability / $ P=10% that EAD < EAD 10 estimated probability distribution histogram P=10% that EAD > EAD 90 Corps of Engineers IWR-HEC EAD 10 Expected Annual Damage ($) Mean EAD EAD 90

49 What can I do with this? If also have a probability distribution of cost (because cost is also uncertain)...can consider cost and benefit to compute: probability B/C ratio is less than 1 probability Net Benefit is less than 0 49Corps of Engineers IWR-HEC

50 Net Benefit Distributions for 2 Projects Probability Density P (NB<0) = 9% P (NB<0) = 27% Project 2 Mean NB = $1 million Project 1 Mean NB = $3 million Net Benefit (million $) 50Corps of Engineers IWR-HEC

51 Other metrics AEP, Assurance, LTEP AEP = Annual Exceedance Probability (variability) = percent of events that exceed certain stage = percent of events that get given structure wet Like EAD, get 1 estimate of AEP in every realization After all realizations, have AEP distribution Assurance = likelihood that AEP is less than a specified value (uncertainty) Relative Frequency distribution of uncertainty in AEP 100 realizations AEP of stage of interest (61') Assurance: 78% chance AEP 1%

52 Other metrics AEP, Assurance, LTEP Long-term Exceedance Probability (LTEP) (formerly called long-term RISK ) = the likelihood of exceeding a stage or getting wet at least once in N years (estimate with binomial distr) LTEP = 1 (1 AEP) N The chance of exceeding the 1% event (100-yr) at least once in 30 years is: LTEP = 1 (1 -.01) 30 = 26% The chance of exceeding the 5% event (20-yr) at least once in 30 years is: LTEP = 1 (1 -.01) 30 = 79%

53 Computational Effort There are at least two methods planned for managing the computation effort of running the system models for 100s or 1,000s of events. 1. Distributed Computing Different instances of the life cycle or realization can be run on different computers, and results returned 100 computers reduces time to 1% 2. Intelligent / Importance Sampling (selective compute) Not all events can cause flooding. Events with no chance of causing damage are not computed Might run only 2% to 3% of events.

54 Summary To evaluate flood damage for a project life, must consider Natural Variabilties in flooding the watershed and Knowledge Uncertainties in our modeling of flooding of the watershed. Both variabilities and uncertainties are described with probability distributions Monte Carlo analysis lets us replace probability distributions with large samples from those distributions event sampling Variability is captured in EAD and AEP, Uncertainty is captured in the distribution of EAD and in Assurance

55 Questions Do the differences between natural variability and knowledge uncertainty matter in decision making? If yes How can we separately consider them in risk analysis computations? What happens if we do not consider them separately? How can we estimate and describe them? How can decisions best consider them?