Managing Uncertainty and Risk in Travel Forecasting: A White Paper

Transcription

1 Managing Uncertainty and Risk in Travel Frecasting: A White Paper Octber 2013 FHWA-HEP

2

3 Managing Uncertainty and Risk in Travel Frecasting Table f Cntents 1.0 Intrductin Imprving Accuracy f Frecasts Acknwledging Frecast Uncertainties Risk Listing and Assessment f Prir Experience Generate Alternative Frecasts by Independent Parties Quantifying Frecast Uncertainty Limitatins f Uncertainty Assessment Managing Frecast Risks Cnclusin...18 List f Figures Figure 1. Stated causes f inaccuracies in traffic frecasts Figure 2. Example Traffic Distributin Figure 3. Cyclical Nature f the Risk Management Prcess Figure 4. Prbability Impact Matrix List f Tables Table 1. Example Risk Listing (Part I) Table 2. Example Risk Listing (Part II)... 8 Table 3. Tll Rad Perfrmance by Categry i

4

5 Managing Uncertainty and Risk in Travel Frecasting 1.0 Intrductin Travel frecasts are an imprtant element f the prcess that transprtatin agencies use t plan and implement plicies and investment prgrams t accmplish bjectives related t mbility. Agency gals typically invlve prviding fast, safe, efficient transprtatin while minimizing negative impacts t the envirnment r the public. The achievement f these gals is cnstrained by finite financial and ther resurces. Travel frecasts culd make significant cntributins t the prcess by which plicies are analyzed and prjects selected fr develpment. Travel mdels attempt t quantify the speed and cnvenience f the transprtatin system and its resulting usage key elements f system efficiency and envirnmental r scial impacts. In the case f revenue prjectins fr tll facilities r transit lines, frecasts can als cntribute t an assessment f financial feasibility f new r existing systems. Unfrtunately, the ptential usefulness f travel prjectins has nt been fully realized. A series f studies have cmpared frecasted and actual usage and cncluded that frecasts frequently exceed actual demand, ften by significant amunts. A number f suggestins have been made regarding hw the frecasting prcess culd be imprved. These include: Imprve versight r gvernance. Examine the distributin f utcmes frm a reference class f past prjects. Quantify the likely distributin f utcmes. Be aware f the cnsiderable ptential fr errr and bias t influence prjectins. All f these have a rle in imprving the accuracy f the frecasts. Hwever, as the last pint suggests, frecasting accuracy can never be assumed. Bth frecasters and users must always be aware f the cnsiderable uncertainties that surrund all travel prjectins and the chance that estimated usage levels will nt be achieved if a particular plicy r prgram is implemented. This paper describes the uncertainties and risks assciated with travel frecasts and suggests methds fr their management. There are many ways f defining risk and uncertainty. In this paper, these terms are used accrding t the Internatinal Standard ISO (2009) definitin f risk: effect f uncertainty n bjectives. By this definitin, risk has tw elements a prbability that an event will r will nt ccur (uncertainty), and a cnsequence (the impact n an rganizatin s bjectives). While errrs can be detected and reslved, uncertainty and risk cannt be avided, nly managed. Risk and reward are ften linked, with higher returns being generated frm pprtunities that present higher levels f risk. In transprtatin, mst prgrams r prjects include uncertainties in csts, benefits, and revenues which must be brne in rder t btain the intended mbility utcmes. The key is nt t blindly avid all risks but rather t manage the prgram f prjects s that risks are balanced with expected benefits. The remainder f this paper discusses the fllwing: Techniques fr imprving the accuracy f frecasts. Appraches fr quantifying uncertainties. Strategies fr managing frecast risks. 1

6 Managing Uncertainty and Risk in Travel Frecasting 2.0 Imprving Accuracy f Frecasts Travel frecasts have been an integral element f the transprtatin planning prcess fr ver 50 years. During that time frame, they have been used t help plan and justify the Interstate Highway System, majr fixed guideway transit facilities, tll rads, and varius plicy initiatives. These systems have imprved mbility fr many Americans and the planning prcess deserves much credit fr these successes. Althugh transprtatin prgrams and plicies have generally succeeded in imprving mbility and minimizing impacts, the rle f frecasts has been relatively mdest. This is partly because frecasts are nly ne element f a large and cmplex prcess t define and implement prjects and prgrams. Anther factr cntributing t its mdest rle is the fact that frecasts are nt perceived as being sufficiently accurate t be a decisive element f the decisin-making prcess. Retrspective reviews f the accuracy f travel frecasts cmpared t actual usage patterns cnfirm these perceptins. Several studies f frecasted versus actual travel demand have fund that prjectins f usage typically misstate actual vlumes t a cnsiderable degree. Examples include: In 1992, Pickrell cmpared frecasted bserved and actual daily ridership fr seven United States rail transit systems and fund that nly ne system (Washingtn Metr) carries ver half f the frecasted ridership. 1 Actual ridership fr the remaining six cities were under half f the frecasted levels. In 2005, Flyvbjerg, Hlm, and Buhl reprted that internatinal rail frecasts were verestimated by an average f 105.6% and that 84% f rail prjects have actual traffic levels mre than 20% belw frecasted. Internatinal rad prjects did better, underestimating actual traffic by 8.7%. Even s, ne half f all rad prjects have errrs exceeding ±20% and ne quarter have errrs that exceed ±40%. 2 Lewis-Wrkman and White reprted that nly three f 19 United States fixed-guideway transit prjects cmpleted between 1990 and 2002 carry mre riders than prjected. Anther five were carrying, r might sn carry, at least 80% f the frecasted number. The remaining 11 were well belw the predicted value. Of the 18 prjects built between 2003 and 2007, eight were carrying at least 80% f the frecasted ridership and 10 were well belw the predicted value. 3 Bain studied 104 internatinal tll rad prjects and fund that, n average, these radways carried 77% f the frecasted ridership, with the wrst attracting nly 14% f the frecasted ridership. 4 1 Pickrell, Dn H. A Desire Named Streetcar, Fantasy and Fact in Rail Transit Planning. Jurnal f the American Planning Assciatin, Vl. 58, Number 2, Spring Flyvbjerg, Hme, and Buhl. Hw (In)accurate Are Demand Frecasts in Public Wrks Prjects? Jurnal f the American Planning Assciatin, Vl. 71, N. 2, Spring Federal Transit Administratin with Supprt frm Vanasse Hangen Brustlin, Inc. The Predicted and Actual Impacts f New Starts Prjects April Bain, Rbert. Errr and ptimism bias in tll rad traffic frecasts. Transprtatin, 28 February

7 Managing Uncertainty and Risk in Travel Frecasting Each authr has attempted t diagnse the surces f frecast inaccuracies. Flyvbjerg s assessment f the causes fr frecast errrs presented in Figure 3 is typical and shws that inaccuracies are related t prblems with input assumptins and the mdels (r mdel cmpnents) that are used t generate frecasts. A substantial number f prjectins were als deliberately slanted t achieve a predetermined utcme Percentage f Prjects Rail Rad 0 Surce: Flyvbjerg Figure 1. Stated causes f inaccuracies in traffic frecasts. Lemp and Kckleman have cmpiled a list the key drivers f frecast failures fr tll rads frm multiple surces. 5 The reasns that frecasts ften misstate actual usage include: Optimism bias; Prly estimated values f time; Reliance n a single value f time; Mis-predictin f future land-use cnditins, verall trip generatin, and trip destinatin; Mis-estimatin f travel time savings; Unfreseen cmpetitin prvided by parallel free rads; and Underestimatin f ramp-up duratin and severity. 5 Lemp, Jasn and Kckelman, Kaa. Understanding and Accmmdating Risk and Uncertainty in Tll Rad Prjects. Presented at TRB, January

8 Managing Uncertainty and Risk in Travel Frecasting Failure t understand truck travel r time-f-day/day-f-year variatins These findings suggest that sme imprvements are pssible thrugh careful design, develpment, and applicatin f frecasting tls. A cmpanin paper t this dcument, Imprving Existing Travel Frecasting Mdels and Prcesses, describes ptential imprvements t the practice f travel frecasting and the accuracy f frecasts. These appraches include the fllwing: Imprve the practice f frecasting Develp an awareness f frecast requirements Remember the limitatins f frecasting tls Cnsider alternative techniques Adpt a sequential frecasting prtcl Understand and dcument frecast uncertainties Cnduct peer reviews Cllect better data Demgraphic infrmatin Transprtatin supply Usage cunts Travel demand patterns Data validatin Cnfirm reliability f input assumptins Imprve capabilities f existing frecasting tls Refine gegraphic units f analysis Stratify mdels accrding t time f day Stratify mdels by sciecnmic class Maintain cnsistency amng all frecasting elements Develp effective highway and transit travel time functins Run highway assignment mdels t full cnvergence Cnfirm mdel validity Review accuracy f mdeled transprtatin supply Cnfirm gegraphic distributin f travel Cnfirm mdel chice patterns Cnfirm mdel respnse t change A cmmn theme f each these steps is that a successful set f frecasts is the prduct f a persn wh utilizes knwledge f current cnditins, tls (such as mdels) t shw what may happen, and judgment t develp the actual prjectins f future travel demand and impacts n mbility. Each aspect f this prcess is imprtant successful prjectins cannt be develped 4

9 Managing Uncertainty and Risk in Travel Frecasting withut data, effective analysis prcedures, and judgment. In each, the persn respnsible fr assembling prjectins plays a key rle. The frecaster must actively review each element f the prcess t cnfirm that the data are accurate and the results make sense. Prjectins that have nt been scrutinized by the frecaster and by independent reviewers are just raw numbers. It is the cmbinatin f internal and external review that transfrms these numbers int useful insights. While these actins will undubtedly imprve the frecasting prcess, Pickrell, Flyvbjerg, and Bain all suggest that imprvements t the frecasting prcess, alne, are unlikely t lead t significant imprvements in frecasting accuracy. Instead, substantial inaccuracies appear t be inevitable, particularly when attempting t understand travel behavirs ccurring 30 years int the future r examining usage f a new facility fr which n histry is available. Suggestins fr imprving utcmes include the fllwing: Bring the frecasting hrizn clser t the present t reduce the ptential factrs that can cause prjectins t g awry. These factrs include changes in the lcal ecnmy, the evlutin f travel patterns, r cmpeting transprtatin mdes. Shrt-term frecasts wuld als remve the effect f prjectins f reduced autmbile speeds and increased csts, which have rarely prven accurate. [Pickrell] Apply reference class frecasting appraches t reduce inaccuracy and bias. These tls take an utside view f the prject being frecasted based n usage experience frm similar prjects that are already pen. [Flybjerg] Acknwledge uncertainty. Frecast errr is unlikely t be eliminated by technical changes in the way mdels are develped and applied. Given this situatin, frecasters shuld cmmunicate the uncertainty f prjectins and the financial and plitical risks that accmpany these prjectins. It is pssible t cnstruct frecasts with a mathematical prbability that it falls within a stated range. Hwever, it is mre valuable t acknwledge that uncertainty exists and cannt be eliminated. [Pickrell] The next sectins describe the prcess that can be used t acknwledge frecast uncertainties and risk. 5

10 Managing Uncertainty and Risk in Travel Frecasting 3.0 Acknwledging Frecast Uncertainties One f the recmmendatins frm Sectin 2.0 is acknwledging the existence f uncertainty and, if pssible, quantifying its ptential impacts. This sectin discusses three ptential appraches: Develping a risk listing that enumerates prject attributes and uses experience with similar prjects t estimate frecast likelihd. Preparing independent frecasts f travel demand. Quantifying frecast uncertainties. 3.1 Risk Listing and Assessment f Prir Experience As Pickrell nted, it is imprtant t understand and acknwledge frecasting risks. One technique is t utilize the risk listing apprach described by Bain. 6 This apprach (illustrated in Table 1 and Table 2) invlves cmparing the characteristics f a ptential prject r prgram t a series f characteristics that prir experience has shwn t be related t relatively lw traffic risk (lw risk scres) r higher traffic risks (high risk scres). This cmparisn can be presented t decisin makers t prvide insights int the likelihd that actual usage will r will nt match frecasted values. It is imprtant t nte nearly all f the prject characteristics that are indicative f higher risk levels relate t the prject circumstances rather than the methdlgy used t frecast demand. These circumstances include Being a new facility rather than an extensin r mdificatin t an existing rad; Being in an area with little lcal experience with tll rads; Having a cmplex r relatively expensive tll rate structure; Being in an area with a relatively weak lcal ecnmy; Lacking prtectin frm cmpetitin; and Being dependent n new crridr develpment r grwth. The nly aspects that relate t the frecasting methdlgy pertain t the apprach used t survey ptential users and whether mdel parameters are lcally derived r imprted frm elsewhere. 6 Bain, Rbert. Errr and ptimism bias in tll rad traffic frecasts. Transprtatin, 28 February

11 Managing Uncertainty and Risk in Travel Frecasting Table 1. Example Risk Listing (Part I). Prject Attributes Gd Traffic Risk Index: Scres Bad Tlling culture Tll rads well established data n actual use available N tll rads in the cuntry uncertainty ver tll acceptance Tariff escalatin Flexible rate setting/escalatin frmula; n gvernment apprval All tariff hikes require regulatry apprval Frecast hrizn Near-term frecasts required Lng-term (30 years+) frecasts required Tll facility details Facility already pen Facility at the very earliest states f planning Surveys/data cllectin Estuarial crssing Radial crridr int urban area Extensin f existing rad Alignment strng ratinale (tlling pints and intersectins) Alignment strng ecnmics Stand-alne (single) facility Highly cngested crridr Few cmpeting rads Clear cmpetitive advantage Only highway cmpetitin Gd, high capacity cnnectrs Active cmpetitin prtectin (e.g., traffic calming, truck bans) Easy t cllect Experienced surveyrs Up-t-date Lcally calibrated parameters Dense urban netwrk Ring-rad/beltway arund urban area Greenfield site Cnfused/unclear rad bjectives (nt where peple want t g) Alignment strng plitics Reliance n ther prpsed highway imprvements Limited/n cngestin Many alternative rutes Weak cmpetitive advantage Multi-mdal cmpetitin Hurry-up and wait Autnmus authrities can d what they want Difficult/dangerus t cllect N culture f data cllectin Histrical infrmatin Parameters imprted frm elsewhere (anther cuntry?) Existing zne framewrk Develp framewrk frm scratch Surce: Bain, Rbert. Errr and ptimism bias in tll rad traffic frecasts. Transprtatin, 28 February

12 Managing Uncertainty and Risk in Travel Frecasting Table 2. Example Risk Listing (Part II) Prject Attributes Gd Traffic Risk Index: Scres Bad Users: private Clear market segments Unclear market segments Few, key rigins & destinatins Dminated by a single jurney purpse (e.g., cmmute, airprt) High incme, time sensitive market Tlls in line with existing facilities Simple tll structure Flat demand prfile (time-f-day, day-f-week, etc.) Multiple rigins & destinatins Multiple jurney purpses Average/lw incme market Tlls higher than nrm (extended ramp up)? Cmplex tll structure (discunts, frequent users, variable pricing) Highly seasnal r peaky demand prfile Users: cmmercial Fleet peratr pays tll Owner-driver pays tll Clear time/perating cst savings Simple rute chice decisin-making Strng cmpliance with weight restrictins Unclear cmpetitive advantage Cmplicated rute chice decisin-making Overlading f trucks is cmmnplace Micr-ecnmics Strng, stable diversified lcal ecnmy Weak/transitining lcal/natinal ecnmy Traffic Grwth Strict land-use planning regime Stable, predictable ppulatin grwth Driven by/crrelated with existing, established and predictable factrs High car wnership Weak planning cntrls/enfrcement Ppulatin grwth dependent n many exgenus factrs Reliance n future factrs, develpments, structural changes etc. Lw/grwing car wnership Surce: Bain, Rbert. Errr and ptimism bias in tll rad traffic frecasts. Transprtatin, 28 February

13 Managing Uncertainty and Risk in Travel Frecasting A similar message is presented in NCHRP Synthesis This reprt included a cmparisn f frecasted and actual tll revenues fr 26 prjects pened between 1986 and Cmparisns are presented fr the first 5 years f peratin (fr prjects where this data was available). Of the 105 bservatins, 13 were within ±10%. Of the remaining bservatins, nly 10% were cases where revenue was underestimated. The remainder represent cases where actual revenue is less than 90% f frecasted revenue. Over half f all rads generated 30% less revenue than frecasted and a quarter generated 50% less revenue than frecasted. These prjects were categrized int fur grups and the relatinships between frecast errr and grup were identified. The results f this analysis are presented in Table 3. In general, frecasts fr prjects in an already-develped area with relatively mdest tll rates were reasnably accurate. Prjects that depend n high-develpment grwth rates, new travel patterns, r high tlls were cnsiderably less accurate, with actual traffic being between 50% t 70% f frecasted vlumes. 7 Kriger, David; Shiu, Suzette; and Naylr, Sasha. Estimating Tll Rad Demand and Revenue. Natinal Cperative Highway Research Prgram Synthesis 364,

14 Managing Uncertainty and Risk in Travel Frecasting Table 3. Tll Rad Perfrmance by Categry. Authrity/Facility Characteristics Perfrmance Explanatin Grup 1 High cngestin, suburban Three facilities: Apprximated State Rad and Tllway Authrity (GA)/GA r exceeded 400 prjectins Nrth Texas Tllway Authrity/Gerge Bush Expressway Illinis State Tll Highway Authrity/Illinis Nrth Suth Tllway Grup 2 - Outlying Seven facilities: Oklahma Turnpike Authrity/Jhn Kilpatrick Oklahma Turnpike Authrity/Creek Flrida s Turnpike Enterprise/Veteran s Expressway Flrida s Turnpike Enterprise/Seminle Expressway Flrida s Turnpike Enterprise/Plk Transprtatin Crridr Agencies (CA)/Fthill Nrth Orland Orange Expressway Authrity/Central Flrida Greenway Nrth Segment Grup 3. Develped Crridrs Five facilities: Harris Cunty Tll Rad Authrity (TX)/Hardy Harris Cunty Tll Rad Authrity (TX)/Sam Hustn Transprtatin Crridr Agencies (CA)/Fthill Eastern Transprtatin Crridr Agencies (CA)/San Jaquin Hills Santa Rsa Bay Bridge Authrity (FL)/Garcn Pint Bridge Grup 4. Least Develped Eight facilities: E-470 Public Highway Authrity (CO)/E-470 Tll Rad Investment Partnership (VA)/Dulles Greenway Oscela Cunty (FL)/ Oscela Cunty Parkway Orland Orange Expressway Authrity (FL)/Central Flrida Greenway Suth Segment Orland Orange Expressway Authrity (FL)/SR-417 Flrida s Turnpike Enterprise/Sawgrass Expressway Pcahntas Parkway Assciatin (VA)/ Pcahntas Parkway Cnnectr 2000 Assciatin (SC)/ Greenville Cnnectr Well-develped urban/suburban part f large metrplitan area Higher crridr incme Substantial crridr Traffic High value f time Gd cnnectins t Facility N cmpetitive nn-tlled alternatives Mdest prjected traffic grwth Less established traffic patterns Less integral t the existing netwrk These were partial beltways Usually serving abve average incme areas, but with less established develpment patterns Further frm emplyment centers Mderate-t-high tll rates (althugh usage inelastic because drivers already accustmed t paying tlls Crridrs with mre develped r already established traffic patterns Usually cnstructed in large metrplitan areas r active turist areas Slid prjected time savings Mderate prjected revenue grwth Specific traffic generatr serving as prject basis (e.g., airprt) Lcated in undevelped area Tll rad expected t stimulate develpment High revenue grwth rates Assumed peridic tll rate increases Mean ranged between 61% and 67% f frecasts, n average, with cnsiderable variatin Mean ranged between 51% and 60% f frecasts, n average, with cnsiderable variatin Mean ranged between 61% and 67% f frecasts, n average, with cnsiderable variatin Mderate tll rates Very rapid adjustment f traffic patterns fllwing pening Mderate traffic grwth in first 2 3 years, then grwing mre slwly Substantial frecast revenue grwth (35% average ver first 4 years) Frecast errr appears t result frm verestimatin f initial base perid usage (high ramp-up rates) Impacts f nearby nn-tlled alternatives underestimated Overestimated time savings Overly ptimistic ecnmic frecasts Failure t accunt fr recessins Overestimated crridr grwth rates High tll rates Limited histry f tll use in area Unusual ramp-up prblems Expansin f cmpeting nntlled netwrk Substantial frecast revenue grwth (35% average ver first 4 years) Frecast errr appears t result frm verestimatin f initial base perid usage (high ramp-up rates) Surce: Muller and Bun reprted in NCHRP Synthesis 364 Estimating Tll Rad Demand and Revenue 10

15 Managing Uncertainty and Risk in Travel Frecasting Many f these risk factrs apply t situatins beynd tll rad frecasting. Fr instance, the Federal Transit Administratin (FTA) als views frecasts f current-year cnditins t be mre reliable than frecasts with lng-term hrizns. FTA typically limits frecast assumptins regarding autmbile perating csts and parking fees t current values t avid the uncertainties assciated with the future magnitude and impact f these factrs n transit ridership. The factrs that are assciated with frecast accuracy culd easily be incrprated int a statement f frecast uncertainties. Prject characteristics that are knwn t be related t past frecasting prblems shuld be disclsed t frecast users. It is imprtant t emphasize that these uncertainties are nt frecasting deficiencies that can be addressed by better mdels r mre data, but rather are an intrinsic characteristic f the prject. Such statements shuld nt suggest that a prject with uncertain frecasts is necessarily ne that shuld nt be pursued. There are many cases in histry where business decisins were made withut a full knwledge f the market acceptance f a new prduct. Sme f these prducts, in fact, have paved the way fr a cmpany s success. Instead, uncertainty suggests the need t plan fr a range f utcmes s that the prject r rganizatin des nt fail if demand were t be less (r mre) than frecasted. 3.2 Generate Alternative Frecasts by Independent Parties and Independent Means Sectin 2.0 suggests that frecasts might becme mre reliable by using a brader array f tls (e.g., reference class mdels r a variety f cnventinal frecasting methds) and tw r mre independent grups f frecasters. This apprach wuld reduce the chance that frecasts are results f mdel inaccuracies r frecaster ptimism bias. If the frecasters are truly independent f the prject, it wuld als reduce the pressure t slant a frecast t meet a particular utcme. Independent frecasts invlve engaging multiple grups f frecasters t prepare prjectins. Ideally, these frecasting teams might share infrmatin that describe the prject and, pssibly survey data, but therwise wuld prepare frecasts with very little interactin n the nature f their analysis methds r input assumptins. After bth parties cmplete their wrk, utcmes culd be cmpared t identify cmmn utcmes and differences. These results culd prvide key insights int ptential uncertainties that culd be particularly helpful in cases that are judged t be High Risk, as defined in Sectin 4.0. A versin f this apprach was been used in the develpment f high-speed rail frecasts fr the State f Flrida. FTA has als used independent reference class mdels t evaluate sme New Starts frecasts. It is imprtant fr these different frecasts t be disclsed s that decisin makers and the public have a mre cmplete understanding f the range f ptential utcmes. Different alternative frecasts shuld nt be characterized as being right r wrng. Instead similarities r differences shuld be used t highlight what is relatively certain and what is unknwn. 3.3 Quantifying Frecast Uncertainty Althugh a listing f risks and independent review can be helpful, in many cases it is necessary t quantify frecast uncertainties in a mre meaningful way. Several appraches have been used: 11

16 Managing Uncertainty and Risk in Travel Frecasting Cnduct sensitivity analysis in which individual mdel inputs and assumptins are varied t determine the effect n frecast utcmes. These tests are helpful in demnstrating the mdels respnse t input assumptins and the ptential variability f utcmes. Since input values are individually adjusted and are arbitrarily set, these tests d nt attempt t estimate the likelihd f ccurrence. Prepare a series f scenaris that describe bth ptimistic and pessimistic cnditins. These stress tests help t establish the best and wrst expected utcmes but prvide n indicatin f the prbability that any particular utcme will ccur. Cnduct a quantitative risk analysis in which prbability distributins fr key mdel inputs and assumptins are prepared. This can be dne by applying Mnte Carl techniques t generate thusands f different input sets and then running the frecasting prcedures t frecast demand fr each case. Because travel frecasting mdels can run fr hurs r days, Adler et al., suggest using Respnse Surface Analysis techniques t develp a simplified equivalent f the full travel frecasting mdel fr use in the Mnte Carl simulatin 8. With this technique, a simple equatin is used t generate an estimate f the key figure f merit (traffic) as a functin f the input variables that are the key drivers f uncertainty. Adler et al. prvided an example applicatin f this technique t the Orland I-4 Express Lanes prject that includes the fllwing steps: 1. Begin with a detailed macrscpic simulatin mdel and travel demand mdel and cnduct sensitivity analysis t identify input variables that are key drivers f frecast utcmes. In Orland, these included ppulatin grwth, value f time, ther netwrk capacity imprvements, and tll rates. With the exceptin f tll rates, which the rad s peratr sets, these represent the mst imprtant input uncertainties. 2. Develp prbability assumptins fr each f the three uncertain variables based n: (a) accuracy f past ppulatin frecasts and likelihd f ecnmic dwnturns; (b) surveys and sampling errrs assciated with value f time; and (c) likelihd f ther system enhancements estimated by DOT engineers. 3. Accunt fr cvariance amng input variables using histrical data (when knwn) r Delphi techniques (when nt knwn). 4. Run the travel frecasting mdel fr nine different cmbinatins f key inputs fr each f seven frecast years representing an array f values f time, grwth, netwrks, and tll rates. 5. Develp a simple equatin t estimate traffic and revenue as a functin f the uncertain variables. Traffic = C 1 + C 2 grwth + C 3 tllrate + C VOT 4 ln(rampup) + C 5 radec + C 6 Rad150 + yearcn) Where: Traffic = Number f daily ne-way trips using express lanes Grwth = Rati f dwelling units in frecast year t dwelling units in 2010 minus 1 tllrate rampup = Average tll rate in Year 2010 dllars = Number f years prject has been in peratin 8 Adler, Thmas; Dherty, Michael; Kldzinski, Jack; and Tillman, Raymnd. Methds fr Quantitative Risk Analysis fr Travel Demand Mdel Frecasts Paper submitted t Transprtatin Research Bard, August 1,

17 Managing Uncertainty and Risk in Travel Frecasting radec = Rad imprvements assciated with E+C cnditins rad150 = Rad imprvements required t maintain all rads belw V/C f 1.5 yearcn = Vectr f cnstants representing the year 6. Use Mnte Carl simulatin t draw between 100,000 and 1,000,000 sets f inputs and cmpute traffic and revenue fr each case t develp a prbability distributin f likely utcmes. The utcme f this prcess is a prbability distributin f traffic and revenue similar t that shwn in Figure 2. This chart shws bth the median value f traffic and the traffic vlume that will be exceeded with an estimated prbability f 75% Cumulative Prbability th Percentile=110,000 Median=120, Year 2045 Traffic Surce: Adler et al. Figure 2. Example Traffic Distributin. 3.4 Limitatins f Uncertainty Assessment Use f tls such as risk listings, sensitivity and scenari analyses, and quantified risk analyses help frecasters and decisin makers understand the dimensins f uncertainty that surrund each prject r prgram. It is imprtant, hwever, t remember that while they help t understand uncertainty, they als have limitatins that must be understd. These limitatins include the fllwing: Risk analysis tls generally d nt address prblems assciated with the structure and peratin f the travel demand mdel, itself. Travel mdels are very cmplex systems with many interacting elements. Careful and thrugh mdel testing can nly begin t establish the validity f travel frecasting mdels. Frequently, cmbinatins f cnditins can ccur in the future that have n parallel in bserved data used t develp the mdel. As a cnsequence, the mdel sensitivity t these cnditins cannt be verified by cmparing results t actual travel demand. Instead, the mdel can nly be tested fr 13

18 Managing Uncertainty and Risk in Travel Frecasting reasnableness, which leaves pen the pssibility that mdel results will nt match future utcmes. The prbability distributins f key input variables are largely unknwn. Past frecast experience, survey standard errrs, and prfessinal experience are gd indicatins f input uncertainty. Hwever, the future is unknwn and may r may nt resemble past experience. Therefre, the prbability f input cnditins nt being achieved may be materially different frm the assumptins used in a Mnte Carls analysis. Cvariance between variables is particularly imprtant and must nt be bypassed. Emplyment, incme, and willingness t pay can be affected simultaneusly by an ecnmic dwnturn. These variables shuld nt be treated as three separate parameters with independent distributins. All f these factrs mean that, as with the frecasts themselves, the actual uncertainty assciated with thse frecasted vlumes may differ frm the predictins assciated with any f the tls presented in this sectin. 14

19 Managing Uncertainty and Risk in Travel Frecasting 4.0 Managing Frecast Risks In many cases, frecasts f travel demand have an imprtant impact n the decisin fr public and private entities t invest in transprtatin infrastructure. The cnsequence f an errneus (lw) frecast ranges frm a sub-ptimal set f public transprtatin investments t a failure t meet an rganizatin s financial bligatins. Risk is als present if the actual demand is higher than frecasted. Fr example, critical pieces f infrastructure may be undersized if the demand is underestimated, requiring the wner t make expensive upgrades. The prcess f understanding uncertainties and managing their impact is becming increasingly imprtant in a wide array f rganizatins, including transprtatin agencies. A full discussin f risk management is beynd the scpe f this dcument, but it is intrduced in a reprt by Janet D Ignazi fr NCHRP Internatinal experience in this subject is presented in a reprt prepared by Curtis et al. fr FHWA. 10 Risk management ccurs at multiple levels in an rganizatin the enterprise, the prgram, and the prject. As shwn in Figure 3, the prcess is iterative and repeated ver time. Mnitring and Review Identificatin Mitigatin and Planning Assessment Analysis Surce D Ignazi, et al. Figure 3. Cyclical Nature f the Risk Management Prcess. The first step is risk identificatin, which is the prcess f determining which risks (r underlying uncertainties) might affect an rganizatin s bjectives. This is mst ften accmplished with brainstrming sessins r survey techniques using an experienced, qualified, and diverse team. Identified risks are stred in a risk register that rganizes this knwledge and initiates later steps f the prcess. Risks are stated in terms f their impact n the rganizatin s bjectives. Example frecasting-related risks that might be stred in a risk register include Pr selectin f prjects undermining the rganizatin s credibility and Inadequate revenue t 9 D Ignazi, Janet; Hallwell, Matthew; Mlenaar, Keith. Executive Strategies fr Risk Management by State Departments f Transprtatin. NCHRP 20-24(74). May Curtis, Jyce et. al. Transprtatin Risk Management: Internatinal Practices fr Prgram Develpment and Prject Delivery. FHWA Office f Internatinal Prgrams. August

20 Managing Uncertainty and Risk in Travel Frecasting supprt perating r financing csts. Actual demand nt equaling frecasted usage is nt, in and f itself, a risk; hwever, the cnsequences f bad frecasts n an rganizatin s reputatin, financial integrity, r achievement f mbility gals culd be risks. The next step is risk assessment, which invlves defining bth the likelihd and cnsequences f the risk. A prbability-impact matrix (see example in Figure 4) can priritize risks in a qualitative manner. If there is a high prbability that a given frecasted traffic vlume will nt be achieved, but there is relatively little impact t the rganizatin, then the verall risk is Lw. This culd happen if a new (nn-tlled) radway is built befre the area is develped with the bjective f reducing cnstructin csts r accelerating ecnmic develpment. As lng as the develpment and traffic eventually ccur, there may be limited risk assciated with actual vlumes being lwer than frecasted in the shrt term. By cntrast, if the radway were designed t be tlled and financed using revenue bnds, then the same frecast errr culd lead t a high chance f inadequate cash t cver the required payments and a serius cnsequence financial default. This situatin wuld have a High risk assessment. Likelihd VL L M H VH M M H H H Assessment Guide L M M H H H = Unacceptable. Different apprach L L M M H required. L L L M M M = Sme impact. Additinal manage- L L L L M ment attentin needed. VL L M H VH L = Minimal impact. Mnitr t ensure Impact risk remains lw. Surce: D Ignazi, et al. Figure 4. Prbability Impact Matrix. Risks with a high r medium assessment may warrant a mre cmplete risk analysis. Risk analysis can utilize the frecast uncertainties analysis discussed in 3.0. Histric experience with certain types f frecasts (Sectin 3.1), independent frecasts (Sectin 3.2) and mre detailed sensitivity and Mnte Carl analyses (Sectin 3.3) are useful in understanding the prbabilities that certain risks will ccur. The analysis must fcus n the risks (cnsequences) rather than individual uncertainties. Fr instance, if the risk is inadequate cash flw t cver debt service, then the prbability f bth revenue and perating csts being different frm the frecast must be evaluated, since either culd lead t financial difficulty. The next step is risk mitigatin and planning. This invlves identifying a respnse strategy fr each significant risk. There are fur basic types f respnse: 1. Avid. Change the prject s that the risk des nt ccur. Fr example, a tllway might nt be built beynd the limits f current develpment. 2. Transfer. Cntract with an entity better able t mitigate and manage the risk. A tllway may be sld t a cncessinaire r t a tlling agency with multiple radways. Bear in mind that risk transference usually invlves a cst. 3. Mitigate. Develp a strategy t decrease the likelihd r impact f a risk. This culd include an incremental prject develpment strategy that adds lanes (and prject csts) as demand warrants rather than building the entire prject at nce. Impacts f lwer revenue culd be ffset by higher debt service cverage ratis. 16