Cost-benefit analysis and multi-criteria analysis: Competing or complementary approaches? Dr George Argyrous School of Social Sciences and International Studies, UNSW g.argyrous@unsw.edu.au
Cost-benefit analysis and multi-criteria analysis Both are frameworks for assessing options facing decision-makers Both try to construct a common metric for comparing options Both sensitive to assumptions; but different assumptions! Are they aids to decision-making or a way of decision-making? Evidence for policy and decision-making 2
Cost-benefit analysis (CBA) A conceptual framework for the evaluation of projects in the government sector which tries to consider all gains and losses from the project Takes a long and wide view Includes all relevant future dates Include effects on all relevant parties Usually expresses costs and benefits in the common metric of today s money Evidence for policy and decision-making 3
Cost-benefit guidelines UK Department of the Treasury, Appraisal and Evaluation in Central Government (The Green Book), London:2002 <http://www.hmtreasury.gov.uk/data_greenbook_index.htm> NZ Treasury guidelines <www.treasury.govt.nz/publications/guidance/planning/costbenefitanalysis> Australian Government, Office of Best Practice Regulation, <http://www.finance.gov.au/obpr/cost-benefit-analysis.html> See especially Handbook of Cost-Benefit Analysis, and Best Practice Regulation Handbook National Competition Council <www.ncc.gov.au> and Office of Regulation Review <www.orr.gov.au> for their guidance on application of CBA to legislation and regulation Evidence for policy and decision-making 4
Cost-benefit guidelines New South Wales Government Treasury, Policy Paper TPP07-05: NSW Government Guidelines for Economic Appraisal <www.treasury.nsw.gov.au/ data/assets/pdf_file/0016/7414/tpp07-5.pdf> New South Wales Government, Department of Premier and Cabinet Better Regulation Office, 2008, Guide to Better Regulation, <www.betterregulation.nsw.gov.au> Queensland Government, Department of Infrastructure and Planning, Cost Benefit Analysis, <www.dip.qld.gov.au/resources/guideline/project-assurance-framework/pafcost-benefit-analysis.pdf> Government of Western Australia, Department of Treasury and Finance, 2005, Project Evaluation Guidelines, <www.dtf.wa.gov.au/cms/uploadedfiles/project_evaluation_guidelines_2002.pdf> Evidence for policy and decision-making 5
The basic steps of cost-benefit analysis 1. Determine if CBA worth doing 2. Identify objectives and policy alternatives 3. Determine who has standing 4. Identify the costs and benefits of each alternative 5. Sort into measurable and non-measurable costs and benefits 6. Measure costs and benefits that can be measured in money terms 7. Conduct sensitivity analysis 8. Compare costs-benefits across alternatives 9. Adjust for non-measurable costs and benefits(?) 10.Make a decision Evidence for policy and decision-making 6
Advantages of cost-benefit analysis Forces disciplined consideration of choices, including status quo option Recognizes that each choice has a cost (however unpleasant that admission might be) Makes explicit hidden costs and benefits Objective in the sense that it follows an established and open methodology Forces more detailed consideration of what we mean by the adjectives placed in front of the word value (e.g. heritage, historical, cultural, etc.) Overcomes program optimism Evidence for policy and decision-making 7
Disadvantages of cost-benefit analysis Ignorant of the political context of decision-making (or is this a benefit?) Individualistic in its underlying social vision Anthropocentric in its underlying social vision Valuation techniques are imperfect and loaded with assumptions Conclusions often highly sensitive to specific assumptions, such as discount rate and risk/uncertainty Valuation of life is difficult or impossible Difficult to balance non-quantifiable costs/benefits against quantifiable ones within CBA s own terms Limited notion of equity Evidence for policy and decision-making 8
Which discount rate? Cwth NZ Qld WA NSW UK Use market interest rates: opportunity cost of capital. Suggest 7 per cent (real) 10% real discount rate whenever there is no other agreed sector discount rate (p. 27) To be agreed between agency and Queensland Treasury. Suggested reference points: the interest rate for government borrowings for a term relevant to the expected duration of the project the long-term average real economic growth rate, with an additional allowance for major risks and time preference for current consumption. the rate of return on debt and equity for comparable private sector projects No prescribed discount rate. In general, the discount rate should be based on: cost of funds - agencies should apply the Western Australian Treasury Corporation (WATC) longterm borrowing rate as the risk-free rate; adding to this risk-free rate a premium to allow for risk, based on industry norms; and where appropriate, add a further premium to reflect scarcity of capital funds and the availability of other capital investment projects that demonstrate higher internal rates of return 7% real 3.5% real Evidence for policy and decision-making 9
Costs and benefits that can t be monetised quality of life, equity, achieving community norms,ensuring public health and safety, reducing crime and protecting human rights NSW GTBR history, heritage, Indigenous issues, the arts and culture quality of life (e.g. access to recreational facilities; beautification of surroundings) health welfare ecological sustainability (over the time period being assessed) public safety (e.g. road safety, workplace safety) law and order (e.g. crime rates, recidivism) employment (e.g. morale, business confidence) education (e.g. literacy) (Qld) Evidence for policy and decision-making
Handling qualitative costs and benefits Exclude from strict CBA, and Include in final decision-making through other analyses: Specify qualitative pros and cons (Cwth, NZ). If they [intangible costs and benefits] are significant they should be explicitly highlighted and explained in the analysis so that decision-makers are aware of the value judgments they are making in pursuing a particular option. This explanation can be quantitative, qualitative, descriptive, or a combination of these (NZ, p. 38) Social Impact Analysis (WA, Qld), or Multi-criteria Analysis (NSW GTBR, Qld, NZ, UK): it may be better to rank options by their ability to deliver specific outcomes and assess them against the performance criteria for promotional success rather than economic impact (Qld, p. 41) Evidence for policy and decision-making
The role of multi-criteria analysis (MCA) MCA provides a framework to enable decision-makers to overcome difficulties in handling large amounts of complex information in a consistent way MCA establishes preferences between options by reference to an explicit set of objectives that the decision-making body has identified, and for which it has established measurable criteria to assess the extent to which the objectives have been achieved Evidence for policy and decision-making 12
Basic MCA: An overview There are different types of MCA, but they all: evaluate alternatives; by reference to an explicit set of objectives for performance; with alternatives scored as to how well they meet each objective; and scores aggregated to indicate overall performance of options Increasingly used to judge tender bids Evidence for policy and decision-making 13
Steps in MCA MCA follows these broad steps (DTLR, 2000: 25): 1. Establish the decision context 2. Identify the value/performance criteria 3. Describe/rate the performance of each option against the criteria 4. Assign weights across criteria 5. Combine the scores and weights 6. Examine the results and review 7. Conduct sensitivity analysis Evidence for policy and decision-making 14
Performance matrix A performance matrix organizes the inputs into the decision-making process Options should not be excessive Performance criteria should not be dependent on each other Note that there may be legislated criteria for decision-making e.g. equity and diversity. Options Performance criteria criterion 1 criterion 2 criterion 3 criterion n Option 1 score score score score Option 2 score score score score Option 3 score score score score Option m score score score score Evidence for policy and decision-making 15
Who decides the ratings? MCA very flexible with respect to who gets a say in either the criteria or rating the options. Some ways this flexibility has been applied: Democratic decision-making - all members of the decision-making body, or each organizational branch/unit, independently allowed to rate options Panel of experts asked to make judgments; can use different panel to judge different criteria Consensus model - decision-making body thrash it out Stakeholder inclusion Different groups can rate options on different criteria Evidence for policy and decision-making 16
Outcome of MCA Can be used to: Identify a single, most-preferred option Rank options Short-list a limited number of options for subsequent detailed appraisal through other frameworks such as CBA Distinguish acceptable from unacceptable options Combine different options based on relative strengths (e.g. tenderers that maximize performance across criteria) Evidence for policy and decision-making 17
Strengths and weaknesses of MCA Strengths Open and explicit Can incorporate diverse range of information Can be subcontracted Can be communicated Provides an audit trail, especially in situations where decision-making is required to follow rules and to be justified in explicit terms Flexible along many dimensions: choice of options, criteria, weighting, who is involved in weighting and scoring Weaknesses Weighting is hard to derive Private interest dressed as public virtue? Less inclusive Lacks methodological rigor Evidence for policy and decision-making 18
MCA and CBA: Complements or alternatives? MCA can be seen as subsuming a CBA; CBA is one criterion for assessing options MCA can define a narrow set of options then subjected to CBA MCA can sit alongside CBA: one handles measurable costs and benefits, the other handles qualitative costs and benefits; approach of many government guidelines MCA inferior to the objective CBA approach P. Abelson, 2010, Notes on Cost-Benefit Analysis, Applied Economics, Sydney L. Dobes and J. Bennett, 2009, Multi-Criteria Analysis: Good enough for government work? Agenda, vol. 16, no. 3, pp. 7-29. Evidence for policy and decision-making