Cost-Benefit Analysis for Juvenile Justice Programs

Program Evaluation #4 Cost-Benefit Analysis for Juvenile Justice Programs Briefing Series Juvenile Justice Evaluation Center Justice Research and Statistics Association Office of Juvenile Justice and Delinquency Prevention 1

Cost-Benefit Analysis for Juvenile Justice Programs The JJEC, which is supported by the Office of Juvenile Justice and Delinquency Prevention, provides evaluation information, training, and technical assistance to enhance juvenile justice evaluation in the states. For more information about the JJEC project, visit our Web site at http://www.jrsa.org/jjec or e-mail us at jjec@jrsa.org. This is one of a series of briefings prepared by the Justice Research and Statistics Association under the Juvenile Justice Evaluation Center (JJEC) project. The purpose of this briefing series is to provide juvenile justice program managers with information that will help them evaluate their programs. Each briefing addresses a topic that is of particular interest to juvenile justice program managers who are trying to determine the effectiveness of the programs they operate. Juvenile Justice Evaluation Center Justice Research and Statistics Association 777 North Capitol Street, N.E. Suite 801 Washington, D.C. 20002 (202) 842-9330 http://www.jrsa.org/jjec 2

Acknowledgments Steve Aos, Associate Director, Washington State Institute for Public Policy, prepared this briefing. Francesca Garcia Lanier, a Project Manager at the Justice Research and Statistics Association (JRSA), and Stan Orchowsky, JRSA's Research Director, provided input and editing. Nancy Michel, JRSA's Director of Publications, assisted with copy editing. We would also like to thank Eric Peterson, the project's Grant Manager at the Office of Juvenile Justice and Delinquency Prevention, who has provided valuable support throughout this project. May 2002 This project was supported by Grant No. 98-RN-FX-0112 awarded by the Office of Juvenile Justice and Delinquency Prevention, Office of Justice Programs, U.S. Department of Justice. Points of view in this document are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice. 3

Table of Contents Introduction... 5 Briefing Scope... 6 Distinguishing Cost-Benefit Analysis From Program Evaluation... 7 Five Elements of a Cost-Benefit Analysis of Juvenile Justice Programs... 8 Step 1: Identify and Quantify the Benefits in Dollar Terms... 9 Step 2: Subtract the Program Costs... 11 Step 3: Calculate the Economic Bottom Line from Steps 1 & 2... 12 Step 4: Compare the Estimated Bottom Line to the Returns Available From Other Options... 13 Step 5: Test the Riskiness of the Conclusions... 13 Conclusion: Cost Benefit Analysis Can Produce Valuable Information... 14 Putting Cost-Benefit Analysis to Work in the Real World: Washington State... 14 Resources... 16 4

Introduction People make decisions every day. They choose what type of automobile to buy, they select a mutual fund, they support one juvenile justice program while passing over another. Regardless of whether a decision is personal or professional, people are constantly deciding to do, or not do, something. Economists have spent a great deal of time thinking about how people choose. In doing so, they have developed a set of ideas and analytical tools that help people sift through information and make better decisions. This process is known as cost-benefit analysis. While the label may appear intimidating, cost-benefit analysis is, at its core, quite simple: Cost-benefit analysis is a formal way of adding up the advantages and disadvantages of doing one thing as opposed to doing something else. Most of the time, of course, people sum up the pluses and minuses of both mundane and major choices without performing a rigorous analysis often a simple back of the envelope calculation is all that is necessary or possible. Cost-benefit analysis tries to standardize informal calculations and express the results in dollars-and-cents terms. So, while formal cost-benefit analysis may at times appear complicated, all that is really being done is quantitatively weighing the advantages and disadvantages of one course of action over another. 5

Briefing Scope This briefing focuses on how formal cost-benefit analysis can help people make informed decisions about certain aspects of the criminal justice system. Our particular focus is the juvenile justice system, but the techniques described here are applicable to the full range of criminal justice resource decisions. The emphasis is on decisions that affect how money usually taxpayer-supplied money is spent in the criminal justice system. Many other types of decisions, of course, are made in the criminal justice system, but formal cost-benefit analysis applies most readily to decisions that must be made about how dollars get spent. Cost-benefit information can assist decisionmakers in more efficiently allocating scarce public resources among competing demands. For example, should a state government spend more money on a juvenile boot camp or on a therapeutic program like Multi-Systemic Therapy? Costbenefit analysis can help us synthesize available research evidence to inform practical decisions such as this. First, let s consider some of the general elements of a cost-benefit analysis of juvenile justice options. Once we have a grasp of the principles involved in this type of analysis, we will then look at the real-world example of Washington State, which has been using cost-benefit techniques to effectively manage its public resource. At the end of the briefing, we have included a resource section that includes references to cost-benefit work done in Washington State. 6

Distinguishing Cost-Benefit Analysis from Program Evaluation To understand what cost-benefit analysis is, we must first understand what it is not. Since we will discuss how cost-benefit analysis can be used to estimate the economic bottom line of different juvenile justice programs, it might be tempting to think of it as a synonym for program evaluation. That would be a mistake. By itself, cost-benefit analysis does not answer the fundamentally important question asked by a program evaluation: Does a program work? A program evaluation usually seeks to determine the degree to which a program achieves a particular goal or outcome, such as reducing the recidivism rate of program participants. For example, the question might be: Does a drug court lower recidivism rates? Cost-benefit analysis does not contribute anything to answering this basic program evaluation question. Rather, cost-benefit analysis starts with the results of a program evaluation in the previously mentioned example, the measured difference in recidivism rates for the drug court and comparison groups. A cost-benefit analysis then quantifies the dollar value of the benefits of reduced crime, subtracts the program costs of the drug court, and arrives at a bottom line economic estimate for the program. In effect, a costbenefit analysis can be thought of as an economic add-on to the more basic question posed by a program evaluation. In sum, a good program evaluation should be able to answer the fundamental question: Does a program work? A good cost-benefit analysis, on the other hand, should be able to take the answer to this question one step further: Given what was found in the program evaluation, does the dollar value of a program s demonstrated level of success exceed the cost of the program? Additionally, how does one program s economic bottom line compare to other similarly analyzed options facing the decisionmaker? 7

Five Elements of a Cost-Benefit Analysis of Juvenile Justice Programs In theory, a cost-benefit analysis of a criminal justice program should involve five straightforward steps: 1) add up the monetary benefits; 2) subtract the costs; 3) see if the resulting bottom line, expressed in dollar terms, is positive or negative; 4) compare the estimated bottom line to the returns available from other options; 5) test the riskiness of the conclusions. These steps may sound simple in theory, but in practice, of course, complications arise. Still, a good cost-benefit analysis will address all five steps. 8

Step 1 identify benefits 1 subtract costs 2 calculate bottom line 3 compare options 4 test riskiness 5 Identify and Quantify the Benefits in Dollar Terms The most difficult part of most cost-benefit analyses of justice programs is the first of the five steps: quantifying the benefits in dollar terms. For example, suppose a rigorous outcome evaluation of a counseling program for juvenile offenders indicates that the program lowers criminal recidivism from 1.7 average subsequent offenses for juveniles who do not participate in the program to 1.2 offenses for program participants. Costbenefit analysis involves estimating the dollar value of this measured reduction in recidivism. In our example, the question is: What is the value, in dollar terms, of reducing crime by an average of.5 offenses per program participant? We discuss below several issues that must be considered in order to answer this question. Benefits for Whom? One question that must be considered early in the analysis is to whom do the benefits of a reduction in recidivism accrue? Many cost-benefit analyses examine this from two perspectives: the taxpayer and the crime victim. If a program can reduce the future rate of offending, then benefits will flow to taxpayers who do not have to pay for the criminal justice system that would have processed those offenses. Benefits will also accrue to people and communities who would otherwise have been the victims of those crimes, and these benefits need to be estimated as well. Some analysts may also try to estimate the benefits of reduced recidivism from a third perspective: the program participants themselves. This perspective involves measuring the life-course gains (for example, improved schooling outcomes and employment opportunities) that offenders receive when they offend less. This third perspective has rarely been attempted in applied cost-benefit analyses. What is the Dollar Value of Reduced Crime? Once the question of perspective is decided, then the cost-benefit analysis must develop statistical procedures to quantify in dollars-and-cents terms the value of the reduced recidivism. Clearly taxpayers and crime victims benefit each time crime is reduced, but by how much? In our previous drug court example, how many dollars is it worth to avoid an average of.5 crimes per drug court participant? In this briefing, we cannot go into the sometimes elaborate procedures that have been developed by analysts to estimate these values of reduced crime, but methods are available to approximate the gains that taxpayers and crime victims receive each time a unit of crime is reduced. 9

What About the Non Crime-Related Benefits? Sometimes program evaluations measure changes in outcomes such as substance abuse, employment, education, or pregnancy. A comprehensive cost-benefit analysis will attempt to quantify the value, expressed in dollar terms, for changes in these outcomes, in addition to the more often studied crime-related outcome. To date, there have been few attempts to quantify the dollar value of the non-crime benefits of intervention programs. What Do We Know About the Long Run? Program evaluations typically have relatively short follow-up periods often just 6 months to a year over which effects are evaluated. A good cost-benefit analysis, however, seeks to determine the long-run costs and benefits of different program and policy alternatives. This longrun view is particularly important when studying the costs and benefits of criminal justice options. Why? For many programs that attempt to reduce crime, the costs that can be avoided are tied to resources that have very long economic lives, such as prisons, jails, and detention facilities. In order to have an economic analysis that offers apples-to-apples comparisons, it is imperative to estimate long-run benefits and costs of different ways resources can be spent. Fortunately, there is often additional information that can be used to provide reasonable estimates of the long-run costs and benefits of many criminal justice programs. Long-term recidivism studies, for example, produce valuable information about the quantity, type, and timing of future criminality of offenders to which alternative programs might be applied. This information, in conjunction with the results of a short-run program evaluation, can be used to improve estimates of the long-run effects that a program can be expected to have on future criminality. What Are the Marginal Costs? Economists stress the importance of using marginal costs when studying the economics of decisions. Marginal costs describe how the total cost of an operation changes as the unit of activity changes by a small amount. Marginal costs are different from average, or accounting, costs. Average costs are derived by dividing total costs by total workload in a given period of time. Some of those costs, however, are fixed and do not change when workload changes. Marginal costs reflect only those costs that go up or down as workload changes. For example, if the average daily population of a state juvenile justice institution is reduced because a community-based intervention program reduces crime, how then should the taxpayer value (i.e., the benefit) of the reduced average daily population be calculated? To perform this calculation, it is important to recognize that some of the costs of the state institution will not change when the average daily population goes down; for example, the salary of the director will not be eliminated. Other budgeted taxpayer costs that are tied to average daily population levels, however, will drop. An average cost calculation would include the director s salary, while a marginal cost analysis would not include the director s salary. When establishing the values to use in a cost-benefit analysis, it is important to estimate, whenever possible, marginal costs rather than average costs. 10

Step 2 Subtract the Program Costs identify benefits 1 subtract costs 2 calculate bottom line 3 compare options 4 test riskiness 5 After the considerable difficulties in Step 1 of the cost-benefit process, the tasks facing the analyst in Step 2 are much easier. Here the job is to estimate what the programs themselves cost to run. While this step is easier, however, it is not trivial. There are two broad types of costs associated with programs: capital costs and operating costs. A full accounting of the costs associated with a program will include both. The main theoretical consideration is the focus on marginal rather than average costs, reflecting the same logic as that guiding the dollar value estimates for the benefit side of costbenefit analysis. The program cost analysis usually involves reviewing accounting data on program operation, paying careful attention to exclude fixed costs that would be incurred regardless of whether the program operates. 11

Step 3 identify benefits 1 subtract costs 2 calculate bottom line 3 compare options 4 test riskiness 5 Calculate the Economic Bottom Line from Steps 1 and 2 The products of the first two steps of a cost-benefit analysis can then be combined to produce standard economic statistics that summarize the relative costs and benefits of programs. A simple solution is just to subtract the costs (from Step 2) from the benefits (from Step 1). Unfortunately this simple calculation may yield incorrect results unless another cost-benefit concept is introduced. Timing is also important in cost-benefit analysis. The benefits and costs estimated in Steps 1 and 2 often occur unevenly over many years. For example, the costs of a successful treatment program for juvenile offenders may be incurred in the first year, while the benefits of the program (reduced crime) arrive over the course of many subsequent years. Cost-benefit analysis takes this uneven timing of costs and benefits into account by a seemingly arcane process known as discounting. This technique reduces the face value of future benefits to reflect the idea that time is valuable to people. All else being equal, a person would rather have one dollar today than the same dollar sometime in the future. That is, a dollar today opens up possibilities that the same dollar in the future does not. In a formal cost-benefit analysis, the estimated benefits and costs are arranged over time (usually a number of years) and then those two annual flows of dollars are discounted to present value. To carry out the calculations, the analyst must choose a discount rate to apply to all options considered. The result of these calculations is the bottom-line economic estimate that one looks for in a cost-benefit analysis. The estimated annual flows of costs and benefits are usually summarized as net present values, benefit/ cost ratios, or internal rates of return. It is important to note that each of these summary measures is derived from the same set of annual cash flows estimated in Steps 1 and 2 of the cost-benefit process. 12

Steps 4&5 identify benefits 1 subtract costs 2 calculate bottom line 3 compare options 4 test riskiness 5 Compare the Estimated Bottom Line to the Returns Available From Other Options To be most useful to decisionmakers, a cost-benefit analysis must estimate not only the net present value of one action, but also similarly calculated economic measures of a range of alternatives. The cost-benefit analysis should compare one program to another, rather than solely focusing on the absolute value of a particular benefit-tocost ratio. Moreover, as is apparent from our discussion of Steps 1 and 2, a number of assumptions and estimates go into a cost-benefit analysis of criminal justice options. It is therefore critical that the cost-benefit ratios be calculated in the same way for all programs. Once this is assured, users can look at the relative ranking of one option versus another. Test the Riskiness of the Conclusions The preceding steps describe a cost-benefit analysis that produces one answer per program. That is, based on all of the input, a single costbenefit summary statistic can be produced for a program. As we have seen, however, many of the inputs to a cost-benefit analysis are uncertain to some degree. Because of this, it is important to test how sensitive the bottom-line conclusion is to changes in key input assumptions. There are simple and elaborate procedures that provide for testing assumptions and these should be included in a good cost-benefit approach to decisionmaking. 13

Conclusion: Cost-Benefit Analysis Can Produce Valuable Information People make decisions about how resources should be used in the juvenile justice system every time budgets are adopted: Which programs should be funded, which should be expanded, which ought to be eliminated? The cost-benefit analysis outlined in this briefing provides a framework to synthesize program evaluation information and arrive at economic bottom-line measures for some of these choices. As with any investment decision that must be made, this type of economic information can be part of the calculus used by decisionmakers to choose courses of action that have the best chance of producing favorable returns on taxpayer dollars. Putting Cost-Benefit Analysis to Work in the Real World: Washington State Now that we have examined how cost-benefit analysis works and seen how criminal justice programs can be evaluated in terms of whether they are giving an adequate return on the investment made in them, let us turn to a real-world example that illustrates some of the benefits of this type of analysis. In 1997, the State of Washington adopted a strategy to implement cost-beneficial programs for juvenile offenders. Legislation was passed that allowed the state s locally run juvenile courts to propose programs that had a research-proven ability to reduce juvenile offending. To help in selecting these programs, the Washington State Institute for Public Policy conducted a comprehensive cost-benefit analysis of programs that try to reduce crime. For a wide range of approaches from prevention programs designed for young children to correctional interventions for juvenile and adult offenders the Institute systematically analyzed program evaluations produced mostly in North America over the past 25 years. The Institute organized individual evaluations into policyrelevant topics, such as early childhood education programs, drug courts, cognitive-behavioral programs for juvenile sex offenders, and so on. Some program groupings were quite general and some were for very specific programs, such as those identified as part of the Blueprints project of the University of Colorado s Center for the Study and Prevention of Violence. 14

The Institute then developed and used its cost-benefit model to provide an independent determination of whether a program s benefits as measured by the value to taxpayers and crime victims from a program s expected effect on crime were likely to outweigh its costs. This procedure allowed a direct apples-to-apples comparison of the economics of different types of programs designed for widely varying age groups. An example of the Institute s cost-benefit findings is shown in Table 1 (see link to Table 1 at: www.jrsa.org/jjec/about/jjec-briefings.html). The Institute concluded from this analysis that sufficient research on what works had developed over the last two decades to allow decisionmakers in Washington to use this information to improve public resource allocation. That is, the cost-benefit estimates developed from the evaluation findings could be used to assist decisionmakers in directing scarce public resources toward economically successful programs and away from unsuccessful programs, thereby producing net overall gains to taxpayers, even in the absence of new funding sources. The resulting proposal to the legislature was to implement four specific approaches in the juvenile courts: Multi-Systemic Therapy, Functional Family Therapy, Aggression Replacement Training, and Interagency Coordination. The legislature then funded the programs in subsequent legislative sessions. Formal training in the specific programs for court staff and private counselors began and the first juvenile offenders entered the programs in early 1999. During fiscal year 2001, about 2,200 juvenile offenders entered the programs. The Institute, as required by the original legislation, is conducting a formal outcome evaluation and cost-benefit analysis of the programs to see if they actually work to produce favorable returns to taxpayers in Washington. This real world example indicates how cost-benefit analysis can be used in the same two ways any investment analysis can be employed. First, cost-benefit analysis can be used to help select reasonable courses of action based on a rational consideration of research-based evidence. Second, after selecting a course of action, a well-done program evaluation coupled with a cost-benefit analysis can be used to help evaluate the economic consequences of the program choices actually made. In sum, cost-benefit analysis can be used to help direct state resources toward economically successful juvenile justice programs. 15

Resources The publications and Web sites discussed below provide additional information on the concepts and procedures discussed in this briefing. A comprehensive discussion of the five steps that comprise a formal cost-benefit analysis, as well as a comprehensive set of estimates of taxpayer-financed criminal justice costs, can be found in The Comparative Costs and Benefits of Programs to Reduce Crime, Version 4.0 (S. Aos, P. Phipps, R. Barnoski and R.Lieb). Washington State Institute for Public Policy, May 2001 (available online at: http://www.wa.gov/wsipp/crime/pdf/costbenefit.pdf). A standard source for victimization costs of crime is Victim Costs and Consequences: A New Look (T.R. Miller, M.A. Cohen and B. Wiersema). National Institute of Justice, Washington, DC, 1996. A practical source of information on issues surrounding discount rates for public projects, as well as reasonable current estimates, can be found on the Office of Management and Budget s Web site at: http://www.whitehouse.gov/omb/circulars/a094/a094.html#ap-c. Standard texts that discuss methods for summarizing costs and benefits, such as net present values and benefit/cost ratios include: Cost-Benefit Analysis: Concepts and Practice (A.E. Boardman, D.H. Greenberg, A.R. Vining and D.L Weimer). Prentice Hall, 2001. Applied Cost-Benefit Analysis (R.J. Brent). Edward Elgar, 1996. A description of the programs being implemented in the Washington juvenile courts can be found on the Washington State Institute for Public Policy s Web site: http://www.wa.gov/wsipp/crime/cjaa/home.html 16

Program Evaluation Briefing Series #1 Juvenile Justice Program Evaluation: An Overview #2 Hiring and Working With an Evaluator #3 Strategies for Evaluating Small Juvenile Justice Programs #4 Cost-Benefit Analysis for Juvenile Justice Programs Juvenile Justice Evaluation Center Justice Research and Statistics Association 777 North Capitol Street, N.E. Suite 801 Washington, D.C. 20002 (202) 842-9330 www.jrsa.org/jjec 17