Applying Social Return on Investment to Educational and Youth Programming

Transcription

1 1101 Connecticut Ave. NW, Suite 300, Washington, DC P F Applying Social Return on Investment to Educational and Youth Programming Redacted for University of Salford In this report, Hanover Research addresses the relatively new concept of Social Return on Investment (SROI) in the context of educational and youth programming. The report provides an overview of the SROI methodology and how it allows organizations to quantify their impact in order to illustrate the value of their programming to stakeholders. The report provides quantitative and qualitative metrics educational and youth oriented organizations can utilize in the outcomes portion of their SROI process as well as relevant case studies. MARKET EVALUATION SURVEYING DATA ANALYSIS BENCHMARKING INNOVATIVE PRACTICES LITERATURE REVIEW

2 Introduction Social Return on Investment (SROI) is a relatively new concept which allows noncommercial organizations to quantify the impact of their social entrepreneurial activities and programs. The Cabinet Office of the UK government, which is involved in social enterprises and has published substantive work on the subject, states that: SROI is a framework for understanding, measuring and managing the outcomes of an organisation s activities. SROI can encompass all types of outcomes social, economic and environmental but it is based on involving stakeholders in determining which outcomes are relevant. 1 SROI is distinct from other types of social accounting because it places a monetary value on outcomes, allowing these outcomes to be compared to investments. Ultimately, organizations are able to define a ratio of total benefits (social value) to total investments. However, SROI should also be a story of change with both qualitative and quantitative analysis. The Cabinet Office explains the distinction between two types of SROI: Evaluative SROIs, which are conducted retrospectively and based on outcomes that have already taken place, and forecast SROIs, which predict how much social value will be created if activities meet their intended outcomes. 2 This report will primarily focus on the structure and benchmarks of evaluative SROIs. It will view SROI within the context of what it can achieve for an organization: Allows organizations to explore the value of their services and programs Enables organizations to demonstrate commitment to robust impact measurement and a commercial approach to project evaluation Enables organizations to communicate to potential and current funders and investors the value of a service or program Creates a planning and evaluation tool as organizations move forward Forms a model which can be modified and applied to future programs and services Develops an ongoing relationship with stakeholders based on value and cooperative work to achieve objectives 3 The SROI methodology is now being widely applied to non-profit organizations and other efforts whose impact cannot be measured by revenue and profit margins. This report will focus on evaluative SROIs for educational and youth oriented programs. In order to address XYZ s objective to measure the impact of its entrepreneurial 1 Social Return on Investment an introduction. Cabinet Office, Office of the Third Sector, United Kingdom. p Ibid. 3 How SROI Has Helped Me. The SROI Network. 2

3 remediation, intervention and prevention services, this report will be divided into three major sections: Section 1: SROI Methodology Overview o As a foundation to the report, this section provides an overview of the SROI methodology including its principles and processes. The discussion also addresses two organization s SROI approaches and incorporates a fictional case study to illustrate the application of the SROI methodology. Section 2: SROI Metrics for Youth and Educational Programs o This section constitutes the primary material in the report. It will address the outcomes aspect of the SROI methodology and what metrics are applicable to youth and educational programs. Case study material is included. Section 3: Conclusion and Recommendations o This section addresses the potential for a framework of metrics for XYZ to utilize for its remediation, intervention, and prevention programs. The report will conclude that there is significant potential for XYZ to implement a successful SROI process in order to quantify the impact of its social entrepreneurial programs, though it will require significant commitment to follow through the entire process, choosing and applying appropriate metrics. 3

4 SROI Methodology Overview Completing the entire SROI methodology is a committed process for any organization. It requires significant investment in understanding the impact an organization s specific programs are having on all of its stakeholders both in the short and long term. The SROI methodology puts financial value on the important impacts [of a program] identified by stakeholders that do not have market values. 4 Though the process should ultimately result in a ratio of social value to investment inputs, SROI is touted as a framework to structure thinking that should allow an organization to advantageously quantify its impact. This section of the report will be divided into two discussions: the principles and processes of SROI evaluation. Principles of SROI According to the SROI Network, an international network of organizations utilizing the SROI methodology, SROI is based on seven principles. These principles include the objectives driving SROI and how it should be used: Involve stakeholders: Understand the way in which the organization creates change through a dialogue with stakeholders. Understand what changes: Acknowledge and articulate all the values, objectives and stakeholders of the organization before agreeing which aspects of the organization are to be included in the scope; and determine what must be included in the account in order that stakeholders can make reasonable decisions. Value the things that matter: Use financial proxies for indicators in order to include the values of those excluded from markets in the same terms used in markets. Only include what is material: Articulate clearly how activities create change and evaluate this through the evidence gathered. Do not over-claim: Make comparisons of performance and impact using appropriate benchmarks, targets, and external standards. Be transparent: Demonstrate the basis on which the findings may be considered accurate and honest and show that they will be reported and discussed with stakeholders. Verify the result: Ensure appropriate independent verification of the account. 5 These principles highlight the core terminology of the SROI methodology including: stakeholders, scope, financial proxies, social value, benchmarks, and transparency. The SROI methodology is a comprehensive process which holds the value created for stakeholders as its core objective. The metrics, proxies, and indicators used to pursue this objective are incorporated in a process based on clarity. The latter part of this report will focus primarily on the fourth and fifth principles listed above by 4 Welcome. The SROI Network. 5 Ibid. 4

5 addressing the articulation of an organization s value through the use of metrics known alternatively as indicators, external standards, and benchmarks. The SROI Process The second part of this section of the report addresses the actual process of applying SROI to an organization s programming. Sources differ somewhat on the exact steps that should be included in this process. This report utilizes the steps laid out in the SROI Guide 2009 published by the Cabinet Office in the United Kingdom. The six steps move through the process from establishing the scope to identifying and valuing outcomes to calculating and reporting the SROI. SROI users often utilize impact maps, which allow for visualization of the process. Images of these maps are included throughout this section. In addition, the discussion is integrated with a case study of Wheels-to-Meals a fictional organization that transports elderly people to an organized lunch club. For a comprehensive guide to the SROI process, Hanover Research recommends that XYZ utilize the SROI Guide 2009 directly. At the end of this section, there is also a discussion of the SROI approaches of the Roberts Enterprise Development Fund and the New Economics Foundation, which differ in some aspects from the SROI Guide Establishing scope and identifying key stakeholders. It is important to have clear boundaries about what your SROI analysis will cover, who will be involved in the process and how. Often service users, funders and other agencies working with the client group are included in an SROI. 6 This step of the process can be divided into two parts that occur before the actual analysis. First, to establish scope an organization must recognize the limitations of its SROI process in quantifying impact. This means addressing concepts such as purpose (e.g. funding or public relations efforts), audience, background (objectives of the organization), resources, personnel who will work on SROI, range of activities, period of time and how often the SROI will be completed. 7 Stakeholders are people or organizations that experience change, whether positive or negative, as a result of the activity being analysed. 8 Stakeholders might include employees, students, parents, organizations involved, donors, local employers, local law enforcement, local employers, health care organizations, and taxpayers. Stakeholders can also be divided into subgroups such as students requiring different types of intervention. Figure 1.1 illustrates how to organize the stakeholder analysis. 6 Social Return on Investment an Introduction. Op. cit. p A Guide to Social Return on Investment. Cabinet Office, Office of the Third Sector p Ibid, p

6 The SROI process benefits greatly from involving stakeholders in the process by asking directly about how an organization s programming impacts them. Figure 1.1: Organizing the Stakeholder Analysis Source: SROI Guide In this first step, Wheels-on-Meals decided that the goal of their SROI would be to contribute to the upcoming review with the local governing authority and would cover the organization s activities over the period of a year. Wheels-on-Meals chose to pursue a forecast SROI analysis to be completed by internal staff rather than consultants. Stakeholders would include volunteers, the local governing authority, and the service users (the elderly participants) Mapping outcomes. Through engaging with your stakeholders you will develop an impact map (also called a theory of change or logic model) which shows the relationship between inputs, outputs and outcomes. 11 Impact maps can be used to visualize the process of identifying and valuing inputs. Monetary inputs might include grants or budget allotments while non-monetary inputs might include volunteer time. These inputs result in certain outputs. For example, a $50,000 grant plus donations (input) might result in an organization being able to provide a year of hot meals to the homeless (output). It is also necessary to value non-monetary inputs such as valuing volunteer time based on hourly minimum wage (or average wage for that type of work) or putting a monetary value on goods and services donated in kind. If an organization has surplus available from all the inputs (e.g. an organization does not use all its donations over the time period) then that surplus should be accounted for either by the additional social value which might have been generated if the surplus had been used or by reducing the value of the input by the amount of the surplus. 12 The impact map for the SROI process so far is shown in Figure A Guide to Social Return on Investment. Op. cit., p Social Return on Investment an Introduction. Op. cit., p Ibid. 12 A Guide to Social Return on Investment. Op. cit., p

7 Figure 1.2: Impact Map for Steps 1 and 2 Source: SROI Guide 2009 The Impact Map 13 Outputs are then translated into outcomes, which are the objectives of the organization the social value impacts it aims to achieve. It is important to note at this point the difference between outputs and outcomes (outcomes will be addressed in the next step). For example, an organization s activity might be employment mentoring for students. Its output would be 50 students receiving employment mentoring and the outcome would be increased employability. The SROI Guide emphasizes that the outcomes aspect of the SROI process cannot be fully completed without talking to stakeholders about how an organization s activities have changed them. 14 The movement from inputs to outcomes is called a chain of events, theory of change, or impact value chain. The completion of the first two steps of the SROI process by Wheels-to-Meals is shown in Figure 1.3 including the inputs, outputs, and outcomes for its stakeholders. The monetary inputs were calculated for the local governing authority and the value of the volunteers time (based on appropriate wages). Wheels-on-Meals calculated a total input value of 42,375. The impact map illustrates the numerous positive outcomes for the elderly participants in the program, including benefits from the mild exercise, information sessions with doctors, socializing, and healthy meals. Volunteers also benefited from being healthier. One negative outcome Wheels-to-Meals found 13 SROI Guide 2009 The Impact Map. Cabinet Office, Office of the Third Sector A Guide to Social Return on Investment. Op. cit., p

8 when reaching out to its stakeholders during the SROI analysis was that neighbors of the elderly participants in the program had previously shopped for and interacted with the participants. As a result of the program, neighbors were less involved in the elderly participants lives, which many of the participants noted as being a detracting factor. 15 Figure 1.3 illustrates the crucial initial steps of the SROI process the evolution from determining the stakeholders to determining the relevant outcomes. Figure 1.3: Stakeholders to Outcomes for Meals-to-Wheels Source: SROI Guide A Guide to Social Return on Investment. Op. cit., p Ibid, p

9 3. Evidencing outcomes and giving them a value. This stage involves finding data to show whether outcomes have happened and then giving them a monetary value. 17 The most significant step in the SROI process is determining and valuing outcomes. The data gathering necessary for this step involves developing useful metrics as well as data collection methods. Specific metrics for youth programs will be further addressed in the next section of this report. As Figure 1.4 illustrates, there should be an outcome, an indicator, and a data collection method for each stakeholder. Figure 1.4 shows, for example, a young person as a stake holder. As a result of a social entrepreneurial activity by an organization, that young person has improved behavior. To measure this outcome, it is necessary to establish an indicator, in this case, a report from that young person s teacher. 18 Some outcomes will be more subjective than others making different types of indicators or data collection methods necessary. Increased self-confidence, for example, is more subjective. One indicator might be a survey of students who have completed a program stating that they feel more self-confident as a result. On the other hand, the success of work training is more objective. One indicator is likely to be how many students obtain employment. Data collection would involve tracking students who have received the mentoring. It is important to establish how long outcomes endure following the intervention. For example, an organization might determine how long student employment success be considered attributable to work mentoring. At some point, work experience will be more significant. 19 Figure 1.4: How to Measure and Value Outcomes Source: SROI Guide Social Return on Investment an Introduction. Op. cit., p A Guide to Social Return on Investment. Op. cit., p Ibid, p Ibid, p

10 In addition, the indicator must be quantifiable, which requires establishing a financial proxy such as potential salary earnings, cost to the state for incarceration, or cost to a school for dealing with a truant student. By quantifying the benefits in some way, the organization will be able to complete the SROI process. This may require significant research or valuations from stakeholders or other parties. Note that financial proxies can be double counted if the same benefits accrue to different stakeholders. 21 Figure 1.5 shows the impact map for this aspect of the SROI process, including the stages from describing an outcome to establishing its financial proxy. Figure 1.5: The Process of Valuing Outcomes Source: SROI Guide 2009 The Impact Map 22 For Wheels-to-Meals, the completed Impact Map is shown in Figure 1.6. The map illustrates that for the first outcome, better health of elderly participants resulting from exercise, the indicator was the number of falls and associated hospital visits. Following research, Wheels-to-Meals found that there was a change of seven visits over the period of a year (the estimated duration of the effect of the program). The financial proxy utilized by the organization in the SROI process was the cost of these visits to the NHS (the National Health Service in the U.K.) which was public information. For volunteers, the proxy for increased weekly activity was taken to be equal to the cost of a swimming pass since that would be the price of an alternative activity. 21 A Guide to Social Return on Investment. Op. cit., p SROI Guide 2009 The Impact Map. Op. cit. 10

11 Figure 1.6: Valuing Outcomes for Wheels-to-Meals Source: SROI Guide Establishing impact. Those aspects of change that would have happened anyway or are a result of other factors are taken out of the analysis. 24 Before completing the SROI calculations, it is necessary to establish what is actually attributable to the efforts of an organization s programming. For example, if there was an increase in the local economy of seven percent during the time of a program, but five percent could be attributed to national economic growth, then that is 23 A Guide to Social Return on Investment. Op. cit., p Social Return on Investment an Introduction. Op. cit., p

12 considered deadweight which should be subtracted from the final calculations. Of the remaining two percent, an organization would also need to determine for how much it could claim credit. Organizations might also ask stakeholders about which other services they had access to and utilized while they were participating in the activities of the organization completing the SROI process (e.g. if participants were utilizing two employment mentoring services simultaneously). As a general rule, organizations should look at trends before and after the implementation of their program and determine how much of this is occurring in the wider population and therefore how much can be attributable to their own programming. 25 Figure 1.7 shows outcomes and benchmark indicators that allow organizations to determine what impact is deadweight and what impact is theirs. Figure 1.7: How to Determine Deadweight Source: SROI Guide Displacement must also be subtracted from each outcome. This refers to how much of the outcome is displaced by other outcomes. For example, if ex-offenders were given job mentoring so they could pursue employment post-incarceration, then they might take away jobs from otherwise qualified people. This may not be applicable to many organizations. 27 The final two parts of this stage are attribution and drop-off. Attribution is an assessment of how much of the outcome was caused by the contribution of other organisations or people. 28 This means the part of the deadweight that an organization can correctly attribute to the actions of other organizations or efforts. 25 A Guide to Social Return on Investment. Op. cit., p Ibid, p Ibid. 28 Ibid, p

13 Past efforts of other organizations with participants may also be taken into account (e.g. a young person s educational experiences prior to joining a program). Drop-off is the amount by which the efforts of an organization s program lose impact over time. This is especially true for public relations campaigns to make a change such as encouraging medical testing or encouraging reduced energy use. Over time, the efforts of this campaign are reduced, which is usually calculated by deducting a fixed percentage from the remaining level of outcome at the end of each year. 29 Figure 1.8 illustrates how this step of the SROI process fits into the Wheels-to-Meals Impact Map. As an example, for the outcome of healthier volunteers it is possible that if the volunteers had not been volunteering for Wheels-to-Meals, they would have been volunteering elsewhere or engaging in some other physical activity. From an annual assessment of the volunteers, it was found that volunteers estimated they took 45 percent more exercise as a result of being involved in Wheels-to-Meals. Since the benchmark is 100 percent because all volunteers would have done some other exercise with or without being a volunteer, then the estimate of deadweight is 100 percent divided by 145 percent or 70 percent as shown in Figure 1.8. In terms of attribution, the elderly participants might still have joined other clubs and groups due to other people s suggestions. After asking participants, Wheels-to-Meals found that 35 percent of this outcome might be attributed to others A Guide to Social Return on Investment. Op. cit., p Ibid. 13

14 Figure 1.8: Calculating the Impact for Wheels-to-Meals Source: SROI Guide Ultimately, as shown in the final step of Figure 1.8, the organization calculates its impact by taking the quantity of the impact multiplied by the financial proxy and then deducting the deadweight, displacement, attribution, and drop-off as applicable. This results in a grand total for the social value impact of the organization. In the case of Wheels-to-Meals, this amount was 82, A Guide to Social Return on Investment. Op. cit. 32 Ibid. 14

15 5. Calculating the SROI. This stage involves adding up all the benefits, subtracting any negatives and comparing the result with the investment. This is also where the sensitivity of the results can be tested. 33 This final step results in finding informative ratios through various calculations. First the organization must calculate the value of each outcome for each time period and then subtract any drop-off occurring at each time period. Secondly, the organization calculates the net present value or present value using discounting which compiles the costs and benefits paid or received in different time periods. The discounting rate used varies depending on the value placed on the present as opposed to the future, though somewhere around three percent has been applied in the United Kingdom. 34 Detailed calculation examples are provided in the SROI Guide and various case studies through the SROI Network. There are two ratios the SROI Ratio and the Net SROI ratio. The SROI Guide states that both are acceptable as long as organizations clearly state which they are utilizing. SROI Ratio = Present Value / Value of Inputs Net SROI Ratio = Net Present Value / Value of Inputs Finally, organizations conduct a sensitivity analysis which uses a spreadsheet to change the numbers for a range of variables including estimates of deadweight, attribution and drop-off, financial proxies, quantity of outcome, and the value of non-financial inputs. The organization then analyzes how this affects the ratio. If the ratio changed significantly, then data and estimates are double checked for accuracy. Also, areas that have a significant impact could also be potential areas of improvement to increase social value in the future. 35 Wheels-to-Meals projected its impact five years into the future by adding its impact ( 81,742) and then dividing this amount by the total inputs ( 42,375) to gain a ratio of 1.93 to 1, meaning that the organization produces 1.93 of social value for every 1 of investment Reporting, using and embedding. This vital last step involves verification of the report, sharing findings with stakeholders and responding to them, and embedding good outcomes processes. 37 The final step in the SROI guide is to report the results of the analysis. The SROI Guide emphasizes that in communicating results to stakeholders, it is crucial to report more than the ratios. Organizations should report both the process and the results, 33 Social Return on Investment an Introduction. Op. cit., p A Guide to Social Return on Investment. Op. cit., p Ibid, p Social Return on Investment an introduction. Op. cit., p Ibid. 15

16 particularly in justifying and articulating their data collection process and calculations. Reporting the results of discussions with stakeholders illustrates the involved nature of the process and the accountability the organization takes in regards to its stakeholders. In particular, to make the report valuable to funders, the SROI Guide suggests that organizations include the following in their reports: information relating to your organisation, including a discussion of its work, key stakeholders and activities description of the scope of the analysis, details of stakeholder involvement, methods of data collection, and any assumptions and limitations underlying the analysis the impact map, with relevant indicators and any proxies case studies, or quotes from participants that illustrate particular findings details of the calculations, and a discussion of any estimates and assumptions. This section would include the sensitivity analysis and a description of the effect of varying your assumptions on social returns an audit trail for decision-making, including which stakeholders, outcomes or indicators were included and which were not, and a rationale for each of these decisions an executive summary aimed at a broad audience, including participants 38 In addition, organizations can use the opportunity of publishing their first SROI report as a catalyst for change. With the systems in place for evaluation and data collection, organizations now have a tool by which to measure the outcomes of their activities and to make improvements. Organizations should particularly emphasize ongoing communications with stakeholders to maximize social value in the future according to recommendations from those involved in the process. This also allows stakeholders to see change over time in the ratios Social Return on Investment an introduction. Op. cit., p Ibid. 16

17 The New Economics Foundation SROI Approach The New Economics Foundation (NEF) is another UK-based organization that has done substantial work formulating an SROI approach. NEF always uses the following SROI formula: SROI=Net present value of benefits/net present value of investment NEF outlines the following principles as central to its approach: Stakeholders are central: NEF s approach is focused on being peoplecentered and ensuring that it is based on stakeholders achieving their social objectives. Theories of change: This concept is central to understanding the cause-andeffect chain of inputs, outputs, outcomes, and impacts. Materiality: By this, NEF means that SROI analysis should allow organizations to gain better information and that by involving stakeholders, an organization can determine which information is important or material. Understanding Impact: This principle is about deadweight, displacement, and attribution allowing organizations to determine their specific impact. 40 These principles are very similar to those listed in the SROI Guide. However, the NEF is particularly focused on being people-centric and using stakeholders as the primary vehicle by which to engage in the SROI process. As such, NEF includes many recommendations for engaging stakeholders in the process. NEF recommends prioritizing stakeholders at the very beginning of the SROI process because it is neither possible nor relevant for you to consult all stakeholders. 41 As before, stakeholders should be engaged at the beginning of the process to understand their objectives and achievements, and then again during the data collection process. NEF notes that it may be necessary throughout the process to re-assess stakeholder groups and interests and adjust accordingly. NEF also notes the importance of clarifying why certain stakeholders might be excluded using a stakeholder audit trail. Board members, for example, might be excluded because they are not key beneficiaries though they may be important decision-makers. In engaging stakeholders, an organization should be looking to answer these questions: What are their objectives for this project or organization? What are their goals, beyond the scope of the project or organization? As a consequence, what is it important for the SROI to capture? Measuring Value: A Guide to Social Return on Investment (SROI). New Economics Foundation. Second Edition, p Ibid, p Ibid, p

18 In addition, the NEF provides the following best practices for stakeholder engagement: Using statistically robust sampling techniques to ensure that a representative range of stakeholder groups is included in the analysis Ensuring the independence and objectivity of those persons (internal or external) conducting the research Involving stakeholders in the design of the engagement process, and encouraging feedback Acknowledging differences among stakeholders Ensuring confidentiality Documenting the rationale and processes of stakeholder engagement 43 NEF also suggests gathering stakeholders together for this discussion potentially using a workshop format or at a regular meeting to ensure a productive discussion. Overall, the rest of the NEF s SROI approach is very similar to the SROI Guide 2009 covered earlier in this section. 43 Ibid, p

19 The Roberts Enterprise Development Fund SROI Approach The Roberts Enterprise Development Fund (REDF), a San Francisco-based venture philanthropy fund, pioneered the SROI methodology. The following discussion analyzes the differences between the SROI Guide approach and the REDF approach so as to illustrate options organizations can pursue in implementing the methodology. REDF provides an Excel model of its SROI approach, which contains the basic framework social enterprise organizations can use to calculate their SROI. Users of the SROI model need to have specific data to input in the spreadsheet, financial projections (for ten years forward), and must conduct research to make relevant industry and economic assumptions. REDF s objective is that: This model measures return on investment from three distinct perspectives. First, it examines the financial value created (or cash generated) by the social purpose enterprise. This model also examines social savings resulting from the increased tax revenues and the reduction in use of public assistance programs by target employees. These savings are compared to the total investment in the social purpose enterprise to determine the return on investment. 44 The Excel model contains five pages where organizations should input financial data. The calculations and data required by REDF are significantly more complex than that discussed in the SROI Guide 2009 due to a greater emphasis on financial accounting. For example, REDF heavily utilizes the concept of True Cost Counting Analysis (TCAA) which is also known as double bottom line reporting. TCAA allows a nonprofit organization running a business enterprise to more accurately track progress on its financial and social goals by separating subsidy revenue from business revenue, and social operating expenses from business expenses. This avoids the distortion that can result from consolidating sales and grants received in traditional non-profit accounting. REDF also focused on discounting, cost of capital, discounted cash flow, and net present value. 45 The REDF SROI process is divided into six stages of measuring value and return as shown in Figure SROI Excel Model. Roberts Enterprise Development Fund REDF s SROI Approach: Key Concepts and Financial Tools. The Roberts Enterprise Development Fund. 19

20 Figure 1.9: The Six Stages of the REDF SROI Process Measuring Value Measuring Return Calculate Enterprise Value Calculate Enterprise Index of Return Calculate Social Purpose Value Calculate Social Purpose Index of Return Calculate Blended Value Calculate Blended Index of Return Source: The Roberts Enterprise Development Fund 46 Essentially, the product of the REDF SROI process is a Blended Value showing the full monetizable value of the social enterprise by combining Social Purpose Value (social) and Enterprise Value (financial). 47 The combination of these concepts is shown in Figure Figure 1.10: Measuring Value in the REDF SROI Process Source: The Roberts Enterprise Development Fund REDF s SROI Analysis: The Process. The SROI Methodology. The Roberts Enterprise Development Fund. Spring p REDF's+SROI+Analysis.pdf 47 Ibid. 48 Ibid, p

21 The primary difference between the SROI Guide approach and the REDF approach is the level of financial calculation, particularly in regard to forecasting cash flow. The REDF Enterprise Value approach requires organizations to forecast cash flows and discount free cash flows using an appropriate discount rate which puts an organization into the context of its industry instead of focusing on the organization s individual social impact, regardless of projections. This approach largely seems to be more appropriate for organizations that are involved in some kind of product sales. To obtain more details on the REDF approach, Hanover Research recommends consulting REDF s SROI Methodology. 21

22 SROI Metrics for Youth and Educational Programs As noted in the previous section, organizations quantifying their SROI must develop outcomes and indicators as well as ways to measure these indicators. Indicators or metrics are a substantial part of the process. This section of the report will address possible metrics XYZ might use for this part of the SROI process in order to begin quantifying its accomplishments. The discussion will utilize ideas from various case studies involving education and youth programs to best address the short and longterm impact on students. It will ultimately compile a list of outcome metrics for XYZ to consider utilizing in measuring the impact of its remediation, intervention, and prevention services. Relating Outputs to Outcomes for Young People Some of the outcomes that XYZ may be looking to achieve in its programs might include: reduced drug use, higher rates of graduate and student employment, improved self-confidence, better grades, lower absenteeism, higher graduation rates, greater re-integration into traditional campuses, and higher college enrollment. However, before specifically considering the outcomes of an organization s programming and the financial proxies for these outcomes, it is important to relate program outputs to outcomes. A recent report supported by The Rockefeller Foundation cataloged different kinds of SROI methodologies and looked at the metrics for an after-school program, including the outputs and outcomes that are part of the impact value chain for the program. These outputs and outcomes are shown in Figure 2.1. Figure 2.1: The Impact Value Chain for an After-School Program Outputs Outcomes Number of children participating in program Higher self-esteem for participants Percent of children that drop out of Higher educational achievement for program participants Percent of children that re-enroll the following year Source: Double Bottom Line Project Report: Assessing Social Impact in Double Bottom Line Ventures 49 The organization providing the after-school program kept records on its outputs including the number of children participating continuously in the program. Its ultimate objectives (outcomes) were to increase self-esteem and educational 49 Clark, Catherine, Long, David, Olsen, Sara, and Rosenzweig, William. Double Bottom Line Project Report: Assessing Social Impact in Double Bottom Line Ventures. Supported by The Rockefeller Foundation

23 achievement for the young people who participated in the program. 50 After establishing this relationship between outputs and outcomes, the next steps in the SROI process are to assign indicators and financial proxies for each outcome. Appropriate Indicators and Financial Proxies for Outcomes As discussed in the first section of this report, to complete the necessary SROI calculations, it is necessary to establish measurable indicators and financial proxies for each outcome. The SROI Project provides an indicator and financial proxy database which is under constant development. Figure 2.2 is a compilation of outcomes related to prevention, intervention, and remediation services for young people. Each outcome is matched with an indicator and financial proxy in order to calculate the SROI. Note that the indicators and financial proxies included in the database are based on U.K. organizations so have been somewhat adjusted for U.S. purposes. The SROI Project notes several guidelines for utilizing these outcomes: Establish how long the outcome will last: Ideally this is determined by asking stakeholders how long an intervention lasted for them to gain an estimate. There is also research to support the duration of benefits. 51 o Example: A young person s earning potential can only be reasonably attributable to an organization s intervention for a certain period of time after which work experience will be more important Avoid Over-claiming: It is likely that some outcomes would have happened without an organization s intervention. As such, it is important to establish credibly (often through estimates) the actual impact of an organization s programming. Putting Value on Outcome: Some financial proxies can be more easily researched than others. For those that are more challenging, there are options such as: o Contingent Valuation: Ask people directly how they value something. o Revealed Preference: Build up a value from the market values of constituent parts of the service or good being considered. 52 Each indicator listed in Figure 2.2 is noted as being objective or subjective meaning that it can be measured by some objective tool (e.g. number of absences recorded by schools) or that it requires subjective measurement (e.g. mental well-being). Measurement tools to accommodate subjective indicators will be addressed later in this section. In the outcomes listed in Figure 2.2, young people themselves are often the stakeholders, but the stakeholders can also be law enforcement, taxpayers, social 50 Ibid. 51 Establishing how long outcomes last. SROI Project Putting a value on the outcome. SROI Project. 23

24 services, or educational authorities who benefit from the remedial, intervention, and prevention programming provided by an organization like XYZ. The stakeholders are obvious based on the financial proxy. For example, reduced anti-social and criminal behavior by young people reduces costs to law enforcement while increased skills increases employability and future earnings for young people. Figure 2.2: Indicators and Proxies for Outcomes Related to Young People Outcome Indicator Financial Proxy Number of children whose reading The amount by which a child s future Improved life chances for and math skills now reach national earning capability increases as a result children and state grade-level standards of them having a better reading ability (objective) Improved life chances for young people Increased confidence and personal and team skills Change in level and severity of youth offending Improvement in mental health Change in likelihood that a young person will go on to post-secondary education Change in likelihood that a young person will complete high school Freeing up resources associated with the care for children with conduct disorders Reduction in substance abuse Reduction in number of days a young person is truant and reduction in number of suspensions/expulsions at school pre/post intervention (objective) Personal and instructor evaluations, change over time for young person (subjective) Change in level of contact with criminal justice system; Change in severity of crimes committed; Change in number of crimes committed; Change in level of prosocial behavior (objective) Numerous measurement tools as discussed below such as Teen Outcomes Star (subjective) Number of students going on to post-secondary educational institutions (objective) Number of students completing high school rather than dropping out (objective) Number of children no longer requiring social services support (objective) Number of young people who were using drugs or abusing alcohol but who are no longer doing so (objective) The average cost of stopping a truant or the average cost of educating a permanently excluded pupil elsewhere or the average of cost of managing one exclusion for education authorities; The cost to social services for young people who are truant or excluded from school The difference in cost between providing the opportunities with an alternative commercial provider The cost to society of youth offending; The cost of anti-social behavior; The cost of keeping young offenders in custody or sending them to juvenile court for police and justice system Cost of special education and educational support for children with conduct disorders or disabilities; Cost of criminal damage, court and prison costs for a young person with conduct disorder who commits a crime The average extra income earned as a result of going to college rather than leaving school at 18 The average extra income earned as a result of completing high school rather than dropping out early The cost of residential and foster care for children with conduct disorders The average annual health care costs related to substance abuse results; The total annual cost to society of drug and alcohol-related problems, crime and anti-social behavior 24

25 Outcome Indicator Financial Proxy Number of young people showing improved academic results; Reduction in truancy and absenteeism; Reduction in antisocial and criminal behavior (objective) More appropriate support received due to improved monitoring of progress Increased support from family members through better coordination of support and identification of issues within the family through review process Improved employability Source: SROI Project Database 53 Number of young people and their families no longer requiring social work support (objective) Number of young people showing improved academic results; Number of young people completing high school or postsecondary education; Number of young people obtaining employment (objective) Cost of social work support for young person per intervention Average cost of providing a family social worker per vulnerable young person Cost to state of unemployment or welfare benefits for the unemployed or low-income For the first outcome listed in Figure 2.2, Improved life chances, the example used by the SROI Project is that of children involved in a program called the Reading Recovery Programme. Through the program, children with poor reading skills were given intensive one-on-one support in their second year of primary school. As a result, these children reached grade-level reading standards and had increased employment and salary prospects. Ultimately, the Reading Recovery Programme determined that every 1 spent on the program increased future earnings by 12. It costs the program 2,825 to teach a child to read proficiently and this has the potential to increase the child s future salary by 34,700 in total between the ages of 16 and 30 (2005 costs). 54 For the third outcome listed in Figure 2.2, Increased confidence and personal and team skills, the example used by the SROI Project is that of the Scottish Investment Fund which aimed to establish a centre of excellence that provides equal access to outdoor education opportunities and training, and provides learning opportunities for school and youth groups in a residential facility. The Fund researched the cost of alternative programs for young people such as private personal development programs and calculated that the financial proxy was savings of 150 per young person SROI Database. SROI Project SROI Database: Change in literacy skills of children in schools. SROI Project SROI Database: Increased confidence and personal and team skills. SROI Project. 25

26 Subjective Indicator Measurement Tools The Roberts Enterprise Development Fund notes that the SROI methodology does have limitations some outcomes are difficult to quantify. 56 As noted in Figure 2.2, there are subjective indicators necessary to measure certain outcomes. For example, improved family relationships and mental health status are not measurable through objective means, but require some kind of subjective evaluation either by the young people themselves and/or by an instructor or administrator who has monitored progress over time. Though subjective, there are effective measurement tools available to gauge these outcomes. The SROI Project suggests the following measurement tools which are often used to monitor the feelings of young people about their progress: Surveys: Surveys can measure intangible features such as confidence and wellbeing often by allowing participants to rank their emotions on a scale. Surveys can also be used to receive program feedback, evaluations, and improvement suggestions. 57 Outcome Star System Tool: These star systems allow organizations to measure change in variables not usually quantifiable such as Mental Health Recovery or Alcohol. Outcome areas for Mental Health Recovery include trust, hope, identity, and self-esteem. Participants rank themselves on a ladder for each of these elements with each step on the ladder related to a score which can then be recorded over time to quantify progress. 58 The Rickter Scale: This scale is specifically designed for young people. It measures various aspects of young people s lives such as stress, health, and confidence. It includes a hand-held adjustable board on which students can demonstrate how they feel about each area on a scale of 1 to 10. The premise behind this tool is to demonstrate the progress made by clients, moving from a disordered life, to having a sense of consistency. It can be adapted to work with different types of young people such as those struggling with disabilities or drug usage. 59 Employability Skills Wheel: This tool can be used with students to measure how they feel about their skills in certain areas related to employability. Students rate themselves for each skill e.g. dealing with authority. These skills are grouped under How I look, How I feel, and How I Perform. Closer to 56 Javits, Carla. REDF s Current Approach to SROI. The Roberts Enterprise Development Fund Surveys. SROI Project The Outcome Star system tool. SROI Project The Rickter Scale. SROI Project. 26

27 the hub of the wheel indicates less confidence in the skill and further away indicates greater confidence. 60 Step it Up Tool: This tool was designed for use by those working with youth. It assesses competence areas such as communications, relationship and problem-solving and comprises of six questionnaires with five point scales. 61 New Philanthropy Capital Wellbeing Questionnaire: This tool was specifically designed to measure the levels of well-being for 11 to 16 year olds. It involves observing children s levels of wellbeing and how it changes over time. Additionally, the tool allows for exploration of how intervention can affect different aspects of a child s wellbeing. 62 Evaluation Tree: This measurement tool is specifically used for young people because of its visual component. It measures well-being and general feelings by asking young people to point to a color or figure in the diagram of the tree and are then encouraged to discuss why this is and their ideal place on the tree. This is more helpful if there is significant time to see themes and links in a young person s responses. 63 Strengths and Difficulties Questionnaire: This is a brief behavioral screening questionnaire about 3-16 year olds. It includes 25 items on psychological attributes, an impact supplement, and follow-up questions. The psychological attributes include emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems, and pro-social behavior. 64 Children s Self-Concept Scale: The Pier-Harris Scale is designed to determine psychosocial functioning and perceived social support. It is an 80-item self-report instrument designed to assess how children and adolescents feel about themselves in areas of physical appearance, popularity, happiness, anxiety and intellectual status Employability Skills Wheel. SROI Project Step It Up tool. SROI Project New Philanthropy Capital Wellbeing Questionnaire. SROI Project Evaluation tree. SROI Project What is the SDQ? Youth In Mind Piers-Harris Children s Self-Concept Scale. 27