Innovative Outcome Evaluation and Social Return On Investment Analysis of an FASD Mentorship Program

Transcription

1 Innovative Outcome Evaluation and Social Return On Investment Analysis of an FASD Mentorship Program The Experience of the McMan LifeSpan Program in Southern Alberta Hélène Wirzba, Wirzba Consulting Inc. 6 th National Biennial Conference on Adolescents and Adults with FASD April 9-12,

2 Innovative Outcome Evaluation and Social Return On Investment Analysis of an FASD Mentorship Program The Experience of the McMan LifeSpan Program in Southern Alberta I don t have any conflict of interest. 2

3 Innovative Outcome Evaluation and Social Return On Investment Analysis of an FASD Mentorship Program Thank You to: McMan Youth, Family and Community Service Association, South Region Alberta FASD Cross Ministry Committee South Alberta FASD Service Network Alberta Justice, Safe Communities Innovation Fund OrgCode Consulting Inc. Simpact Strategy / Canada SROI Network 3

4 Learning Objectives 1. The FASD LifeSpan Program 2. SPDAT Service Prioritization Decision Assistance Tool and Supplementary Forms 3. SROI Social Return On Investment Analysis 4. Outcomes and SROI findings 5. Use of Evaluation Findings to Improve the Practice 6. Lessons Learned and Replication 4

5 1. The FASD LifeSpan Program 1.1 Locations 2 locations: Urban Lethbridge Since 2009 (Operational Funding) Funded By the South Alberta FASD Network through the Alberta FASD Cross-Ministry Committee Rural South West Alberta, Pincher Creek and Crowsnest Pass Since 2011 (3 year funding) Funded by Safe Communities Innovation Fund (SCIF), Safe Communities and Strategic Policy, Alberta Justice and Attorney General Requirement to include evaluation and SROI. Theory of Change: When individuals with FASD receive supports and services meeting their lifelong special needs, and when caregivers feel supported in their role, they will experience more stability and the quality of their lives will improve. 5

6 1. The FASD LifeSpan Program 1.2 Primary Stakeholders 2 Primary Stakeholders Primary stakeholder #1: youth transitioning to adulthood and adults with or suspected of having FASD. Intensive case management approach, facilitated by professional mentors. Continuum of services guided by individualized service plans. 106 unique clients since 2011 Primary stakeholder #2: caregivers of adults, youth and children with FASD: Information, one-on-one counseling, support groups, virtual support (Facebook), and referral to other community supports Services usually time-limited and based on individual needs About 25 unique caregivers per year. 6

7 1. The FASD LifeSpan Program 1.3 An emphasis on evaluation As a member of the Commission on Accreditation of Rehabilitation Facilities (CARF) McMan focuses on quality and results. Preliminary 1 st year findings suggested that the Lethbridge LifeSpan program resulted in a decreased criminal involvement for primary stakeholders by addressing the risk factors for crime. Desire to build evidence about the practice? SCIF grant included designated funding for evaluation and SROI. 7

8 1. The FASD LifeSpan Program 1.4 Evaluation Challenges Most youth and adults had a suspicion of FASD; limited opportunities for a multidisciplinary FASD assessment and diagnosis followed by recommendations. Only a few FASD support programs systematically evaluated (mostly in a the context of a research project). No consensus on realistic outcomes and indicators for FASD support programs. No standardized assessment tools tested for adults with FASD. ( 8

9 2. SPDAT and Supplementary Forms 2.1 Introduction Service Prioritization Decision Assistance Tool (SPDAT). Used with 10,000 unique individuals in more than 100 communities across North America. An evidence-informed tool designed to assess the housing needs of homeless individuals and families ( SPDAT uses 15 dimensions to inform professionals about the following: Assist in determining the type of interventions and the intensity of services; Provide assistance with case planning and encourage reflection on the prioritization of case plan elements; and Track the depth of need and service response to clients over time. 9

10 2. SPDAT and Supplementary Forms 2.2 SPDAT Elements 1. Self-care and Daily Living 2. Meaningful Daily Activity 3. Social Relationships and Networks 4. Mental Health and Wellness 5. Physical Health and Wellness 6. Substance Use 7. Medication 8. Personal Administration & Money Management 9. Personal Responsibility and Motivation (Now Abuse and/or Trauma) 10. Risk of Personal Harm / Harm to Others 11. Interaction with Emergency Services 12. Involvement in High Risk and Exploitive Situations 13. Legal 14. History of Homelessness and Housing 15. Managing Tenancy 10

11 2. SPDAT and Supplementary Forms 2.3 SPDAT Elements Graphing / Client Specific Graphing Changes: Female Client 28 years old Aboriginal Rural 11

12 2. SPDAT and Supplementary Forms 2.4 SPDAT Elements Graphing / Client Specific 12

13 2. SPDAT and Supplementary Forms 2.5 SPDAT Elements Graphing / Program Overall SPDAT Score by Gender 13

14 2. SPDAT and Supplementary Forms 2.5 SPDAT Elements Graphing / Program Scoring: Legal 14

15 2. SPDAT and Supplementary Forms 2.6 SPDAT Supplementary Qualitative Data 1. Demographic 7. Substance Use Information 2. FASD Status and Change in Status 3. Meaningful Daily Activity 4. Availability of a caregiver 8. Legal source of income 9. Occurrence of Sexual and other Exploitive Situation 10.Legal issues: occurrence, 5. Mental health issue charges, and 6. Medical, Dental and disposition Vision Check-ups 11.Housing 15

16 2. SPDAT and Supplementary Forms 2.6 How has the program used SPDAT (1/2)? To screen clients referred to the program (SPDAT 0): Initial assessment at referral time. If space permits, clients admitted immediately If not, then intake staff ensures that high scored elements addressed either in-house or through referrals To guide the initial development of a case management plan (SPDAT 30): Ensures that the assessment is thorough (no missed elements) Provides a standard guide for discussion with clients and other stakeholders Helps identify higher priorities 16

17 2. SPDAT and Supplementary Forms 2.6 How has the program used SPDAT (2/2)? To track the depth of need and service responses over time (ideally every 3 months) Decrease in scoring tracks whether objectives are achieved Increase in scoring helps identify new priorities / issues To determine the intensity of case management Consistent low scoring over a period of time may indicate that client may be ready for less intensive supports in a particular area. To ensure consistency in assessments, and smooth transition from one life coach to the next. Ideally, if people are well trained, they should use very similar scoring. 17

18 3. Social Return On Investment 3.1 Definition The purpose of an investment is to generate a return. SROI illustrates the value of change to individuals, families and community circumstances by expressing the value of that change, in a dollar value wherever possible. SROI is a combination of social, financial and environmental value. SROI is a story, not a number. SROI Canada

19 3. Social Return on Investment 3.2 Why do an SROI? Understand the social, environmental and economic value of the project; Maximize the change created by the project and manage negative outcomes; Increased engagement with stakeholders; Collect more useful and better data; Promote your work. Benefits: Strategic planning Managing activities Communication Attracting investment or making investment decisions 19

20 3. Social Return On Investment 3.3 SROI Values Change 1. An investment is made into community. 2. An activity is resourced and something happens as a result. 3. Tracking the result (outcome evaluation) illustrates that change has occurred. 4. Where possible, the change in valued in financial terms. 5. The value of change is presented in relation to the value of the investment made (ratio). 20

21 3. Social Return on Investment 3.4 The 7 Principles of SROI 1. Involve stakeholders 2. Understand what changes 3. Value the things that matter 4. Only include what is material 5. Do not over claim 6. Be transparent 7. Verify the results 21

22 3. Social Return On Investment 3.5. SROI Methodology (1) Establish scope of the project and identify stakeholders Map outcomes Evidence outcomes and give them a value Establish impact Calculate SROI Report, use and embed 22

23 3. Social Return On Investment 3.6 Monetizing Changes Changes in household income Increase in taxes paid Cost reallocation (diverting need for public support or human services) Cost savings due to a shrinking problem or reducing need or reduced cost of service delivery Improvement in personal well being 23

24 3. Social Return On Investment 3.7 Reality Check Questions DEADWEIGHT %: what would have happened without the support. DISPLACEMENT %: has the program just moved things around, rather than creating a real change? ATTRIBUTION: Has anyone else contributed to the achievement of these outcomes? If so, what % can the project claim? DROP OFF: How long will the change last? Would it last beyond the year, or does the support need to continue? Percentages need to be determined for each of the indicators included in the SROI analysis. 24

25 3. Social Return On Investment 3.8 SROI and Other Approaches Outcomes evaluation Measuring outcomes is common to many evaluation approaches, including SROI SROI emphasizes the involvement of stakeholders SROI assigns financial value to outcomes Cost-benefit Analysis: Both use money as a proxy of costs and benefits arising from an investment SROI is a management tool; focuses on the local environment; includes perspective of different stakeholders. Cost-benefit analysis takes a macro perspective; values costs and benefits for the whole society. 25

26 4. Outcomes and SROI Findings 4.1 Demographic information Number of youth or adults with FASD: 106 unique clients (41% female and 59% male) Initial Assessments: 69 (43% female & 57% male) Unique clients in remainder of year 1: 48 (44% female & 56% male) 96 assessments Unique clients in year 2: 37 (38% female, 62% male) 96 assessments Unique clients in year 3: 24 s (25% female, 75% male) 48 assessments Location of services: Urban (75%) versus Rural (25%) In urban areas: 38% female, 62% male In rural area: 50% female, 50% male Whether clients identify as Aboriginal: Urban: 50% of women and 52% of men identify as Aboriginal Rural: 92% of women and 85% of men identify as Aboriginal Age at admission: 16 to 57 years old (58% 24 year old or younger, 42% 25 years old or more) 26

27 4. Outcomes and SROI Findings 4.2. Meaningful Daily Activity Outcomes: 6% returned to school 14% found employment % of clients with no legal source of income decreased from 39% to 4% 20% reconnected with families and caregivers Financial Proxy: Part-time income for one year ($9,400/year) 27

28 4. Outcomes and SROI Findings 4.2. Housing and Homelessness Outcomes: 19% homelessness at admission only 1% in subsequent years 33% rehoused every year Financial Proxy: Cost of homelessness: $14,398/year Costs of damage to landlords: $675/year 28

29 4. Outcomes and SROI Findings 4.3 Mental Health Outcomes: 73% reconnected with a physician. Increase in the percentage of people diagnosed with a mental health issue from 44% to 67% Increase in the percentage of people treated for a mental health condition from 27% to 52% Financial Proxy: Short-term work loss due to undiagnosed mental health issues: $3,551/year 29

30 4. Outcomes and SROI Findings 4.4 Legal issues Outcomes: Decrease in percentage of persons with current legal issues from 55% at admission to 35% in years 2 and 3. 66% fully met legal requirements while in program % with new legal issues every year (more minor issues) Financial Proxy: Cost of crime among homelessness people: $14,935 (IHE) 30

31 4. Outcomes and SROI Findings 4.5 SROI calculation SROI Analysis took into account the value created for individual clients, their caregivers, and other stakeholders Increased income and resources for clients and caregivers More streamlined use of community resources Increased compliance with legal dispositions and decreased use of justice system Decreased use of transitional housing Increased wellbeing of clients and caregivers SROI Timeline Input Total Present SROI Ratio Value Funding Year (TPV) (TPV / Input) $518,209 $873, : $682,575 $1,329, : $696,600 $1,589, : 1 Total $1,897,384 $3,792, : 1 31

32 5. Use of Evaluation Findings to Improve the Practice Improved screening process Informed case management decisions Ability to track client outcomes over time Ability to demonstrate program outcomes and social return of the program Recognition of the role played by all stakeholders in meeting outcomes Evidence that persons with FASD have a permanent disability, and that outcomes are met to a certain extent only Increased ability to promote the practice 32

33 6. Lessons learned and Opportunities for Replication (1/2) There are positive outcomes and social returns in tertiary prevention programs for adults with an FASD. It is possible to successfully integrate the tracking of outcomes in client case management practices. SPDAT is a tool that is relevant as an assessment and monitoring tool for FASD support programs. A sound methodology, documentation, and transparency are necessary when analyzing outcomes and doing a SROI analysis. Programs need specialized evaluation resources and expertise to do that type of work. 33

34 6. Lessons learned and Opportunities for Replication (2/2) SPDAT : Contact OrgCode Consulting: info@orgcode.com Get all staff and supervisors trained No other cost For information on how to track SPDAT, or a copy of supplementary forms (for qualitative information): Contact Hélène Wirzba: wirzbah@telus.net SROI Analysis: Find a practitioner: Costs: time / community consultation / financial resources 34

35 References McMan Youth Family and Community Services Association Alberta FASD Cross Ministry Initiative Evaluation of FASD programs SPDAT OrgCode Consulting SROI Canada Simpact Strategy SROI Network International 35

36 Questions? Hélène Wirzba Wirzba Consulting Inc or Marge Fisher Director of McMan Southwest Region ext. 229 or 36