SA HEALTH EVALUATION OF THE INDIVIDUAL PSYCHOSOCIAL REHABILITATION & SUPPORT SERVICES (IPRSS) PROGRAM

Transcription

1 SA HEALTH EVALUATION OF THE INDIVIDUAL PSYCHOSOCIAL REHABILITATION & SUPPORT SERVICES (IPRSS) PROGRAM FINAL REPORT MAY 2011 Health Outcomes International Suite 4, 51 Stephen Terrace, Kent Town SA 5069 Phone: Facsimile: ABN

2 C CONTENTS EXECUTIVE SUMMARY... 1 E.1 The IPRSS program... 1 E.1.1 Introduction... 1 E.1.2 Program description... 1 E.1.3 Program activity... 2 E.2 The evaluation objectives and design... 2 E.2.1 Evaluation objectives... 2 E.2.2 Evaluation design... 2 E.3 The consumer journey... 3 E.3.1 Needs identification, referral and allocation... 3 E.3.2 Access... 3 E.3.3 The individual support plan (ISP) process... 3 E.3.4 Exiting the program... 4 E.3.5 Recovery orientation... 4 E.4 The partnership... 4 E.5 Governance... 5 E.6 Consumer outcomes... 5 E.7 Recommendations... 6 INTRODUCTION Background to IPRSS Brief description of program IPRSS program objectives Evaluation objectives Evaluation design and methodology Report structure THE CONSUMER JOURNEY Key elements of service model Program activity Consumer characteristics Needs identification, referral and allocation i

3 2.5 Individual support plans and reviews Development of ISPs ISP reviews Carers perspective Exiting the program Discharges Readmission Recovery orientation Recovery Consumer involvement Carer involvement Service approaches Accessibility Aboriginal consumers CALD CARS data quality THE PARTNERSHIP IPRSS partnership survey introduction Key findings Detailed analysis Determining the need for the partnership Choosing partners Making sure partnerships work Planning collaborative action Implementing collaborative action Minimising the barriers to partnerships Reflecting on and continuing the partnership Building community relationships/capacity G OVERNANCE Key governance elements Program management committee Funding Regional allocation committees Quality Psychosocial rehabilitation service standards Workforce development ii

4 4.5.3 Workforce supervisory and support structures Key findings CONSUMER OUT COMES Introduction Approach to analysis Impact on NOCC assessment scores overall summary Impact on NOCC scores HONOS Impact on LSP Impact on K Impact on inpatient admissions Impact on government community mental health services Consumer case studies CONSUMER PERSP EC TIVES Introduction to REE survey Feedback on elements of recovery enhancement programs Achievement of recovery markets Organisational climate Consumer satisfaction surveys CONCLUSION APPENDICES ABBREVIATIONS -..I APPENDIX A PROGRAM STATEMENT... III APPENDIX B IPRSS PROGRAM LOGIC... III APPENDIX C ADDITIONAL PROGRAM ACTIVITY TABLES... IV APPENDIX D BRIEF DESCRIPTION OF OUTCOME TOOLS IN USE... VI APPENDIX E CONSUMER CASE STUDIES... XI APPENDIX F REE SURVEY ANALYSIS... XIV APPENDIX G- NGO PROVIDER CONSUMER SATISFACTION SURVEYS... XXXV APPENDIX H- CARS DATA QUALITY...XXXVIII iii

5 T TABLES Table 1.1: Report structure...12 Table 2.1: Gender by provider...16 Table 2.2: IPRSS consumer complexity...16 Table 2.3: Average days from referral to first contact Table 2.4: Hours per consumer...21 Table 2.5: Goal profile - August Table 2.6: Number of ISP reviews recorded on CARS compared to expected...23 Table 2.7: ATSI status by region...30 Table 2.8: CALD consumers by region...31 Table 3.1: Categorisation of total scores...33 Table 3.2: Determining the need for the partnership...35 Table 3.3: Choosing partners...36 Table 3.4: Making sure partnerships work...37 Table 3.5: Planning collaborative action...37 Table 3.6: Implementing collaborative action...38 Table 3.7: Minimising the barriers to partnerships...39 Table 3.8: Reflecting on and continuing the partnership...39 Table 4.1: IPRSS funding...43 Table 4.2: Quality review status...44 Table 4.3: Qualification profile Table 4.4: Summary of additional training opportunities offered...47 Table 5.1: Change in NOCC assessment scores pre and post IPRSS commencement...51 Table 5.2: Change in per item and subscale HoNOS score pre and post IPRSS commencement 1,2, Table 5.3: Change in per item and subscale HONOS 65+score pre and post IPRSS commencement...53 Table 5.4: Change in adult per item and sub scale LSP-16 score a pre and post IPRSS commencement...54 Table 5.5: Change in older people per item LSP-16 score pre and post IPRSS commencement...55 Table 5.6: Change in adult per item K 10 score pre and post IPRSS commencement...56 Table 5.7: Change in older people per item K 10 score pre and post IPRSS commencement...56 Table 5.8: Change in acute mental health admissions and LOS pre and post IPRSS commencement per consumer...57 iv

6 Table 5.9: Change in non mental health total admissions and LOS pre and post IPRSS commencement per consumer Table 5.10: Change in total admissions pre and post IPRSS commencement per consumer Table 5.11: Change in LOS for all admissions pre and post IPRSS commencement per consumer Table 5.12: Change in government community health service contacts pre and post IPRSS Table 5.13: Areas of improvement from IPRSS consumer case studies Table 6.1: Overview of consumer feedback v

7 F FIGURES Figure 2.1: IPRSS program service model overview Figure 2.2: Number of consumers by provider April 09 to August Figure 2.3: Total consumer hours by each provider April 2009 to August Figure 2.4: Regional distribution of consumers Figure 2.5: Age profile Figure 2.6: Overall gender distribution Figure 2.7: Referrals per month Figure 2.8: Average service hours per consumer per month Figure 2.9: Average length of stay by provider Figure 2.10: Discharges per month Figure 2.11: ATSI consumers Figure 2.12: CALD consumers Figure 3.1 Overall strength of partnership vi

8 E EXECUTIVE SUMMARY appointed Health Outcomes International (HOI) on 10 May 2010 to undertake an evaluation of the Individual Psychosocial Rehabilitation and Support Services (IPRSS) program. E.1 THE IPRSS PROGRAM E.1.1 INTRODUCTION The non-government sector is an integral component of an effective mental health system recognised for its distinctive contribution in many jurisdictions both nationally and internationally. Within South Australia there is an increasing emphasis on funding to the non-government sector for the provision of community mental health services and the IPRSS program in particular, in line with Social Inclusion Board recommendations. There are seven non-government organisation (NGO) providers involved in the delivery of the IPRSS program. E.1.2 PROGRAM DESCRIPTION The IPRSS program is based on an evidence based model of care and supports people with severe mental illness and a high level psychiatric disability on their recovery journey. It commenced on 1 April A key element of the IPRSS program is the partnership between NGO providers and government mental health services (MHS) with the needs of the shared consumers being central to decision making. A recovery philosophy underpins the delivery of psychosocial rehabilitation and support services. IPRSS NGO providers and government MHS work in partnership with other key stakeholders including housing to provide structured, goal focused and individually tailored services at a level of intensity and duration appropriate to consumers needs. An important component of individual psychosocial rehabilitation is community capacity building. Service types include: services delivered to assist the consumer engage in meaningful daytime activity and employment; services delivering combined housing and support programs provided the service is not facility based; services delivered in community settings intended to promote community engagement and social connectedness; independent living skills support and training to enable day to day living in the community; and transition from facility based services to home and community living. The service model is summarised in the following diagram. 1

9 E.1.3 PROGRAM ACTIVITY For the period 1 April 2009 and 31 August 2010 (the period for which data was made available) a total of 936 consumers received an IPRSS service. The total hours recorded were 169,363 compared to 172,380 funded hours, a variation of 1.8% which we consider to be minor in the context of the evaluation. Fifty one percent of consumers were serviced by NGO providers operating in the central and northern regions, which correlates with the 52% of funds being allocated to NGO providers in those regions. The majority of consumers fall within the 20 to 49 year age bracket (73%). There were 21 consumers less than 20 years of age which reflects the fact that the program is not focused on younger consumers. For adult services, males comprise the greater share of consumers (59%) and for older people services females comprise 74% of consumers, the higher percentage being attributed to higher male mortality rates. The complexity of IPRSS consumers (as measured by HoNOS), is significantly greater than the average national ambulatory mental health consumer for both adults and older people. This indicates that IPRSS is reaching the target group. E.2 THE EVALUATION OBJECTIVES AND DESIGN E.2.1 EVALUATION OBJECTIVES The evaluation aims to build a strong evidence base for the provision of best practice psychosocial rehabilitation and support services in South Australia and specifically sought to: E.2.2 assess whether the IPRSS program was implemented as planned determine whether consumer outcomes were optimised improve IPRSS program arrangements and performance inform future service planning, delivery and funding and make recommendations regarding specific aspects of the service that require review or development. EVALUATION DESIGN A number of qualitative and quantitative processes have been utilised to evaluate the program. These include: an analysis of Consumer Activity Reporting System (CARS) program data in respect to consumer demography, access to the program, and program activity an analysis of government community mental health outcome data and inpatient admission data to review the potential impact of the program an analysis of consumer perspectives obtained from the REE tool survey administration of the partnership analysis survey tool developed by VicHealth designed to test various aspects of the partnership attendance at regional allocation committee meetings in all metropolitan areas and two country regions (Murray Bridge and Port Pirie) as an observer and having group discussions about the program where time permitted (i.e. Eastern, Murray Bridge and Port Pirie) stakeholder consultations with government MHS team leaders and managers, NGO providers and the Mental Health Carer Advisory Group. 2

10 E.3 THE CONSUMER JOURNEY E.3.1 NEEDS IDENTIFICATION, REFERRAL AND ALLOCAT ION Regional allocation committee meetings are resource intensive but provide a valuable role based on the current program structure. They are a key element of the service partnership. Regional allocation committee meetings and associated processes help to ensure the program is reaching the right consumers. Allocation committees that cover multiple programs facilitate a streamlined service approach for the consumer. Regional allocation committees vary in the priority setting process for consumers entering the IPRSS program. There can be significant waiting lists and it is not clear how consumers who have been on the list for some time are reprioritised when a vacancy arises. There is limited discussion on the capacity of NGO providers to accept referrals, and the provision, for example, of an NGO provider regional monthly report to allocation committee meetings, incorporating current consumers and hours allocated per consumer, would further strengthen the service partnership and help to prioritise need. E.3.2 ACCESS Aboriginal and Torres Straight Islanders (ATSI) currently comprise 5.3% of IPRSS consumers and this is relatively constant between regions, although the number of unknowns recorded in country is high (27%). Overall 13% of consumers identified as having a CALD background, relatively consistent across regions. ATSI and CALD groups access the program to the extent that they are referred by government MHS and some regions have implemented innovative solutions to ensure ATSI have access to the program. It is not clear whether the IPRSS program is the most suitable model for ATSI people experiencing psychological distress and there is no benchmark for the percentage of ATSI IPRSS consumers. E.3.3 THE I NDIVIDUAL SUPPORT PL AN PROCESS In the majority of instances the development and review of Individual Support Plans (ISPs) works well. The average time from referral to first conact was 16.9 days. Variations between providers and other data held by the Department of Health are likley to reflect recording and definitional issues. There are no benchmarks established for the program in this area. When setting benchmarks, the time from the need being identifed to the time support begins to be provided is the key. Any benchmark or indicator established would need to take into account the steps and responsibilities across this pathway. In a number of situations it is possible that the Mental Health Care Plan and the Individual Support Plan are not compatible potentially causing conflict between the non-government and government service provider. ISPs set consumer goals across a range of domains. Supporting and promoting self-management/health and wellbeing is the most frequent goal domain with 93% of consumers having a goal in this area. The average hours of NGO provided service per consumer per month is 18.6 hours (median 15 hours). There are significant variations between NGO providers. While it is reasonable that there are variations between consumers, the overall average variation between NGO providers needs to be investigated. Overall the average length of stay in the IPRSS program is 333 days (11 months). There are some significant variations between NGO providers for which there are no obvious explanations and this requires further investigation. 3

11 Government mental workers attendance at ISP reviews is variable in some cases. There has been an increased workload for government mental health workers because of the requirement to attend ISP reviews. There is a need for improved co-ordination of reviews and better articulation of roles and responsibilities in operational protocols and there should be a more concerted effort to involve carers in the ISP process where appropriate. There are instances when the IPRSS program does not effectively engage with consumers to achieve program outcomes through effective goal setting, the provision of meaningful activities and/or effective community connection. E.3.4 EXITING THE PROGRAM For consumers and carers a seamless transition between services and a clear service pathway is critical. The service partnership requires that there is timely communication around exiting consumers from the program which occurs in most, but not all, instances The empirical data indicates that the group of consumers within the evaluation period were being discharged appropriately (i.e. a low number of 11 readmissions with the average time from discharge to readmission being 6.3 months). Qualitative feedback indicated there may be a need for some adult consumers to receive time limited support while exiting the program when they are no longer an active consumer of the government MHS. Developing more specific exit guidelines would assist in strengthening the exit process and enhancing the consumer journey. E.3.5 RECOVERY ORIENTATION NGO providers have demonstrated an understanding of a recovery orientation and have incorporated it into their organisational culture. Care needs to be (and is) taken in the way that the philosophy and ideals are meaningfully translated to elderly consumers. Service providers are active in involving consumers at the service level and NGO providers are constantly working on ways to include consumers in program management. Service providers try and accommodate and encourage the involvement of carers to some degree. However as most consumers indicate that they do not have carer involvement, service providers are not as active as they could be in promoting carer involvement. E.4 THE PARTNERSHIP A key element of the IPRSS program is the partnership between NGO providers and government MHS. Approximately half of the manager level stakeholders (46%) felt a partnership based on genuine collaboration had been established with the challenge being to maintain its impetus and build on current success. The remaining 54% felt the partnership was moving in the right direction but will need more attention if it is going to be really successful. No one indicated that the whole idea of partnership should be rigorously reviewed. Overall there is a clear commitment to continuing and strengthening the collaboration in the medium term. Critical success factors identified by NGO providers who reported strong partnerships included: developing good working relationships at the senior and middle management level which then filters down to NGO support workers, government MHS providers and eventually the consumer 4

12 developing open and honest relationships at every management level where each partner recognises and respects the role and responsibilities of the other and adopts the intent of the operational protocols recognising from the start that the teams are not always going to agree but the consumers interests must always be at the heart of all decisions/ discussions. Where these success factors do not exist it can lead to NGO providers and the contribution they make being undervalued. The majority of people who felt that the partnership still needed some improvement, identified that many of the problems they were experiencing arose from different conceptual understandings of the recovery approach and the role of the NGO support worker in the IPRSS program. Partnership success could be improved by greater clarity of vision, roles and responsibilities, clearly articulated boundaries and concepts of recovery and recovery practice. One practical opportunity for improvement around which most agreed was the need to review and update the operational protocols and conduct joint training sessions on the recovery approach and further definition around the role of the NGO support worker, supplemented by action learning case studies. This would assist in developing a shared understanding of, and commitment to respective roles and responsibilities. E.5 GOVERNANCE The governance structure has served the program well since its inception Regional allocation committee meetings are resource intensive but provide a valuable role based on the current program structure. They are a key element of the service partnership. The program has been operating for two years and it is timely for the leadership i.e. the Program Management Committee (PMC), to recommit to the shared vision required to sustain the success of the program. There is no structured approach to monitor the performance of individual service providers or regions. HOI consider the program would benefit if an annual contract management meeting was conducted with each service provider. Regional allocation committee meetings could be utilised more strategically to assist the PMC in implementing strategic requirements at the local level and also to further strengthen the partnership. There is a strong demonstrated commitment to quality in the program. Consideration should be given to endorsing an approach where only one recognised quality process need be adopted. Overall 87.8% of the staff have a relevant qualification with 41% having a Bachelor degree and there is an active staff development program offered by NGO providers. Opportunities for joint training initiatives with government mental health providers should be explored. Promotion of the non-government support worker role as a career opportunity is an important strategy to improve service outcomes. NGO provider supervisory and support structures are satisfactory. E.6 CONSUMER OUTCOMES The IPRSS program has had a positive impact on consumer outcomes. There have been significant decreases in mean scores across all National Outcomes and Casemix Collection (NOCC) indicators. In particular: HoNOS adult score has improved by 14.3% to 12.3 HoNOS 65+ for older persons score has improved by 11% to

13 K10 adult score has improved by 8.7% to 20.1 K10 older persons score has improved by 12.2% to 19.4 (note very small sample) LSP 16 adult score has improved by 9.1% to 15.5 LSP 16 older persons score has improved by 19.4% to 9.0. An analysis of consumer outcomes from the case studies and feedback from consultations reflect that NGO providers are taking a holistic approach when goal setting. They are identifying issues across a wide range of domains and working with consumers to address them. Increased independence, improved health (both physical and mental) management, social connectedness, family relationships and improved domestic and self-care skills are common areas being addressed. Having said that, carer and consumer feedback indicates that while they are very positive about the program and its staff, they feel that it does not always: provide real choices, desirable options and opportunities for participants assist participants to become involved in meaningful activities support them in building or rebuilding positive relationship with family members or friendships with people outside the mental health system. and further analysis, discussion and training with staff around these issue needs to occur. The IPRSS program has had a very positive impact on the rate on mental health related hospital admissions and associated average length of stay (ALOS). The rate of mental health admissions have reduced by 39% and ALOS has reduced by 16%. The IPRSS program has had a positive impact on the rate of non-mental health related hospital admissions and associated ALOS in that consumers are more aware of their physical health needs and attending to them (increase in rate of admission of 20%), and when they are in hospital, they are more capable of being discharged home earlier than was the case previously (ALOS reduced by 60%). E.7 RECOMMENDATIONS 1. Regional allocation committees vary in the priority setting process utilised for consumers entering the IPRSS program. To ensure equity and consistency it is recommended that: The Department of Health (DH) and the PMC should review the current priority ranking process for entry into the program and establish an agreed state-wide approach. 2. Aboriginal and Torres Strait Islanders (ATSI) currently comprise 5.3% of IPRSS consumers. While this is greater than their overall percentage of the population, ATSI people are two and a half times as likely as non-indigenous people to have experienced high/very high levels of psychological distress. It is not clear whether the IPRSS program is the most suitable model for ATSI people experiencing psychological distress and what should be an appropriate number of ATSI consumers. Some regions are exploring innovative ways to increase ATSI access to the IPRSS program. It is recommended that: All partners should continue to explore innovative solutions to ensure Aboriginal and Torres Strait Islanders have ongoing and increased access to the program. 3. The average hours of service per consumer per month is 18.6 hours (median 15 hours). There are significant variations between providers in average hours of service per consumer per month and average length of stay on the program. While it is reasonable that there are variations between consumers, the size of the overall avearge variations needs to be investigated. It is recommended that: The DH investigates variations between providers in average hours of service per consumer per month and average length of stay on the program. 6

14 Monitoring and review should highlight the need for further analysis and discussion if variations are significant and continual. It is important that the quality of the services being provided is analysed as well as indentification of the number of hours. 4. For consumers and carers a seamless transition between services and a clear service pathway is critical. In the majority of situations the quantitative and qualitative data indicates that consumers are being discharged appropriately. However the exit process and partnership would be strengthened if there were more specific exit guidelines to assist with the exit process and ensure there is always timely communication around the exit process. For adult consumers there may be a need for some consumers to receive time limited support while exiting the program, when they are no longer an active consumer of the government mental health service. It is recommended that: To strengthen the exit process, DH and PMC review the exit process and consider developing guidelines for staff. 5. Feedback from carers indicates that service providers could be more active in promoting carer involvement in the program and there should be a more concerted effort to involve carers in the ISP process where appropriate. It is recommended that: The operational protocols and their application are reviewed to enhance the involvement of carers in the program at all levels. 6. There are instances when the IPRSS program does not effectively engage with consumers to achieve program outcomes. Areas that were consistently raised as being problematic include: Goal Setting: consumer goals can be unrealistic and there is a perception that consumers could be more actively assisted to achieve goals Meaningful Activities: there is a pathway gap for consumers to be: o provided with real choices and opportunities o engage in relevant interests and activities Community Connection: that is supporting consumers to build or rebuild positive relationships with family members or friendships with people outside the mental health system. To a limited degree the regional allocation committees provide a forum for addressing and reviewing pathway gaps, however this requires further work. It is recommended that: A process is more clearly identified within the operational protocols for collecting and reviewing information pertaining to consumers long term outcomes after leaving the IPRSS program. This should be a partnership response that encourages services to review practices and achievement of objectives within the context of the program type. 7. There is a need for improved co-ordination of reviews and better articulation of roles and responsibilities in operational protocols. Partnership success could be improved by greater clarity of vision, roles and responsibilities, clearly articulated boundaries, concepts of recovery and recovery practice. The program has been operating for two years and it is timely for the leadership to recommit to the shared vision required to sustain the success of the program. It is recommended that: 7

15 The PMC recommit to the shared vision for IPRSS, review and update the operational protocols and conduct joint training sessions on the recovery approach and further definition around the role of the NGO support worker, supplemented by action learning case studies. This would assist in developing a shared understanding of, and commitment to respective roles and responsibilities. This will also assist in maintaining quality governance arrangements including high functioning allocation committees. 8. There is a strong demonstrated commitment to quality in the program with providers having more than one accreditation process which is time consuming and costly. It is recommended that: Consideration be given to endorsing an approach where only one recognised quality process need be adopted for NGO providers. 8

16 1 INTRODUCTION The South Australian Department of Health (DH) provides funding to non-government organisations (NGO providers) through a competitive tender process to provide individual psycho-social rehabilitation services working in partnership with government mental health services (MHS). The DH allocated funds through a contract arrangement to nine non-government organisations over the period 1 April 2009 to 30 June Health Outcomes International (HOI) was engaged to undertake the evaluation of the Individual Psychosocial; Rehabilitation and Support Services (IPRSS). The evaluation aims to build a strong evidence base for the provision of best practice psychosocial rehabilitation and support services in South Australia This chapter sets out the background to the IPRSS program, the program objectives, a brief description of the program, the purpose and objectives of this evaluation, an overview of the evaluation methodology, and an outline of the structure of this report. 1.1 BACKGROUND TO IPRSS The non-government sector is an integral component of an effective mental health system recognised for its distinctive contribution in many jurisdictions both nationally and internationally. Within South Australia there is an increasing emphasis on funding to the non-government sector for the provision of community mental health services. NGO providers are now providing a wider range of services to a larger number of mental health consumers than ever before. Many of these consumers have complex needs. The South Australian Government accepted the recommendations of the Social Inclusion Board s report Stepping Up: A Social Inclusion Action Plan for Mental Health Reform With regard to non-government sector the Board recommended that: South Australia should continue to build capacity in the non-government sector to deliver psychosocial rehabilitation and support services. The development should be framed within a partnership approach that builds on a system that will have community mental health at its centre. Rehabilitation and support services should be focussed on helping people to step down from formal care and on maintaining ordinary associations in society that support a meaningful life. Within South Australia, non-government community mental health services funded by the Department of Health would include Individual Psychosocial Rehabilitation Support Services BRIEF DESCRIPTION OF PROGRAM The IPRSS program is based on an evidence based model of care and supports people with severe mental illness and a high level psychiatric disability on their recovery journey. A key element of the IPRSS program is the partnership between NGO providers and government MHS with the needs of the shared consumers being central to decision making. A recovery philosophy underpins the delivery of Statewide Operation Protocol. Individual Psychosocial Rehabilitation and Support Services. Government of South Australia 9

17 psychosocial rehabilitation and support services. This philosophy is internationally and nationally recognised, having been developed by the mental health consumer movement. Services can operate over seven days (although they are usually provided on a five day basis) in accordance with the person s needs and are delivered regardless of where the person lives in the community, for example in private or public housing, a boarding house, Supported Residential Facility (SRF), an Aboriginal hostel, private hotel or caravan park. IPRSS NGO providers and government MHS work in partnership with other key stakeholders including housing to provide structured, goal focused and individually tailored services at a level of intensity and duration appropriate to the consumer s need. An important component of individual psychosocial rehabilitation is community capacity building. Services are generally targeted at adults (over the age of 18 years) and older people who have high and complex needs and are registered with the public mental health system. The program is underpinned by a service model, operational protocols and a formal governance structure, which includes local allocation processes and a state-wide Program Management Committee that provides operational leadership and broad oversight of the program. IPRSS NGO providers are required to work towards compliance with the formally endorsed South Australian Psychosocial Rehabilitation Support Services Standards (PRSSS) and minimum training requirements for the employment of staff. Service types include: services delivered to assist the consumer engage in meaningful daytime activity and employment services delivering combined housing and support programs provided the service is not facility based services delivered in community settings intended to promote community engagement and social connectedness independent living skills support and training to enable day to day living in the community transition from facility based services to home and community living. 1.3 IPRSS PROGRAM OBJECTIVES The Program objectives are outlined in the IPRSS service model and include to: assist people to self manage their own recovery and build on their interests, aspirations and strengths to live full and active lives develop skills to improve competence and confidence in community living improve health and well-being improve independence and resilience prevent relapse and limit severity of any crisis engage the consumer with desired community and social activities reduce social and physical dislocation by assisting people to sustain suitable housing and to develop improved social relationships increase opportunities to participate in the workforce reduce demand on acute and emergency services. Other Program objectives include: 10

18 to provide individually tailored services at a level of intensity and duration appropriate to the consumer s needs to provide services within a recovery oriented framework to enhance partnerships between government and non-government community mental health services, consumers and carers to enhance partnerships with the community to build capacity. 1.4 EVALUATION OBJECTIVES The evaluation aims to build a strong evidence base for the provision of best practice psychosocial rehabilitation and support services in South Australia and will specifically seek to: assess whether the IPRSS program was implemented as planned determine whether consumer outcomes were optimised improve IPRSS program arrangements and performance inform future service planning, delivery and funding make recommendations regarding specific aspects of the service that require review or development. More specifically the evaluation is required to undertake an assessment of the extent to which: 1. The services are achieving the aims and objectives of the program. 2. Consumer outcomes are being achieved and service delivery reflects a recovery orientation. 3. Each stage of the service model and operational protocols are implemented effectively with a particular focus on the consumer journey from engagement to exit. 4. Quality services are provided. 5. The services are accessible to the target population and in particular specific populations such as Aboriginal people and people from culturally and linguistically diverse backgrounds. 6. Partnerships between government and non-government community mental health services, consumers and carers optimise outcomes. 7. The program enhances partnerships with the community to build capacity and optimise outcomes. 8. The program s governance arrangements and structures are effective and efficient enhancing desired outcomes. 9. The program is reaching the right people through the allocation process. 1.5 EVALUATION DESIGN AND METHODOLOGY The evaluation was conducted in four broad stages namely: 1. Developing a project plan and an evaluation framework - which identified the service program logic, data set and appropriate information gathering tools that would be required to support both process and impact evaluation. Appropriate ethics approval to undertake the evaluation was obtained from the SA Department of Health Human Research Ethics Committee as part of this stage. 2. Conducting the consultation process - with NGO service provider, government community mental health services, the SA Mental Health Carer Support Advisory Group and consumers via the Ridgeway Recovery Environment Enhancement (REE) tool survey. 11

19 3. Quantitative consumer outcome data analysis collection and analysis of available consumer outcome data. 4. Reporting - interim and final reports. A number of qualitative and quantitative processes have been utilised to evaluate the program. These include: an analysis of Consumer Activity Reporting System (CARS) program data in respect to consumer demography, access to the program, and program activity an analysis of government community mental health outcome data and inpatient admission data to review potential impact of the program an analysis of consumer perspectives obtained from the REE tool survey administration of the partnership analysis survey tool developed by VicHealth designed to test various aspects of the partnership attendance at allocation committee members in all metropolitan areas and two country regions (Murray Bridge and Port Pirie) by attending meetings as an observer and having group discussions about the program where time permitted (i.e. Eastern, Murray Bridge And Port Pirie) analysis of feedback from government MHS, NGO provider and carer stakeholder consultations. 1.6 REPORT STRUCTURE This report is structured as follows: Table 1.1: Report structure Chapter two Chapter three Chapter four Chapter five Chapter six Chapter seven Presents an evaluation of key aspects of the service model. Examines stakeholder perspectives on the partnership. Provides an analysis of governance arrangements for the IPRSS program. Analyses the impact of the program on consumer outcomes. Reviews consumer perspectives on the program. Outlines the overall conclusion to the evaluation. 12

20 2 THE CONSUMER JOURNEY Chapter two presents an evaluation of key aspects of the service model from the perspective of the consumer journey. 2.1 KEY ELEMENTS OF SERVICE MODEL The IPRSS program service model as promulgated by is presented in appendix A. The key elements to the service model include: adopting a recovery orientation approach the service partnership between NGO providers and government MHS governance arrangements a regional allocation meeting convened by the government MHS and attended by NGO providers to assess and allocate potential consumers developing individual support plans (ISPs) where goals are set with consumers involving the consumer, carer where appropriate, NGO providers and the government MHS regular three monthly ISP reviews with the consumer, the NGO provider, the government mental health worker and the carer where appropriate exiting consumers. A high level overview of the service model is presented in figure 2.1. Figure 2.1: IPRSS program service model overview The program logic developed as part of the evaluation framework is presented in appendix B. The service partnership, governance and consumer outcomes are considered in chapters three, four and five respectively. 13

21 2.2 PROGRAM ACTIVITY Figure 2.2 presents the number of consumers by provider who received an IPRSS service for the period April 2009 and August 2010 (the period for which data was made available). A total of 936 consumers received an IPRSS service. Figure 2.2: Number of consumers by provider April 09 to August 2010 Note (1): Each consumer is counted once. Provider Figure 2.3 presents the total consumer service hours recorded on the Consumer Activity Reporting System (CARS) by provider, for the period April 2009 to August The total hours recorded were 169,363 compared to 172,380 funded hours, a variation of 1.8% which we consider to be minor in the context of the evaluation. Figure 2.3: Total consumer hours by each provider April 2009 to August 2010 Provider Note (1): Hours include direct and indirect client hours. Figure 2.4 presents the regional distribution of all IPRSS program consumers. The figure shows that that 51% of consumers were serviced by NGO providers operating in the central and northern regions, which correlates with the 52% of funds being allocated to NGO providers in those regions. 14

22 Figure 2.4: Regional distribution of consumers 2.3 CONSUMER CHARACTERISTICS Figure 2.5 presents the age profile of all IPRSS program consumers. The majority of consumers fall within the 20 to 49 year age bracket (73%). There were 21 consumers less than 20 years of age which reflects the fact that the program is not focused on younger consumers. Figure 2.5: Age profile 15

23 Figure 2.6 presents the gender distribution of consumers on the program and table 2.1 presents the distribution by provider. For adult services males comprise the greater share of consumers (59%) and for older people services females comprise 74% of consumers, the higher percentage being attributed to higher male mortality rates. Figure 2.6: Overall gender distribution Table 2.1: Gender by provider Provider Gender Male Female Total Adult Providers (n=525) 59.0% 41.0% 100.0% Older Person Providers (n=78) 25.6% 74.4% 100.0% Overall (n=603) 54.7% 45.3% 100.0% The following is a general description of IPRSS consumers (excluding older people) as articulated by stakeholders. Consumers: have high and complex needs have been engaged with a government MHS over a long period of time are often at risk of becoming homeless are evenly distributed between male and female are at risk of relapse if they do not receive support more often than not they are living alone - in some cases aged parents have just moved into nursing homes or passed away and they often have no or limited skills in how to maintain their house. Table 2.2 compares IPRSS program consumer complexity as measured by the National Outcomes and Casemix Collection (NOCC) scores to the national average. The table shows that the complexity of IPRSS consumers is significantly greater than the average national ambulatory mental health consumer for both adults and older people. Table 2.2: IPRSS consumer complexity Assessment Type National Mean 1 IPRSS Mean Score Before First Contact Percentage Difference HoNOS Adult % HoNOS % K10 Adult % 16

24 K10 Older Person % LSP16 Adult % LSP16 Older Person % Note (1): Source NOCC web decision support tool Ambulatory Occasion review 2008/ NEEDS IDENTIFICATION, REFERRAL AND ALLOCATION Figure 2.7 presents the number of referrals to the IPRSS program per month. The figure shows that once established, new referrals to the program each month are relatively stable, although there was a large increase to 44 referrals in March Figure 2.7: Referrals per month NEED AND APPLICATION FOR RESOURCES The program guidelines require that the potential need for an IPRSS service is identified by the government MHS worker and confirmed by a government MHS senior clinician. An application for resources is completed and submitted to the relevant allocation committee. The government MHS worker involves the consumer in the application and gains consumer consent to proceed. 17

25 REFERRAL AND ALL OC ATION The regional allocation committee comprised of relevant NGO providers and government MHS considers the application and if approved, a referral is made. Regional allocation committee meetings are resource intensive given the number and time of people attending. Based on our observation of the regional allocation committees, application for resources are completed and presented for potentially relevant consumers. Ideally the relevant government MHS worker should present the IPRSS referral as this provides for a deeper and more meaningful discussion about the consumer than if the referral is presented by a third party although it is understood that this is not always practical. PRIORITY SETTING Different committees have different approaches to priority setting e.g. the west uses a first on the waiting list, first in the program approach whereas southern categorises consumers into one of three priority categories as well as taking time on the waiting list into account. ACCEPTANCE Once a consumer is referred and accepted onto the IPRSS waiting list, it can be some months before they actually commence on the program. It is not clear how changed consumer circumstances are taken into account when a vacancy in the program arises. At each meeting, NGO providers give a verbal update about their capacity to take new entries and whatever is stated is generally accepted. Some government MHS expressed concern that very little information in relation to consumer hours was provided to allocation meetings, and that this resulted in a lack of knowledge of who is receiving what level of service, and what may be their priority compared to another consumer on the waiting list. OTHER PROGRAMS Some allocation committee meetings cover a number of different programs (e.g. CSS/PHaMs/housing) and this helps link the consumer to the most appropriate service and ensures adequate support is provided. This appears to work well. A more strategic discussion of the role of the allocation committee is in section

26 Key Findings Allocation committee meetings and associated processes help to ensure the program is reaching the right consumers. Allocation committee meetings are resource intensive but provide a valuable role based on the current program structure. They are a key element of the service partnership. There is a need to establish an agreed priority ranking process for entry into the program state-wide as currently each region has a different approach. Allocation committees that cover multiple programs facilitate a streamlined service approach for the consumer. There can be significant waiting lists and it is not clear how consumers who have been on the list for some time are reprioritised when a vacancy arises. The provision of an NGO provider regional monthly report to allocation committee meetings, incorporating current consumers and hours allocated per consumer, would further strengthen the service partnership and help to prioritise need. 2.5 INDIVIDUAL SUPPORT PLANS AND REVIEWS DEVELOPMENT OF ISPS Individual Support Plans (ISPs) are generally developed with each consumer on entry into the program (or within the first few weeks) and then reviewed every three months, and in some cases updated as required. Input from government mental health workers occurs on a regular basis, although they are not able to get to all meetings. Communication is maintained as, in that situation, meetings take place without the key worker and relevant information is subsequently forwarded to them. The ISP is developed in collaboration with the consumers (and carers or significant others where agreed by the consumer although in many cases carers are not involved). Each area of the ISP is discussed with the consumer, with most service providers utilising the Camberwell Assessment of Need as a prompt if needed. The focus is on identifying consumers strengths and setting goals to improve their lives. It was identified as important for there to be a good understanding of where a consumer was at on their journey, as was having male, female and ethnic workers to suit individual consumer preferences. ISPs are signed by the consumer and both the consumer and government MHS receive a copy. 19

27 One suggestion was the concept of having one service/care plan across government MHS and IPRSS NGO providers. This would help to improve communication and provide clarity for providers and consumers alike. Currently there may be some duplication and the two care plans can be seen in contrast with each other and may cause conflict between the teams. Government MHS generally thought an electronically shared care plan was a good concept that they would support, however there were some concerns around privacy and technical logistical issues. The IPRSS operational protocols identify the need to develop with the consumer a Mental Health Care Plan (primary function of the government MHS) and the ISP (primary role of the NGO provider). It also articulates the process for developing these plans and their relationship to each other. In discussion with the both government and non-government providers it would seem this is not always well understood and/or there are barriers to implementing a shared plan utilising the current operational protocols. One factor impacting on the timeliness of ISP completion is the time from referral to first contact. This is also a proxy measure of capacity and efficiency. Table 2.3 presents the average days from referral to first contact by provider and by region. The average time from referral to first conact was 16.9 days, the shortest was 0.4 days and the longest 45.6 days. This represents a significant variartion which needs further exploration. There are also variations between providers and other data held by the DH that may account for some difference, however there needs to be greater clarity around recording and definitions. There are no benchmarks established for the program in this area. When setting benchmarks, the time from the need being identifed to the time support begins to be provided is the key. Any benchmark or indicator established would need to take into account the steps and responsibilities across this pathway. Organisation Table 2.3: Average days from referral to first contact Central Northern Adelaide (n=181) Country (n=162) Southern Adelaide (n=102) Overall (n=445) Provider 1 NA 2 NA Provider NA NA 10.1 Provider Provider Provider NA 37.7 Provider NA Provider NA 13.7 Overall (n=445) Note (1): Data sourced from CARS Note (2): NA = not applicable ie there are no consumers from the given provider in the given region The ISP and subsequent reviews are the key drivers of determining the intensity and duration of the service provision to a consumer. Figures 2.8 and table 2.4 present the average hours of service per consumer per month both by provider. The average hours of service per consumer per month is 18.7 hours. The pattern of hours per month was relatively consistent for the April 2009 to August 2010 period. There are significant variations between providers. While it is reasonable that there are variations between consumers, the level of variation demonstrated by figure 2.8 in our view should be a priority for further investigation. 20