Drug Treatment Funding Program Client Recovery Monitoring Project

Transcription

1 Drug Treatment Funding Program Client Recovery Monitoring Project Final Report for the Ministry of Health and Long Term Care Brian Rush Nooshin Khobzi Rotondi Nancy Chau April Furlong Alexandra Godinho Christina Schell Christine Kwong Safa Ehtesham Health Systems and Health Equity Research Centre for Addiction and Mental Health Toronto, ON August

2 TABLE OF CONTENTS 1.0 BACKGROUND AND RATIONALE THE DRUG TREATMENT FUNDING PROGRAM (DTFP) PROJECT OBJECTIVES METHODS OVERVIEW OF STUDY DESIGN STAKEHOLDER CONSULTATIONS ENVIRONMENTAL SCAN SELECTION OF TOOLS SITE VISIT TO CHESTNUT HEALTH SYSTEMS GAIN COORDINATING CENTRE Literature Review and Tool Selection Criteria MEASURES Stage 1 Assessment and Follow Up Tools: Global Appraisal of Individual Needs Quick 3 (GAIN Q3) Motivational Interviewing (MI) and Standard Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) Return to Treatment (RTT) Protocol PILOT SITES Rideauwood Addiction and Family Service Four Counties Addiction Services Team (Fourcast) Addictions Centre (Hastings/Prince Edward Counties) Manitoulin Community Withdrawal Management Services STUDY PARTICIPANTS QUANTITATIVE DATA COLLECTION PROCEDURES Recruitment and Consent Process Baseline Assessment and Within Treatment Outcome Monitoring Three and Six Month Interviews PILOT SITE ENGAGEMENT, TRAINING AND MONITORING OF PROCEDURES ENGAGEMENT OF PILOT SITES

3 3.2 TRAINING OF PILOT AGENCY STAFF MONITORING OF PROCEDURES AND DATA QUALITY Monitoring Visits and Support to Pilot Sites Data storage and Central Database Development LAYING THE GROUNDWORK FOR IMPROVEMENTS IN MEASURING CLIENT OUTCOMES AND PERFORMANCE INDICATORS IN THE SUBSTANCE USE TREATMENT SYSTEM DEVELOPING AN ONTARIO VERSION OF THE GAIN Q GAIN Revisions and Assessment Building System (ABS) GAIN Training DEVELOPING A RECOVERY MONITORING SYSTEM Overview of the Follow Up Protocol Changes to the protocol EVALUATION STRATEGY QUALITATIVE DATA ANALYSIS QUANTITATIVE DATA ANALYSIS AND DESCRIPTION OF STUDY SAMPLE FINDINGS AND INTERPRETATIONS FEASIBILITY AND IMPLEMENTATION ISSUES: QUANTITATIVE RESULTS Representativeness of sample Administration of Tools Follow Up Rates Qualitative Feedback LESSONS LEARNED AND CHALLENGES RECOMMENDATIONS, POTENTIAL IMPACTS AND NEXT STEPS RECOMMENDATIONS POTENTIAL FOR IMPROVEMENTS IN MEASURING CLIENT OUTCOMES AND PERFORMANCE INDICATORS IN THE SUBSTANCE USE TREATMENT SYSTEM NEXT STEPS AND IMPLICATIONS

4 LIST OF TABLES TABLES PAGE 1 Summary of data collection activities 31 2 Summary of training attendance for each pilot site 35 3 Evaluation questions and indicators of success 49 4 Comparison of Study Sample to Treatment Population in Ontario 56 5 Completion Times for Recovery Monitoring Tools 60 6 Characteristics of clients (from London and Peterborough) who 67 completed both 3 month and 6 month interviews VS those lost to follow up (among clients with GAIN Q3 MI) 4

5 LIST OF FIGURES FIGURES PAGE 1 Framework for the Stages of Client Engagement across 16 Screening, Assessment and Recovery Monitoring 2 An Overview of the Recruitment Process 53 3A Follow up data at 3 months 65 3B Follow up data at 6 months 66 5

6 LIST OF APPENDICES APPENDIX PAGE 1 Glossary and List of Acronyms 82 2 Recovery Monitoring Project Tools 85 6

7 Executive Summary In 2008 Health Canada announced the Drug Treatment Funding Program (DTFP), a key element of the National Anti Drug Strategy. The focus of the DTFP was on enhancing the systems of services for people with substance use problems in Canada, emphasizing three broad target areas for investment: implementation of evidence based practices; strengthening evaluation and performance measurement; and knowledge exchange. Each province and territory was invited to submit proposals for system enhancement. The Ontario submission included the present project, the objectives being to assess the feasibility of implementing a comprehensive recovery monitoring system for addiction services based on a hybrid approach which included a withintreatment and a post intake component. The project built upon other work undertaken for the MOHLTC in the past few years on screening and assessment tools and processes in Ontario and outcome monitoring models and measures. Specifically, the project examined the feasibility of implementing a dedicated recovery monitoring service and the appropriateness and usefulness of standardized recovery monitoring tools for measuring change over time. A broad range of stakeholders were engaged in the project via a Program Advisory Committee and Working Group. Partners included the Ministry of Health and Long Term Care, specialized mental health and substance use agencies (directors, managers, clinicians and researchers), Local Health Integration Networks (LHINs), and consumer representatives. This group, and a special sub group specific to this project, informed all stages of the decision making related to the selection of the tools and pilot sites, procedures for implementation, results interpretation, and wider dissemination of project findings. The selection of outcome tools involved several steps, the first being a comprehensive literature review that evaluated recovery monitoring models and tools used in mental health and/or addiction settings. The project team also conducted an environmental scan in Ontario to determine the scope of use of any kind of standardized tools to assess client outcomes in publicly funded addiction agencies, 7

8 common outcomes measured and what measurement tools and processes were in place. As a result of this literature review and environmental scan, a list of potential tools and the criteria for selecting the tools for pilot testing for a hybrid recovery monitoring system were summarized and presented to the Advisory Committee and Working Group for a final decision. The post intake recovery monitoring tool endorsed by the Advisory Committee and Working Group for this project was the Global Appraisal of Individual Needs Quick 3 (GAIN Q3) Standard. The GAIN Q3 is a multi purpose targeted assessment tool used to identify and address a wide range of life problems among both adolescents (age of yrs) and adults (age of 18 yrs+) in diverse treatment settings. It is a valid and reliable tool and is sensitive in detecting change in the many life areas assessed at baseline by its sister tool, the GAIN Q3 MI. The Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) was developed specifically for this project, in collaboration with project stakeholders, to monitor clients general well being during their treatment process. The measure consists of nine items; six were adopted from the Life Satisfaction Scale in the GAIN Q3 MI, which assesses clients general satisfaction with life. The remaining three items concern self efficacy, perceptions of the relevance of treatment to clients needs, and motivation for change and were developed by the project team and project stakeholders. Five publicly funded Ontario based addiction agencies of various geographic locations, sizes and service types participated in this project and piloted the use of GAIN Q3 MI (baseline assessment tool) and the OWTOM A within treatment outcome measure. Agency staff and clinicians were involved in data collection after obtaining appropriate training related to the administration of the study tools, including interpretation of the GAIN Q3 and the data collection procedures. The project followed up participating clients for up to 6 months after their intake assessment. During these 6 months, the clients were asked to complete the OWTOM A on a monthly basis with their treating clinicians as long as they were in treatment. They 8

9 were also to complete the GAIN Q3 in a telephone interview with the follow up team to assess change in clients life problems since intake. Through this follow up process, the project facilitated timely sharing of the recovery monitoring information between the follow up team and agency clinicians and encouraged discussion of assessment results with clients. The project also implemented the Return to Treatment protocol through which it actively provided support for linking clients to treatment either through their recruiting agencies or ConnexOntario, if a need or desire was indicated during the follow up process. The feasibility of the recovery monitoring system that was piloted was evaluated using both quantitative and qualitative methods. The project conducted statistical analyses on a list of technical indicators and collected feedback on the strength and limitation of the recovery monitoring protocol and the tools from participating clinicians and the follow up team. Clients feedback was reported indirectly through clinicians and follow up team members. Data collection occurred between June 2012 and March In total, 234 clients were enrolled into the Recovery Monitoring Project and 218 of them agreed to the recovery monitoring process and provided the follow up team with their locating information. The focus of the analyses was on a combined sample of two pilot sites that made up about 90% of our total study sample. Of the 200 clients included in the analyses, 148 completed a baseline interview and could potentially provide change data for up to 6 months. The follow up team was able to reach 99 (67%) of these 148 clients at 3 months after intake and 89 (69%) of the 129 clients who were due for a 6 month follow up. It is important to note that this project did not provide any monetary incentives for participants. Results of analyses showed the OWTOM A to be a quick and easy tool for monitoring client progress over time and while still in treatment. The participating clinicians found that the tool facilitated their clinical process and was helpful in guiding treatment sessions with clients. Yet some clinicians noted that change between 9

10 administrations were hard to detect and others reported not feeling comfortable to be candid about their therapeutic relationship. The collected feedback on the GAIN Q3 showed that the tool provided opportunity for clients to discuss their progress and recognize changes in their behaviour. The clients, however, perceived some items as being repetitive. This was compounded by issues with telephone administration such as cost and privacy concerns with household lines. The Return to Treatment protocol was shown to facilitate re engagement of clients with agencies and provided clients with a sense of ongoing support. Clients also expressed appreciation of efforts to keep them connected to services. Challenges raised during implementation of the protocol included difficulties in confirming an appointment between clients and agencies to take full advantage of the opportunity of re engagement. In addition, little interest was shown among clients in being connected to ConnexOntario. The follow up protocol implemented by the follow up team was adapted from the one developed by Dr. Christy Scott, who specializes in recovery monitoring. Overall, the follow up protocol was well received by clients. Clients indicated that the follow up process allowed them to feel connected and to extend their support network and that they expressed appreciation for the attention the follow up team had shown them about their well being. On the other hand, some clients perceived the follow up process as intrusive, although it was recognized as being necessary to obtaining useful outcome data. In conclusion, this pilot project has demonstrated that recovery monitoring in Ontario s addiction services is feasible. Overall, agency staff, clients and the follow up team found great value in the recovery monitoring process. It was useful from a clinical perspective, provided more opportunities for building rapport with clients, and allowed for re engagement with the treatment system. Nonetheless, there were challenges raised in the piloting process as outlined earlier. The team were also concerned that the follow up procedures did not successfully engage less stable, more severe and 10

11 potentially more marginalized clients. Given these study findings and the resourceintensive nature of this work, the project team feels that the follow up protocol and related resource requirements is not ready to be implemented province wide. The project team recommends more piloting so that procedures and additional components of the follow up protocol that were not incorporated during the initial phase of piloting may be refined. These procedures and test components include use of paid community trackers, different strategies for follow up interviews that decrease costs for clients, and other approaches to reconnect clients with services. Future studies also need to involve more diverse clients with more severe substance use problems, and those receiving residential services. Although the piloted recovery monitoring system is not ready for a provincial dissemination, the project team encourages its application at the individual agency and LHIN levels if adequate resources and administrative support can be provided. The results from these more local implementation efforts would then serve to support plans for broader implementation in other parts of the province. 11

12 1.0 BACKGROUND AND RATIONALE At the client level, outcomes of addiction treatment are the changes in their symptoms, behaviour and function that can be attributed to their participation in treatment (McLellan, Chalk, & Bartlett, 2007). Quantifying clinical and functional outcomes during a specified period of time is the main component of outcome monitoring. An effective system of outcome monitoring can provide accountability information to funders and administrators; contribute to addiction service research as a complement to more controlled clinical research studies; and complement information on service utilization and cost, client satisfaction, and treatment process. It can also facilitate clinicians work with their clients, for example, by providing quick feedback on clients progress so as to adjust the treatment plan. In 2008 the Ministry of Health and Long Term Care (MoHLTC) commissioned a project that built upon a previous evaluation of the provincial Admission and Discharge Criteria and Assessment Tools (ADAT) 1, to review models of outcome monitoring for addiction treatment, with a view to eventually developing a feasible outcome monitoring system across Ontario s addiction agencies (Rush, Martin & Corea, 2008). The earlier MoHLTC funded project on outcome monitoring closely engaged representatives of Ontario s addiction treatment network. Subsequent recommendations called for a client model of outcome monitoring whereby the baseline assessment would be conducted by agency staff, one or more brief measures of outcome would be obtained from the client periodically during the course of treatment, and an independent, third party would follow up clients at quarterly intervals thereafter (Rush, Martin, Corea & Rotondi, 2012). The within treatment monitoring procedure developed by McLellan and colleagues (2005) was an important component of the proposed model. Within treatment measures record client progress on selected domains of functioning such as substance use, drug injection behaviour, criminal activity, 1 ADAT is comprised of a common set of assessment instruments (8) and a decision tree for assigning the client to the appropriate level of care, for application across all Ministry funded addiction services. 12

13 and health and social functioning on broader treatment goals such as life satisfaction and self efficacy. These measures of client progress are intended to assist clinicians in planning treatment, providing feedback to clients about their treatment progress, and systematically record client progress over time. When combined with follow up data, these within treatment measures provide a comprehensive description of client outcomes. In a related model known as Recovery Monitoring Check ups (RMC) developed by Dennis and colleagues (Dennis, Scott, & Funk, 2003; Scott, Dennis, & Foss, 2005;) another unique feature was introduced into the follow up component, whereby a defined protocol is used at follow up to determine if the client had experienced relapse, or was at significant risk of relapse, and is used to recommend the clients return to treatment. In short, the recovery monitoring model that was recommended at the conclusion of the MoHLTC funded project was referred to as a hybrid model (Rush, Martin, Corea & Rotondi, 2012) and builds upon the literature on recovery monitoring that had been developing over the past years. This model has moved away from the traditional pre post follow up strategy by integrating outcome determination directly into the treatment and support process rather than viewing it as a costly research add on (McLellan, McKay, Forman, Cacciola & Kemp, 2005). The current project was based on this key conceptual shift; i.e., the evolution from outcome monitoring to recovery monitoring. The latter is the preferred terminology as it better reflects the chronicity of substance use problems for the most severe clients; provides a focus on the continuous monitoring of treatment outcomes; and places an emphasis on Return to Treatment for those clients in need. Another important contextual factor for the present project is the considerable research and development over the past decade in the area of co occurring mental and substance use disorders and, in particular, work related to the more effective and earlier identification of individuals with co occurring disorders in mental health, addictions and 13

14 other types of settings. The seminal Canadian report on concurrent disorders (Health Canada, 2001), a more recent update (Canadian Centre on Substance Abuse, 2009), and other international reviews have advocated for universal screening of mental disorders in addiction services and for substance use disorders in mental health services. As a response to this, system planners and managers of mental health and addiction programs in Ontario have been very active in selecting and implementing screening tools into their services and broader community treatment systems (Centre for Addiction and Mental Health, 2008). The GAIN Short Screener (GAIN SS), in particular, has been well received and widely implemented in Ontario due to its utility in mental health as well as addiction and other settings; its brevity, low cost and utility as an outcome measure, and the supporting validation data that extends to age Ontario researchers also undertook a major validation study of various screening tools in Ontario addiction treatment settings with positive results obtained for the GAIN SS (Rush, Castel, Brands, Toneatto, & Veldhuizen, 2013). A major research synthesis was also undertaken of screening tools for children and adolescents and the GAIN SS emerged as one of the best supported options for this younger population (Rush, Castel, Somers, Duncan & Brown, 2009). An integrated view of screening, assessment and recovery monitoring is articulated by Rush and Castel (2011) and represented in Figure 1 below. In sum, the work on screening for concurrent disorders has evolved to the point where a review of Ontario s assessment tools and processes, and consideration of models and measures for recovery monitoring, must include screening within its scope. In this screening and assessment framework the process is divided into the following stages: Stage 1 and Stage 2 Screening, and Stage 1 and Stage 2 Assessment (Rush & Castel, 2011). The staged approach works to ensure a progressive and efficient use of screening and assessment resources to guide treatment planning and eventually, recovery monitoring. The main benefit of using a stepped approach in screening is 2 The original validation data on the GAIN-SS extended to the lower age limit of 10. However, experience in the national DTFP project led by Gloria Chaim and Joanna Henderson that involved a network of youth serving agencies suggested the tool is better used for youth 12 years of age and up. 14

15 through sequential case finding, as both the clinician and client time can be saved by reserving the more resource intensive screening and assessment tools for those who score above the cut off on briefer, more economical screening tools. Recovery monitoring is also greatly facilitated by choosing screening as well as assessment tools that can reliably and validly assess change over time. Figure 1. Framework for the Stages of Client Engagement across Screening, Assessment and Recovery Monitoring 1 adapted from Rush and Castel (2011) The recent developments in the addiction field that consider recovery monitoring as a continuous process occurring both during treatment as well as after 15

16 treatment are major steps forward in conceptualizing and operationalizing the links between screening, assessment and recovery monitoring. The flow and linkage across the stages of client engagement is the framework on which this project was based. 1.1 THE DRUG TREATMENT FUNDING PROGRAM (DTFP) In 2007, the Canadian federal government introduced the National Anti Drug Strategy to focus efforts on reducing the demand for, and supply of, illicit drugs; as well as to address the crime associated with illegal drugs. The National Anti Drug Strategy emphasized the need for effective treatment services to better manage the illicit drug problem when it occurs, and to help those in need. The need to make improvements at the level of the treatment system emerged as a priority for action, based on countrywide consultations and national level studies on treatment needs and gaps carried out in support of the development of the National Anti Drug Strategy. These studies demonstrated the need for systemic change to move treatment systems toward more evidence informed practices, while also increasing systems capacity to evaluate practices for their efficiency and effectiveness (Health Canada, 2008). In response to these findings, the Drug Treatment Funding Program (DTFP) 3 was announced in 2008, providing new, five year, time limited funding ( ) 4 to assist provincial and territorial governments in addressing critical treatment needs in three investment areas (Health Canada, 2008): 1) Implementation of evidence informed practice Although evidence informed practices to optimize treatment exist, and are continually being developed, many do not easily find their way into service delivery. Evidence informed practice is based on interventions that effectively integrate the best research evidence with clinical expertise, cultural competence and the values of the persons receiving 3 Re oriented funding from Health Canada s Alcohol and Drug Treatment and Rehabilitation Program, effective April Timelines varied across provinces and territories depending on the release of funds; Ontario s projects received funding for 19 months of work (April 2011 March, 2013). 16

17 the services. These interventions have evidence showing improved outcomes for families, clients, and/or communities. 2) Strengthening evaluation and performance measurement Performance measurement and evaluation activities across jurisdictions are limited. While all jurisdictions collect, manage and analyze performance information pertaining to their treatment services and programs, the type and nature of data collected, as well as the approach to data collection and analysis varies considerably. This strategic investment area strengthens evaluation and performance measurement capacity and activities. 3) Knowledge exchange This investment area is an essential element of work undertaken in investment areas #1 and #2. Knowledge exchange activities can include such things as mentoring and the provision of technical expertise; mechanisms that develop and/or enhance knowledge sharing and dissemination of lessons learned from communities of practice in the uptake of best practices and in performance measurement/evaluation; and activities that can effectively reach health professionals within organizations and service sectors to increase their awareness and participation in knowledge exchange activities across the continuum of treatment services. Each province and territory was invited to submit proposals for enhancement of their substance use treatment system. One component in the Ontario submission is the focus of this report, namely testing the feasibility of a provincial recovery monitoring system. As a key component of investment area #2, this project lays the groundwork for the systematic collection, management and analysis of performance indicators related to client outcomes. Another related project that is the subject of a separate report is the review of Ontario s screening and assessment tools and processes within substance use treatment services; with a view to replacing the current package of client assessment tools with a more up to date suite of evidence informed tools for treatment planning and that are better linked to recovery monitoring. 17

18 It is important to highlight the significant delay that occurred between the broader Ontario proposal development and actual funding. Funding was approved in April of 2011 and the DTFP national funding ended in March 2013; thus Ontario effectively lost over two years of project time. This had significant implications for the project on screening and assessment tools as well as the recovery monitoring project. Specifically, the original proposal for the recovery monitoring project called for four quarterly check ups over a one year period. However, the project was implemented with only two quarterly check ups over six months. 1.2 PROJECT OBJECTIVES The recovery monitoring model and the specific outcome measures were pilot tested in the same five addiction agencies as the piloting of the screening and assessment tools, since the system that is envisaged in the future is an integrated screening, assessment, within treatment and follow up recovery monitoring system. It is important to understand that the project engaging these five pilot sites was not about the evaluation of the treatments that they offered, but rather the feasibility assessment of a provincial recovery monitoring system. These goals are reflected in the following statement of objectives: 1. To examine the feasibility of implementing common within treatment outcome measures and the potential usefulness of the results among decision makers at the clinical, program and system planning levels. 2. To assess the feasibility of a comprehensive system of client recovery monitoring and the potential use of the results among decision makers at the clinical, program and system planning levels. 18

19 2.0 METHODS 2.1 OVERVIEW OF STUDY DESIGN This feasibility study was conducted from June 2012 to March 2013 and involved the piloting of evidence informed tools and processes related to assessment and recovery monitoring in Ontario s addictions treatment services. Ethics approval was obtained from the Centre for Addiction and Mental Health and Ottawa Carleton Research Advisory Committee. As noted earlier, the model implemented in this project was based on the hybrid approach recommended in a MoHLTC funded project by Rush and colleagues (Rush, Martin, Corea & Rotondi, 2012). Measures were used to: 1) record client progress on various domains of functioning on a monthly basis while the client was receiving treatment; and 2) compare outcome data collected quarterly for up to six months 5 with baseline data from the intake assessment. Clients seeking help for their substance use problems 6 from five addiction treatment agencies across Ontario were approached for participation in the project. Following consent, the recruited clients completed the baseline assessment tool: Global Appraisal of Individual Needs Quick 3 Motivational Interviewing (GAIN Q3 MI). During treatment sessions following the baseline assessment, counsellors assessed clients general wellbeing monthly by administering a short tool: Ontario Within Treatment Outcome Measure for Addictions (OWTOM A). All recruited clients were also followedup at three and if possible, six months after the baseline assessment (depending on their recruitment date) by a member of the DTFP follow up team. The DTFP follow up team is a four person unit assembled to locate clients quarterly on the telephone and conduct the follow up interviews. The team administered the Global Appraisal of Individual Needs Quick 3 Standard (GAIN Q3 Standard) and assessed clients needs for additional treatment or support using the Return to Treatment (RTT) protocol. The 5 The original project plan called for one-year follow-up. This was subsequently reduced due to delays in funding and project start-up. 6 People seeking help specifically for gambling-related problems or process addictions such as video gaming or sex addiction were not included. 19

20 team then facilitated linkage to treatment for clients in need by either connecting them to ConnexOntario or to the clients original treatment agency. 2.2 STAKEHOLDER CONSULTATIONS A broad range of stakeholders were engaged in the planning stages via an overall the Program Advisory Committee and a Working Group. Collaborators included relevant CAMH clinical programs, Addictions and Mental Health Ontario 7, the MoHLTC, Local Health Integration Networks (LHIN), specialized mental health and addiction agencies (directors, managers, clinicians and researchers), and a consumer representative. Stakeholders informed key decisions regarding the selection of tools, pilot sites and procedures for implementation. Members continued to be involved in the implementation stage, some by participating as pilot sites and others through an advisory function. In the later stages, this group of stakeholders also informed analysis and interpretation of the evaluation feedback, and offered many suggestions for next steps for both continued testing and provincial implementation. The group(s) also reviewed and approved all recommendations emanating from this and the other related DTFP projects (screening and assessment; client perception of care; and costing). 2.3 ENVIRONMENTAL SCAN In September 2011, all publicly funded addiction treatment agencies in Ontario, identified using contact information provided by ConnexOntario, were approached to participate in an online survey. The goal was to obtain a province wide perspective on the current practices and tools used for either within treatment monitoring of outcomes, or client follow up. Out of the 190 agencies that were contacted to complete the survey, 46 programs/services responded. Specifically, about 36 programs indicated routinely monitoring client outcomes during treatment, and a small number (19) monitored outcomes post treatment. Clients were reported as being followed up for an average of six to twelve months, and a variety of outcomes were measured, including change in 7 Addictions Ontario and Ontario Federation of Community Mental Health and Addiction Programs merged into a new organization as of 2012, now called Addictions and Mental Health Ontario. 20