Drug Treatment Funding Program Client Recovery Monitoring Project

Drug Treatment Funding Program Client Recovery Monitoring Project Final Report for the Ministry of Health and Long Term Care Brian Rush Nooshin Khobzi Rotondi Nancy Chau April Furlong Alexandra Godinho Christina Schell Christine Kwong Safa Ehtesham Health Systems and Health Equity Research Centre for Addiction and Mental Health Toronto, ON August 2013 1

TABLE OF CONTENTS 1.0 BACKGROUND AND RATIONALE... 12 1.1 THE DRUG TREATMENT FUNDING PROGRAM (DTFP)... 16 1.2 PROJECT OBJECTIVES... 18 2.0 METHODS... 19 2.1 OVERVIEW OF STUDY DESIGN... 19 2.2 STAKEHOLDER CONSULTATIONS... 20 2.3 ENVIRONMENTAL SCAN... 20 2.4 SELECTION OF TOOLS... 21 2.4.1 SITE VISIT TO CHESTNUT HEALTH SYSTEMS GAIN COORDINATING CENTRE... 21 2.4.2 Literature Review and Tool Selection Criteria... 21 2.5 MEASURES... 24 2.5.1 Stage 1 Assessment and Follow Up Tools: Global Appraisal of Individual Needs Quick 3 (GAIN Q3) Motivational Interviewing (MI) and Standard... 24 2.5.2 Ontario Within Treatment Outcome Measure for Addictions (OWTOM A)... 25 2.5.3 Return to Treatment (RTT) Protocol... 25 2.6 PILOT SITES... 26 Rideauwood Addiction and Family Service... 27 Four Counties Addiction Services Team (Fourcast)... 27 Addictions Centre (Hastings/Prince Edward Counties)... 28 Manitoulin Community Withdrawal Management Services... 28 2.7 STUDY PARTICIPANTS... 28 2.8 QUANTITATIVE DATA COLLECTION PROCEDURES... 29 2.8.1 Recruitment and Consent Process... 30 2.8.2 Baseline Assessment and Within Treatment Outcome Monitoring... 31 2.8.3 Three and Six Month Interviews... 31 3.0 PILOT SITE ENGAGEMENT, TRAINING AND MONITORING OF PROCEDURES... 33 3.1 ENGAGEMENT OF PILOT SITES... 33 2

3.2 TRAINING OF PILOT AGENCY STAFF... 33 3.3 MONITORING OF PROCEDURES AND DATA QUALITY... 35 3.3.1 Monitoring Visits and Support to Pilot Sites... 35 3.3.2 Data storage and Central Database Development... 36 4.0 LAYING THE GROUNDWORK FOR IMPROVEMENTS IN MEASURING CLIENT OUTCOMES AND PERFORMANCE INDICATORS IN THE SUBSTANCE USE TREATMENT SYSTEM... 37 4.1 DEVELOPING AN ONTARIO VERSION OF THE GAIN Q3... 37 4.1.1 GAIN Revisions and Assessment Building System (ABS)... 37 4.1.2 GAIN Training... 39 4.2 DEVELOPING A RECOVERY MONITORING SYSTEM... 40 4.2.1 Overview of the Follow Up Protocol... 40 4.2.2 Changes to the protocol... 46 5.0 EVALUATION STRATEGY... 47 5.1 QUALITATIVE DATA ANALYSIS... 51 5.2 QUANTITATIVE DATA ANALYSIS AND DESCRIPTION OF STUDY SAMPLE... 52 6.0 FINDINGS AND INTERPRETATIONS... 55 6.1 FEASIBILITY AND IMPLEMENTATION ISSUES: QUANTITATIVE RESULTS... 55 6.1.1 Representativeness of sample... 55 6.1.2 Administration of Tools... 60 6.1.3 Follow Up Rates... 63 6.1.4 Qualitative Feedback... 70 6.2 LESSONS LEARNED AND CHALLENGES... 75 7.0 RECOMMENDATIONS, POTENTIAL IMPACTS AND NEXT STEPS... 77 7.1 RECOMMENDATIONS... 77 7.2 POTENTIAL FOR IMPROVEMENTS IN MEASURING CLIENT OUTCOMES AND PERFORMANCE INDICATORS IN THE SUBSTANCE USE TREATMENT SYSTEM... 78 7.3 NEXT STEPS AND IMPLICATIONS... 80 3

LIST OF TABLES TABLES PAGE 1 Summary of data collection activities 31 2 Summary of training attendance for each pilot site 35 3 Evaluation questions and indicators of success 49 4 Comparison of Study Sample to Treatment Population in Ontario 56 5 Completion Times for Recovery Monitoring Tools 60 6 Characteristics of clients (from London and Peterborough) who 67 completed both 3 month and 6 month interviews VS those lost to follow up (among clients with GAIN Q3 MI) 4

LIST OF FIGURES FIGURES PAGE 1 Framework for the Stages of Client Engagement across 16 Screening, Assessment and Recovery Monitoring 2 An Overview of the Recruitment Process 53 3A Follow up data at 3 months 65 3B Follow up data at 6 months 66 5

LIST OF APPENDICES APPENDIX PAGE 1 Glossary and List of Acronyms 82 2 Recovery Monitoring Project Tools 85 6

Executive Summary In 2008 Health Canada announced the Drug Treatment Funding Program (DTFP), a key element of the National Anti Drug Strategy. The focus of the DTFP was on enhancing the systems of services for people with substance use problems in Canada, emphasizing three broad target areas for investment: implementation of evidence based practices; strengthening evaluation and performance measurement; and knowledge exchange. Each province and territory was invited to submit proposals for system enhancement. The Ontario submission included the present project, the objectives being to assess the feasibility of implementing a comprehensive recovery monitoring system for addiction services based on a hybrid approach which included a withintreatment and a post intake component. The project built upon other work undertaken for the MOHLTC in the past few years on screening and assessment tools and processes in Ontario and outcome monitoring models and measures. Specifically, the project examined the feasibility of implementing a dedicated recovery monitoring service and the appropriateness and usefulness of standardized recovery monitoring tools for measuring change over time. A broad range of stakeholders were engaged in the project via a Program Advisory Committee and Working Group. Partners included the Ministry of Health and Long Term Care, specialized mental health and substance use agencies (directors, managers, clinicians and researchers), Local Health Integration Networks (LHINs), and consumer representatives. This group, and a special sub group specific to this project, informed all stages of the decision making related to the selection of the tools and pilot sites, procedures for implementation, results interpretation, and wider dissemination of project findings. The selection of outcome tools involved several steps, the first being a comprehensive literature review that evaluated recovery monitoring models and tools used in mental health and/or addiction settings. The project team also conducted an environmental scan in Ontario to determine the scope of use of any kind of standardized tools to assess client outcomes in publicly funded addiction agencies, 7

common outcomes measured and what measurement tools and processes were in place. As a result of this literature review and environmental scan, a list of potential tools and the criteria for selecting the tools for pilot testing for a hybrid recovery monitoring system were summarized and presented to the Advisory Committee and Working Group for a final decision. The post intake recovery monitoring tool endorsed by the Advisory Committee and Working Group for this project was the Global Appraisal of Individual Needs Quick 3 (GAIN Q3) Standard. The GAIN Q3 is a multi purpose targeted assessment tool used to identify and address a wide range of life problems among both adolescents (age of 12 17 yrs) and adults (age of 18 yrs+) in diverse treatment settings. It is a valid and reliable tool and is sensitive in detecting change in the many life areas assessed at baseline by its sister tool, the GAIN Q3 MI. The Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) was developed specifically for this project, in collaboration with project stakeholders, to monitor clients general well being during their treatment process. The measure consists of nine items; six were adopted from the Life Satisfaction Scale in the GAIN Q3 MI, which assesses clients general satisfaction with life. The remaining three items concern self efficacy, perceptions of the relevance of treatment to clients needs, and motivation for change and were developed by the project team and project stakeholders. Five publicly funded Ontario based addiction agencies of various geographic locations, sizes and service types participated in this project and piloted the use of GAIN Q3 MI (baseline assessment tool) and the OWTOM A within treatment outcome measure. Agency staff and clinicians were involved in data collection after obtaining appropriate training related to the administration of the study tools, including interpretation of the GAIN Q3 and the data collection procedures. The project followed up participating clients for up to 6 months after their intake assessment. During these 6 months, the clients were asked to complete the OWTOM A on a monthly basis with their treating clinicians as long as they were in treatment. They 8

were also to complete the GAIN Q3 in a telephone interview with the follow up team to assess change in clients life problems since intake. Through this follow up process, the project facilitated timely sharing of the recovery monitoring information between the follow up team and agency clinicians and encouraged discussion of assessment results with clients. The project also implemented the Return to Treatment protocol through which it actively provided support for linking clients to treatment either through their recruiting agencies or ConnexOntario, if a need or desire was indicated during the follow up process. The feasibility of the recovery monitoring system that was piloted was evaluated using both quantitative and qualitative methods. The project conducted statistical analyses on a list of technical indicators and collected feedback on the strength and limitation of the recovery monitoring protocol and the tools from participating clinicians and the follow up team. Clients feedback was reported indirectly through clinicians and follow up team members. Data collection occurred between June 2012 and March 2013. In total, 234 clients were enrolled into the Recovery Monitoring Project and 218 of them agreed to the recovery monitoring process and provided the follow up team with their locating information. The focus of the analyses was on a combined sample of two pilot sites that made up about 90% of our total study sample. Of the 200 clients included in the analyses, 148 completed a baseline interview and could potentially provide change data for up to 6 months. The follow up team was able to reach 99 (67%) of these 148 clients at 3 months after intake and 89 (69%) of the 129 clients who were due for a 6 month follow up. It is important to note that this project did not provide any monetary incentives for participants. Results of analyses showed the OWTOM A to be a quick and easy tool for monitoring client progress over time and while still in treatment. The participating clinicians found that the tool facilitated their clinical process and was helpful in guiding treatment sessions with clients. Yet some clinicians noted that change between 9

administrations were hard to detect and others reported not feeling comfortable to be candid about their therapeutic relationship. The collected feedback on the GAIN Q3 showed that the tool provided opportunity for clients to discuss their progress and recognize changes in their behaviour. The clients, however, perceived some items as being repetitive. This was compounded by issues with telephone administration such as cost and privacy concerns with household lines. The Return to Treatment protocol was shown to facilitate re engagement of clients with agencies and provided clients with a sense of ongoing support. Clients also expressed appreciation of efforts to keep them connected to services. Challenges raised during implementation of the protocol included difficulties in confirming an appointment between clients and agencies to take full advantage of the opportunity of re engagement. In addition, little interest was shown among clients in being connected to ConnexOntario. The follow up protocol implemented by the follow up team was adapted from the one developed by Dr. Christy Scott, who specializes in recovery monitoring. Overall, the follow up protocol was well received by clients. Clients indicated that the follow up process allowed them to feel connected and to extend their support network and that they expressed appreciation for the attention the follow up team had shown them about their well being. On the other hand, some clients perceived the follow up process as intrusive, although it was recognized as being necessary to obtaining useful outcome data. In conclusion, this pilot project has demonstrated that recovery monitoring in Ontario s addiction services is feasible. Overall, agency staff, clients and the follow up team found great value in the recovery monitoring process. It was useful from a clinical perspective, provided more opportunities for building rapport with clients, and allowed for re engagement with the treatment system. Nonetheless, there were challenges raised in the piloting process as outlined earlier. The team were also concerned that the follow up procedures did not successfully engage less stable, more severe and 10

potentially more marginalized clients. Given these study findings and the resourceintensive nature of this work, the project team feels that the follow up protocol and related resource requirements is not ready to be implemented province wide. The project team recommends more piloting so that procedures and additional components of the follow up protocol that were not incorporated during the initial phase of piloting may be refined. These procedures and test components include use of paid community trackers, different strategies for follow up interviews that decrease costs for clients, and other approaches to reconnect clients with services. Future studies also need to involve more diverse clients with more severe substance use problems, and those receiving residential services. Although the piloted recovery monitoring system is not ready for a provincial dissemination, the project team encourages its application at the individual agency and LHIN levels if adequate resources and administrative support can be provided. The results from these more local implementation efforts would then serve to support plans for broader implementation in other parts of the province. 11

1.0 BACKGROUND AND RATIONALE At the client level, outcomes of addiction treatment are the changes in their symptoms, behaviour and function that can be attributed to their participation in treatment (McLellan, Chalk, & Bartlett, 2007). Quantifying clinical and functional outcomes during a specified period of time is the main component of outcome monitoring. An effective system of outcome monitoring can provide accountability information to funders and administrators; contribute to addiction service research as a complement to more controlled clinical research studies; and complement information on service utilization and cost, client satisfaction, and treatment process. It can also facilitate clinicians work with their clients, for example, by providing quick feedback on clients progress so as to adjust the treatment plan. In 2008 the Ministry of Health and Long Term Care (MoHLTC) commissioned a project that built upon a previous evaluation of the provincial Admission and Discharge Criteria and Assessment Tools (ADAT) 1, to review models of outcome monitoring for addiction treatment, with a view to eventually developing a feasible outcome monitoring system across Ontario s addiction agencies (Rush, Martin & Corea, 2008). The earlier MoHLTC funded project on outcome monitoring closely engaged representatives of Ontario s addiction treatment network. Subsequent recommendations called for a client model of outcome monitoring whereby the baseline assessment would be conducted by agency staff, one or more brief measures of outcome would be obtained from the client periodically during the course of treatment, and an independent, third party would follow up clients at quarterly intervals thereafter (Rush, Martin, Corea & Rotondi, 2012). The within treatment monitoring procedure developed by McLellan and colleagues (2005) was an important component of the proposed model. Within treatment measures record client progress on selected domains of functioning such as substance use, drug injection behaviour, criminal activity, 1 ADAT is comprised of a common set of assessment instruments (8) and a decision tree for assigning the client to the appropriate level of care, for application across all Ministry funded addiction services. 12

and health and social functioning on broader treatment goals such as life satisfaction and self efficacy. These measures of client progress are intended to assist clinicians in planning treatment, providing feedback to clients about their treatment progress, and systematically record client progress over time. When combined with follow up data, these within treatment measures provide a comprehensive description of client outcomes. In a related model known as Recovery Monitoring Check ups (RMC) developed by Dennis and colleagues (Dennis, Scott, & Funk, 2003; Scott, Dennis, & Foss, 2005;) another unique feature was introduced into the follow up component, whereby a defined protocol is used at follow up to determine if the client had experienced relapse, or was at significant risk of relapse, and is used to recommend the clients return to treatment. In short, the recovery monitoring model that was recommended at the conclusion of the MoHLTC funded project was referred to as a hybrid model (Rush, Martin, Corea & Rotondi, 2012) and builds upon the literature on recovery monitoring that had been developing over the past 10 15 years. This model has moved away from the traditional pre post follow up strategy by integrating outcome determination directly into the treatment and support process rather than viewing it as a costly research add on (McLellan, McKay, Forman, Cacciola & Kemp, 2005). The current project was based on this key conceptual shift; i.e., the evolution from outcome monitoring to recovery monitoring. The latter is the preferred terminology as it better reflects the chronicity of substance use problems for the most severe clients; provides a focus on the continuous monitoring of treatment outcomes; and places an emphasis on Return to Treatment for those clients in need. Another important contextual factor for the present project is the considerable research and development over the past decade in the area of co occurring mental and substance use disorders and, in particular, work related to the more effective and earlier identification of individuals with co occurring disorders in mental health, addictions and 13

other types of settings. The seminal Canadian report on concurrent disorders (Health Canada, 2001), a more recent update (Canadian Centre on Substance Abuse, 2009), and other international reviews have advocated for universal screening of mental disorders in addiction services and for substance use disorders in mental health services. As a response to this, system planners and managers of mental health and addiction programs in Ontario have been very active in selecting and implementing screening tools into their services and broader community treatment systems (Centre for Addiction and Mental Health, 2008). The GAIN Short Screener (GAIN SS), in particular, has been well received and widely implemented in Ontario due to its utility in mental health as well as addiction and other settings; its brevity, low cost and utility as an outcome measure, and the supporting validation data that extends to age 12 2. Ontario researchers also undertook a major validation study of various screening tools in Ontario addiction treatment settings with positive results obtained for the GAIN SS (Rush, Castel, Brands, Toneatto, & Veldhuizen, 2013). A major research synthesis was also undertaken of screening tools for children and adolescents and the GAIN SS emerged as one of the best supported options for this younger population (Rush, Castel, Somers, Duncan & Brown, 2009). An integrated view of screening, assessment and recovery monitoring is articulated by Rush and Castel (2011) and represented in Figure 1 below. In sum, the work on screening for concurrent disorders has evolved to the point where a review of Ontario s assessment tools and processes, and consideration of models and measures for recovery monitoring, must include screening within its scope. In this screening and assessment framework the process is divided into the following stages: Stage 1 and Stage 2 Screening, and Stage 1 and Stage 2 Assessment (Rush & Castel, 2011). The staged approach works to ensure a progressive and efficient use of screening and assessment resources to guide treatment planning and eventually, recovery monitoring. The main benefit of using a stepped approach in screening is 2 The original validation data on the GAIN-SS extended to the lower age limit of 10. However, experience in the national DTFP project led by Gloria Chaim and Joanna Henderson that involved a network of youth serving agencies suggested the tool is better used for youth 12 years of age and up. 14

through sequential case finding, as both the clinician and client time can be saved by reserving the more resource intensive screening and assessment tools for those who score above the cut off on briefer, more economical screening tools. Recovery monitoring is also greatly facilitated by choosing screening as well as assessment tools that can reliably and validly assess change over time. Figure 1. Framework for the Stages of Client Engagement across Screening, Assessment and Recovery Monitoring 1 adapted from Rush and Castel (2011) The recent developments in the addiction field that consider recovery monitoring as a continuous process occurring both during treatment as well as after 15

treatment are major steps forward in conceptualizing and operationalizing the links between screening, assessment and recovery monitoring. The flow and linkage across the stages of client engagement is the framework on which this project was based. 1.1 THE DRUG TREATMENT FUNDING PROGRAM (DTFP) In 2007, the Canadian federal government introduced the National Anti Drug Strategy to focus efforts on reducing the demand for, and supply of, illicit drugs; as well as to address the crime associated with illegal drugs. The National Anti Drug Strategy emphasized the need for effective treatment services to better manage the illicit drug problem when it occurs, and to help those in need. The need to make improvements at the level of the treatment system emerged as a priority for action, based on countrywide consultations and national level studies on treatment needs and gaps carried out in support of the development of the National Anti Drug Strategy. These studies demonstrated the need for systemic change to move treatment systems toward more evidence informed practices, while also increasing systems capacity to evaluate practices for their efficiency and effectiveness (Health Canada, 2008). In response to these findings, the Drug Treatment Funding Program (DTFP) 3 was announced in 2008, providing new, five year, time limited funding (2008 2013) 4 to assist provincial and territorial governments in addressing critical treatment needs in three investment areas (Health Canada, 2008): 1) Implementation of evidence informed practice Although evidence informed practices to optimize treatment exist, and are continually being developed, many do not easily find their way into service delivery. Evidence informed practice is based on interventions that effectively integrate the best research evidence with clinical expertise, cultural competence and the values of the persons receiving 3 Re oriented funding from Health Canada s Alcohol and Drug Treatment and Rehabilitation Program, effective April 2008 4 Timelines varied across provinces and territories depending on the release of funds; Ontario s projects received funding for 19 months of work (April 2011 March, 2013). 16

the services. These interventions have evidence showing improved outcomes for families, clients, and/or communities. 2) Strengthening evaluation and performance measurement Performance measurement and evaluation activities across jurisdictions are limited. While all jurisdictions collect, manage and analyze performance information pertaining to their treatment services and programs, the type and nature of data collected, as well as the approach to data collection and analysis varies considerably. This strategic investment area strengthens evaluation and performance measurement capacity and activities. 3) Knowledge exchange This investment area is an essential element of work undertaken in investment areas #1 and #2. Knowledge exchange activities can include such things as mentoring and the provision of technical expertise; mechanisms that develop and/or enhance knowledge sharing and dissemination of lessons learned from communities of practice in the uptake of best practices and in performance measurement/evaluation; and activities that can effectively reach health professionals within organizations and service sectors to increase their awareness and participation in knowledge exchange activities across the continuum of treatment services. Each province and territory was invited to submit proposals for enhancement of their substance use treatment system. One component in the Ontario submission is the focus of this report, namely testing the feasibility of a provincial recovery monitoring system. As a key component of investment area #2, this project lays the groundwork for the systematic collection, management and analysis of performance indicators related to client outcomes. Another related project that is the subject of a separate report is the review of Ontario s screening and assessment tools and processes within substance use treatment services; with a view to replacing the current package of client assessment tools with a more up to date suite of evidence informed tools for treatment planning and that are better linked to recovery monitoring. 17

It is important to highlight the significant delay that occurred between the broader Ontario proposal development and actual funding. Funding was approved in April of 2011 and the DTFP national funding ended in March 2013; thus Ontario effectively lost over two years of project time. This had significant implications for the project on screening and assessment tools as well as the recovery monitoring project. Specifically, the original proposal for the recovery monitoring project called for four quarterly check ups over a one year period. However, the project was implemented with only two quarterly check ups over six months. 1.2 PROJECT OBJECTIVES The recovery monitoring model and the specific outcome measures were pilot tested in the same five addiction agencies as the piloting of the screening and assessment tools, since the system that is envisaged in the future is an integrated screening, assessment, within treatment and follow up recovery monitoring system. It is important to understand that the project engaging these five pilot sites was not about the evaluation of the treatments that they offered, but rather the feasibility assessment of a provincial recovery monitoring system. These goals are reflected in the following statement of objectives: 1. To examine the feasibility of implementing common within treatment outcome measures and the potential usefulness of the results among decision makers at the clinical, program and system planning levels. 2. To assess the feasibility of a comprehensive system of client recovery monitoring and the potential use of the results among decision makers at the clinical, program and system planning levels. 18

2.0 METHODS 2.1 OVERVIEW OF STUDY DESIGN This feasibility study was conducted from June 2012 to March 2013 and involved the piloting of evidence informed tools and processes related to assessment and recovery monitoring in Ontario s addictions treatment services. Ethics approval was obtained from the Centre for Addiction and Mental Health and Ottawa Carleton Research Advisory Committee. As noted earlier, the model implemented in this project was based on the hybrid approach recommended in a MoHLTC funded project by Rush and colleagues (Rush, Martin, Corea & Rotondi, 2012). Measures were used to: 1) record client progress on various domains of functioning on a monthly basis while the client was receiving treatment; and 2) compare outcome data collected quarterly for up to six months 5 with baseline data from the intake assessment. Clients seeking help for their substance use problems 6 from five addiction treatment agencies across Ontario were approached for participation in the project. Following consent, the recruited clients completed the baseline assessment tool: Global Appraisal of Individual Needs Quick 3 Motivational Interviewing (GAIN Q3 MI). During treatment sessions following the baseline assessment, counsellors assessed clients general wellbeing monthly by administering a short tool: Ontario Within Treatment Outcome Measure for Addictions (OWTOM A). All recruited clients were also followedup at three and if possible, six months after the baseline assessment (depending on their recruitment date) by a member of the DTFP follow up team. The DTFP follow up team is a four person unit assembled to locate clients quarterly on the telephone and conduct the follow up interviews. The team administered the Global Appraisal of Individual Needs Quick 3 Standard (GAIN Q3 Standard) and assessed clients needs for additional treatment or support using the Return to Treatment (RTT) protocol. The 5 The original project plan called for one-year follow-up. This was subsequently reduced due to delays in funding and project start-up. 6 People seeking help specifically for gambling-related problems or process addictions such as video gaming or sex addiction were not included. 19

team then facilitated linkage to treatment for clients in need by either connecting them to ConnexOntario or to the clients original treatment agency. 2.2 STAKEHOLDER CONSULTATIONS A broad range of stakeholders were engaged in the planning stages via an overall the Program Advisory Committee and a Working Group. Collaborators included relevant CAMH clinical programs, Addictions and Mental Health Ontario 7, the MoHLTC, Local Health Integration Networks (LHIN), specialized mental health and addiction agencies (directors, managers, clinicians and researchers), and a consumer representative. Stakeholders informed key decisions regarding the selection of tools, pilot sites and procedures for implementation. Members continued to be involved in the implementation stage, some by participating as pilot sites and others through an advisory function. In the later stages, this group of stakeholders also informed analysis and interpretation of the evaluation feedback, and offered many suggestions for next steps for both continued testing and provincial implementation. The group(s) also reviewed and approved all recommendations emanating from this and the other related DTFP projects (screening and assessment; client perception of care; and costing). 2.3 ENVIRONMENTAL SCAN In September 2011, all publicly funded addiction treatment agencies in Ontario, identified using contact information provided by ConnexOntario, were approached to participate in an online survey. The goal was to obtain a province wide perspective on the current practices and tools used for either within treatment monitoring of outcomes, or client follow up. Out of the 190 agencies that were contacted to complete the survey, 46 programs/services responded. Specifically, about 36 programs indicated routinely monitoring client outcomes during treatment, and a small number (19) monitored outcomes post treatment. Clients were reported as being followed up for an average of six to twelve months, and a variety of outcomes were measured, including change in 7 Addictions Ontario and Ontario Federation of Community Mental Health and Addiction Programs merged into a new organization as of 2012, now called Addictions and Mental Health Ontario. 20

substance use, housing stability, employment status, etc. While some programs used standardized measures (Behavioural and Symptom Identification Scale (BASIS 32), Global Assessment of Functioning (GAF)), others administered home grown questionnaires. In general, most of the participants voiced concerns regarding the validity of the self report data and the challenges in locating clients for recovery monitoring. The results were essential in guiding the decision making process around tool selection and implementation of the pilot. 2.4 SELECTION OF TOOLS 2.4.1 Site Visit to Chestnut Health Systems GAIN Coordination Centre While waiting for project funding, members of the research team and various stakeholders visited the Chestnut Health Systems in Illinois to obtain information and updates on the development of the GAIN screening and assessment tools, as well as to attain firsthand knowledge of how training on GAIN administration are conducted. The team was impressed with the progressive suite of tools (GAIN family of instruments) developed by Chestnut Health Systems that supports a number of treatment practices from initial screenings to treatment planning (e.g., GAIN SS, GAIN Q3 and GAIN I). It is also suitable for use with both adolescents and adults from varied populations in different levels of care. The information obtained by the visiting team was subsequently conveyed to the Program Advisory Committee group members. 2.4.2 Literature Review and Tool Selection Criteria Rush and colleagues were commissioned to review outcome monitoring models in 2008 (Rush, Martin & Corea, 2008). This review included: An extensive literature review Discussion of preliminary ideas with experts in the field in Canada and internationally Development of a conceptual framework Identification and conceptualization of four models of outcome monitoring 21

Consultation events with relevant stakeholders to obtain feedback regarding issues related to service delivery, system planning and accountability issues Group The above report was focused on models but not specific measures and provided an important foundation for the review process undertaken for the current project. Building on this foundation, our project team supplemented the existing literature review with newly published research on both models and measures for outcome monitoring. Although the review process was mainly based on peer reviewed articles, some grey literature was also considered. Guided by the stakeholder consultation, the province wide environmental scan, and the derived selection criteria endorsed by the Program Advisory Committee/Working Group, results from this literature review were synthesized for thirteen tools. Six of these tools were developed for addiction services; and one was developed for mental health services. Five were developed for both addiction and mental health services. Most of these tools were validated for adolescents and adult populations. The majority of the reviewed tools were suitable for assessment, and could detect change over time; i.e., during and post treatment. The results of the review were examined by the Working Group to identify any gaps; e.g., tools not included, review criteria. In the end, a short list of tools was presented to the Working Group for final selection based on the relative weight of each of the selection criteria. This work was also synchronized with the parallel review of screening and assessment measures. The tool selection criteria were developed based on the findings of the environmental scan, the review of the literature, and the expertise of our Working Group. The selected tool(s) was expected to meet the following criteria: Measures outcomes in the following domains: o Substance use (decrease in alcohol and drug use) o Personal health (increase in personal health; i.e. improvements in medical and psychiatric health) 22

o Social functioning (improvements in social functioning; e.g. employment, family and social relationships) o Public health and safety (reductions in threats to public health and safety) Can be used for assessment, within treatment and follow up evaluations in order to allow for valid determinations of change over time Can be used for baseline and post intake follow up, while a shortened (brief) version of that same tool could be used for within treatment assessments Is relevant to different types of treatment programs (e.g. outpatient, residential, withdrawal management) The following selection criteria were also considered: Good psychometric properties Valid for a wide age range and for different gender/ethnic groups Flexible administration Easy to build into treatment process Link to assessment and treatment planning Low demands on time and low cost Minimal training requirements Availability in multiple languages At the project start, the DTFP project team compiled a summary of the review results, supplemented with a search for any newly published research. This review was organized by the selection criteria endorsed by the Program Advisory Committee/Working Group. A short list of tools was then presented to the Working Group for final selection based on the relative weight of each of the selection criteria. 23

2.5 MEASURES 2.5.1 Stage 1 Assessment and Follow Up Tools: Global Appraisal of Individual Needs Quick 3 (GAIN Q3) Motivational Interviewing (MI) and Standard The GAIN Q3 is a multi purpose, targeted assessment tool used to identify and address a wide range of life problems among both adolescents (age of 12 17 yrs) and adults (age of 18 yrs+) in diverse settings (GAIN Co ordinating Center, 2013). The overall aim of the GAIN Q3 is to fairly quickly sort individuals into three groups: a) those who do not appear to have problems in need of attention, b) those who appear to have mild problems that can be addressed in a brief intervention, and c) those whose results indicate the need for a more detailed assessment and/or specialized treatment. In the case of individuals whose results indicate mild problems, the GAIN Q3 MI system includes instructions to conduct a brief intervention based on the principles of motivational interviewing. Domains covered in the GAIN Q3 include school problems, work problems, physical health, sources of stress, risk behaviours for infectious diseases, mental health, substance use, crime and violence and life satisfaction (Lighthouse Institute, 2011). In this study, the GAIN Q3 MI was administered at baseline and the GAIN Q3 Standard at three and six month follow up. The GAIN Q3 Standard includes items that focus on the individuals behaviour during the past 90 days, and the GAIN Q3 MI includes the same items, as well as a set of items for each domain that ask about individuals reasons and readiness to change behaviours. The instruments were interviewer administered both electronically and via paper and pencil. The average time of administration for the GAIN Q3 MI and the GAIN Q3 Standard is stated as 45 and 35 minutes, respectively. Furthermore, there is a substantial body of evidence regarding the tools psychometric properties, and the existing database compiled by other users can be used for the purposes of benchmarking and research (Chestnut Health Systems, 2013; Dennis, et al., 2002; Godley, Godley, Dennis, Funk, & Passetti, 2002; Lennox, Dennis, Scott, & Funk, 2006; Shane, Jasiukaitis, & Green, 2003). Some of 24

the items in the GAIN Q3, particularly questions related to demographics, are appropriate only to the U.S. setting. With the support of Chestnut Health Systems, we developed an Ontario version of the GAIN Q3 (see Appendix 2) in order to increase the tool s relevance to the Ontario context. We also worked with a research group from Quebec to standardize the revisions across Quebec and Ontario. For more details, refer to the section below regarding GAIN revisions and the Assessment Building System (ABS). In general, the suite of tools was selected as they met most of the criteria outlined above, and received overwhelming support from the Program Advisory Committee/Working Group. 2.5.2 Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) The Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) was developed specifically for this project, in collaboration with project stakeholders, to monitor clients general well being during their treatment process (Appendix 2). The measure consists of nine items; six were adopted from the Life Satisfaction Scale in the GAIN Q3, which assesses clients general satisfaction with life. The remaining three items concern self efficacy, perceptions of the relevance of treatment to clients needs, and motivation for change. These items were developed by the project team and the Working Group. The OWTOM A can be interviewer or self administered (via paper and pencil) on a monthly basis, and it was estimated to take three to five minutes to complete. 2.5.3 Return to Treatment (RTT) Protocol The RTT is a brief set of questions that was administered during the three and six month follow up interviews. Designed by the project team and Working Group and working from the model developed by Dr. Christy Scott and colleagues, the RTT asks the client s needs for and interests in further treatment or support and provides linkage between clients and services accordingly (see Appendix 2). It is interviewer administered via paper and pencil and was estimated to take five minutes to complete. 25

2.6 PILOT SITES Five pilot sites participated in this study. The sites, described in detail below, were selected to represent a diversity of addiction agencies in Ontario, and thus include a range of programs and clientele 8. Pilot sites were selected based on the following criteria: Geographic location rural/urban, north/south Size ranging in size from small, single program sites to multi program sites Client flow covering a range in client flow Program type a range of program types (e.g., community withdrawal management, community based treatment) Populations served targeting youth (12 years of age and older) and/or adults Staff and clinicians at the pilot agencies were involved in data collection, and obtained appropriate training related to the administration of the tools and recovery monitoring procedures. Each pilot site also selected one staff member to serve in the role of study lead. This position involved certain responsibilities; for example, study leads were required to complete the Tri Council Policy Statement 2 ethics module; store study recruitment materials and hard copies of study data securely; serve as the central point of contact (i.e., liaise between the project coordinators and all the staff at the agency); encourage feedback from staff related to the project; and provide assistance on scheduling study visits for monitoring purposes. Service contracts were developed with the pilot agencies outlining agreements related to roles and responsibilities, and requirements. As all of the pilot sites provide more than one program/service those to be included in the pilot were negotiated at the outset and outlined in the service contracts. 8 At the onset of the project a short-term residential treatment program joined the project but eventually withdrew for internal reasons unrelated to the project. 26

Addiction Services of Thames Valley (ADSTV) Addiction Services of Thames Valley is a community based service, operating in co operation with local addiction and health care agencies, through the Southwest LHIN. There are 8 programs that are operated by ADSTV, and through these programs, ADSTV offers assessment, counselling, support, education, employment and housing services for a wide diversity of individuals involved with substance abuse or gambling problems. They offer services in London, Strathroy (Middlesex), St. Thomas (Elgin), Woodstock, Ingersoll and Tillsonburg (Oxford). All ADSTV programs, except for those related to education participated in our pilot. http://adstv.on.ca/ Rideauwood Addiction and Family Service Founded in 1976, Rideauwood Addiction and Family Service is a registered charity serving adults, adolescents and family members in Eastern Ontario. They provide non residential, group and individual treatment, public education, training and consultation. Rideauwood also has an extensive volunteer program that provides essential services to the agency. For the pilot, only the school based program and the agency based youth program participated in the study. http://www.rideauwood.org/ Four Counties Addiction Services Team (Fourcast) Fourcast is a community addiction treatment agency offering professional counselling services for anyone concerned about substance use or problem gambling, whether for themselves or for others. The goal of Fourcast is to support clients by empowering them to make their own choices in an open, non judgmental atmosphere, with a focus on encouraging positive change. Fourcast provides initial assessment and treatment planning services, as well as community addiction treatment programs in the counties of Peterborough, Northumberland, Haliburton and the City of Kawartha Lakes. The pilot tools and procedures were implemented in all of these programs/services. http://www.fourcast.ca/ 27

Addictions Centre (Hastings/Prince Edward Counties) The Addictions Centre (Hastings/Prince Edward Counties) offers standardized addiction assessment and outpatient counselling (community treatment services) for individuals 16 years of age or older who are concerned with their substance use or gambling problems. The Centre also provides short term residential addiction treatment for men at a different site. The pilot tools and procedures were implemented in all of the services, except for the residential program. http://www.addictionscentre.ca/ Manitoulin Community Withdrawal Management Services Manitoulin Health centre includes two hospitals that serve the Manitoulin region. Manitoulin Community Withdrawal Management Services (MCWMS), a program of Manitoulin Health Centre provides support to clients voluntarily withdrawing from alcohol and/or other drugs. Clients may be residing at their home, the home of a significant other, or in another safe setting. MCWMS staff members also provide information and assistance to guide the support provider(s) supervising the "in home" care. MCWMS offers three main components in managing withdrawal: intake and assessment, withdrawal management, and continuing care. Assessment/treatment planning services and case management are also available. http://www.manitoulinhealthcentre.com/news/154/ 2.7 STUDY PARTICIPANTS Participants of the project were clients aged twelve or older, presenting for screening/assessment/treatment at one of the five pilot agencies across Ontario. Only clients who were in the intake stages of treatment and not currently receiving treatment services at the pilot agency were eligible to participate. An exception was made for Rideauwood as counsellors indicated that clients in the school based programs were typically not ready to complete assessment tools, prior to treatment sessions. As such, they were hesitant to approach intake clients for participation in our study. In 28

order to address this issue we asked counsellors at Rideauwood to recruit clients who were in the intake stages of treatment and those whom they had been counselling for one month or longer. Clients included in the study were presenting for a substance use problem, clinically stable, had consented to participate, were able to speak or understand English and showed no evidence of cognitive impairment based on a structured, validated scale assessing this domain (Katzman et al., 1983). Clients were not reimbursed for their participation. The purpose of this project was to determine the feasibility of implementing the tools and procedures piloted in this study across all addictions agencies in Ontario. Since clients are not paid for receiving substance use treatment in Ontario, doing so in our pilot would possibly overestimate the feasibility of our approach (i.e., in terms of participation rates). 2.8 QUANTITATIVE DATA COLLECTION PROCEDURES The data collection period began in mid June, 2012 (mid September, 2012 for Rideauwood Addiction and Family Service) and extended through March, 2013 to allow for two months of client recruitment and six months of recovery monitoring (both within treatment and post intake). For sites that had started data collection in June 2012, the recruitment period was extended until the end of September, 2012 to ensure adequate time to obtain a reasonable sample size for analysis purposes. The recruitment period was also extended for Rideauwood (ending early December, 2012). Furthermore, the six month follow up only applied to clients who completed the GAIN Q3 MI in early September, 2012. Clients who completed the GAIN Q3 MI later in the recruitment period were only followed up for three months. The project tools were administered via paper and pencil and electronically at particular time points, as summarized in Table 1. Agency staff were required to administer the baseline and within treatment tools, while members of the follow up team administered the GAIN Q3 Standard and Return to Treatment (RTT) Protocol by telephone interview. 29

Table 1. Data collection activities Instrument Baseline Recovery Monitoring Within Treatment 3 and 6 Month Check Up Global Appraisal of Individual Needs Quick3 (GAIN Q3) MI Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) Global Appraisal of Individual Needs Quick3 (GAIN Q3) Standard Return to Treatment (RTT) Protocol Monthly max of 4x) 2.8.1 Recruitment and Consent Process Clients admitted to addictions treatment services routinely undergo screening and assessment. Potential participants were approached by trained staff involved in the intake and assessment function at each pilot agency once some or all of the ADAT tools (less the DHQ) were administered. The recruitment process was typically initiated during the client s second visit, at which time they were provided with a description of the pilot project via a Letter of Information and Consent to Participate form. For clients under sixteen years of age, an assent form was provided and consent was obtained from parents or legal guardians. After providing informed consent/assent, clients cognitive competence was assessed using the GAIN Cognitive Impairment Screener. Clients who were not cognitively impaired were then asked to complete a Locator Form, which was used to locate clients for the follow up phase of the project (Appendix 2). The Locator Form requests a wide range of information proven to be useful in locating clients in other follow up studies specifically those operated by Dr. Christy Scott for Chestnut Health Systems. Upon completion of the Locator Form(s), clients were connected to a member of our follow up team (if possible) by the agency staff. The purpose of this call was to initiate rapport with the client, describe the follow up process, and set up a time to review the locater information with the client using a structured script. 30

Eligible clients were informed that their participation in this study was completely voluntary and that they could refuse to join the study or withdraw from it at any time without having any impact on their current or future services. They were also informed that in the event they decide to leave the study, they would have the right to allow or restrict the use of data that was collected from them up until withdrawal. However, clients were asked to provide reasons for refusal or withdrawal, as this would provide information about potential biases in the sample and contribute to the feasibility assessment for scale up across Ontario. 2.8.2 Baseline Assessment and Within Treatment Outcome Monitoring The assessment tool (GAIN Q3 MI) was administered by the assessment or treatment clinician at the beginning of the assessment phase. Once the client started treatment (e.g., client was seen regularly by a clinician following the assessment phase), they were asked by their clinician to complete the within treatment measure. This was usually done one month after the completion of the GAIN Q3 MI. The set of questions were re administered monthly to a maximum of four times, while the client was in treatment. Clinicians were trained in administering the questions, interpreting the results, and were expected to use the data collected to inform treatment planning and monitoring of goal achievement. The model was based on the concurrent recovery monitoring approach, which assumes: 1) an ongoing relationship between the client and the clinician; and 2) that the client is living in the community (e.g., queried behaviours can change between treatment sessions) (McLellan et al., 2005). 2.8.3 Three and Six Month Interviews GAIN Q3 Standard The follow up interviews were conducted by a member of the follow up team via telephone; all responses to the GAIN Q3 Standard were entered into an electronic interface (see section 4.1 for more information). The three month interview was conducted three months subsequent to the administration of the GAIN Q3 MI. The six 31

month interview took place three months subsequent to the completion of the threemonth interview. Based on the recommendations of Chestnut Health Systems, a onemonth window was used to administer the GAIN Q3 Standard. With the permission of the client, follow up data were made available to agency staff (if the client returned to or was still receiving treatment at the pilot site) in order to facilitate and enhance the treatment planning process. Return to Treatment (RTT) Protocol Immediately following the administration of the GAIN Q3 Standard at follow up, clients were asked to respond to the RTT protocol questions. If clients had not completed a GAIN Q3 MI at baseline, then clients were only asked to complete the RTT protocol during the three and six month interviews. In the event that clients selfreported the need for additional help, follow up workers encouraged them to talk to their counsellor if they were still receiving services at the recruiting agency. If clients were no longer receiving services at the recruiting agency, the follow up team would offer them the option of reconnecting to the agency for services. If clients voiced concerns about having their needs being met at the recruiting agency, a referral was made to ConnexOntario (http://www.connexontario.ca/); a helpline that provides information about drug and alcohol treatment services in Ontario and assists clients in their choice of program. With the client s permission, calls to ConnexOntario were made with the client on hold, thus they were linked immediately to a ConnexOntario referral agent. It was up to the client to decide whether or not they would contact the referred agency for an appointment. Names of clients who expressed an interest in returning to their recruiting agencies were sent to the study leads immediately after their follow up interviews. The recruiting agencies would then call the clients within one week upon receiving the clients requests. The research coordinator also followed up with study leads every week to determine the outcome of the reconnecting process. As with the GAIN Q3 Standard, 32

the results of the RTT protocol were shared with the pilot sites, but only if permitted by the client and assuming he/she was returning to, or was still receiving, treatment at the agency. 3.0 PILOT SITE ENGAGEMENT, TRAINING AND MONITORING OF PROCEDURES 3.1 ENGAGEMENT OF PILOT SITES In February 2012, initial visits were made to all of the pilot agencies by key members of the team (project lead, project scientist, and research coordinator). The goals of these engagement visits were to: Introduce the research team and meet all of the staff Share information regarding the background of DTFP and introduce the project Learn more about agency specific processes Review next steps prior to training and pilot launch The one day visits informed the finalization of the pilot protocol, and facilitated the building of relationships prior to training and launch. 3.2 TRAINING OF PILOT AGENCY STAFF Agency staff members participating in the pilot project were trained on the implementation of project processes and administration of project tools. They also underwent formal training and a certification process to administer the GAIN Q3 instruments as recommended by Chestnut Health Systems to ensure the validity and reliability of assessment results. The GAIN Q3 training is detailed in Section 4.1.3. Members of the DTFP research team visited the sites in May 2012 (August 2012 for Rideauwood) to provide training to all participating staff. The 1.5 day training provided information regarding the rationale, protocol, and procedures to administer and interpret the study tools. Specifically, the training reviewed the: 33

process of recruiting eligible clients, obtaining study participant consent and gathering locator information for follow up procedures; pilot tools, and the procedures for administering, scoring and interpreting the tools method for liaising with the follow up staff to locate clients for three and sixmonth interviews; and Return to Treatment protocol Participants in the training were provided with detailed training manuals which included: training slides laminated flowcharts outlining in detail the various components of the protocol consent, assent and withdrawal forms study eligibility checklists a copy of each of the tools to be piloted information on ethics project administration materials contact information for the DTFP project team, study leads and GAIN trainers A mix of clinicians and administration staff from each of the pilot sites were present at the training session (Table 2). Table 2. Summary of training attendance for each pilot site Number of Attendees Participating Pilot Sites Addiction Services of Thames Valley (ADSTV) Addictions Centre Admin. Clinicians Staff N (%) N (%) 22 (73) 8 (27) 30 7 (88) 1 (12) 8 Total 34

Number of Attendees Admin. Participating Pilot Sites Clinicians Staff N (%) N (%) Four Counties Addiction Services Team (Fourcast) 5 (83) 1 (17) 6 Manitoulin Community Withdrawal Management Services 3 (75) 1 (25) 4 Rideauwood Addiction and Family Services 6 (86) 1 (14) 7 Total 43 (78) 12 (22) 55 Total The training was evaluated using a structured questionnaire developed by the DTFP research team, which was administered at the end of the training sessions to assess participants understanding of the training materials/content. Almost all of the participants stated that they had a good understanding of the purpose, various project processes, and project tools at the end of training. After the training visits, the DTFP research team followed up with participants to review any sections of the training that needed further clarification. 3.3 MONITORING OF PROCEDURES AND DATA QUALITY 3.3.1 Monitoring Visits and Support to Pilot Sites During recruitment, research staff conducted site visits to review implementation of the study protocol, requirements for storage of study files, and overall status of project implementation. This was done in part to ensure the integrity of the protocol implementation, but also to ensure all requirements were being met for potential project audit by the CAMH Research Ethics Board. In general, once a pilot site had recruited approximately five participants, members of the research staff initiated visits to the agency. If there were no issues in the piloting of the project, then the research team returned at the end of the study recruitment period. If a pilot site was experiencing issues with any aspect of the project implementation, then the research team returned to the site as often as needed. During the site visits, all study related 35

materials were photocopied and brought back to CAMH for data entry by members of the research team. Site visits also included a de briefing with the study leads regarding any outstanding scheduling and procedural issues. Prior to data entry, research staff checked all study documents for missing data or inconsistencies, and followed up with study leads to make needed corrections. Throughout the duration of the pilot, ample support was provided to the participating staff at the agencies. This included a weekly email to all study leads, in which we provided an update on the project (in terms of recruitment numbers, etc.), as well as reminders on key aspects of the protocol (e.g., the importance of collecting complete locator information). Members of the research team were readily available to discuss any aspect of the project with study leads, either over the telephone or via email. Every attempt was made to facilitate the pilot process for the agencies, so as to reduce the burden on clinical staff. 3.3.2 Data storage and Central Database Development A unique identifier (alphanumeric code) was assigned to each participant. The key to the unique ID numbers was kept in a password protected Microsoft Excel file, which could only be accessed by the study leads and the research coordinator. Occasionally, access was granted to other members of the research team for the purpose of data entry. All completed study tools were stored in clients clinical files at the pilot sites in order for clinicians to use the information in treatment planning. In order to truly assess the usefulness and feasibility of the tools for treatment planning, clinicians were permitted to send summaries and reports generated from the study tools to outside agencies for the purpose of client referrals. This was necessary as agency staff often need to show proof or provide a reason as to why a client is being referred to a particular program. All recruitment and consent material, including the Locator Form, were kept in a locked filing cabinet that remained at each site in a research file (separate from the study tools). 36

All photocopied study material (except for the consent/assent forms and Locator Forms) that were brought back to CAMH were immediately de identified, i.e., clients names were removed and replaced with their unique identifiers. Client files were stored in locked filing cabinets at CAMH. A master database for entering all of the study data (except for the GAIN Q3 MI and Standard) was created in Microsoft Excel. All of the data were entered by one member of the research team, while another verified the information, that is checked to ensure that the data entered into the Microsoft Excel file matched the paper copies of the study documents. This was an important component of our quality control procedures. The database was password protected and saved onto a CAMH networked drive. 4.0 LAYING THE GROUNDWORK FOR IMPROVEMENTS IN MEASURING CLIENT OUTCOMES AND PERFORMANCE INDICATORS IN THE SUBSTANCE USE TREATMENT SYSTEM 4.1 DEVELOPING AN ONTARIO VERSION OF THE GAIN Q3 4.1.1 GAIN Revisions and Assessment Building System (ABS) Following the selection of the GAIN Q3 instruments, the Program Advisory Committee and Working Group recommended revisions to Canadianize the language from the instrument developed in the U.S. and in collaboration with a treatment research group in Quebec heavily invested in the use of the GAIN tools in treatment agencies in that province. Proposed revisions were vetted through an extensive consultation process. For example, items related to ethnicity/cultural background and sexual orientation were reviewed with colleagues within CAMH who have expertise in the area of health equity and with colleagues within the Drug and Alcohol Treatment 37

Information System (DATIS) to ensure consistency with mandatory reporting elements within the Ontario addiction sector. 9 Further edits and additions were also required to reflect the Ontario context, most notably the addition of questions related to trauma and adjustments in language to reflect a harm reduction philosophy and approach. Additional items from the larger GAIN I instrument were also added (e.g., housing status, living environment, barriers to treatment and physical health) so as to better cross walk with existing ADAT admission and discharge criteria. Relevant changes were also reflected in the GAIN Recommendation and Referral Summary (GRRS), which is automatically generated based on responses inputted into the GAIN ABS system (a web based application that allows the GAIN family of instruments to be administered and summarized by computer). All revisions to the instrument and GRRS were reviewed by a sub group of the project s Working Group, comprised of individuals with extensive experience with addictions treatment service delivery and with the ADAT admission and discharge criteria. A parallel process involved the integration of the GAIN ABS system into the Catalyst Software. Catalyst is a browser based computer application, managed by DATIS, which houses client level clinical and administrative data for MOHLTC funded addictions agencies in Ontario. Following negotiations with Chestnut Health Systems, the GAIN ABS application and database were hosted in Catalyst via encrypted web services connections and housed in a secure server located in Toronto, Ontario. Support staff at Chestnut Health Systems have remote access to the server system to upload patches and modifications (including the revised GAIN Q3 instruments and report templates) to the GAIN ABS platform as needed. For the purposes of the pilot, only the GAIN Q3 MI and the GAIN Q3 Standard were mounted onto the Catalyst ABS system. 9 DATIS contributes to the understanding and enhancement of problem gambling and addiction treatment in the Province of Ontario through the on going development and maintenance of a comprehensive, province wide client information system (http://www.datis.ca/index.php). 38

4.1.2 GAIN Training Formal training and certification to administer the GAIN Q3 instruments is required by Chestnut Health Systems to ensure the validity and the reliability of the assessment results. As such, a train the trainer model was used for the purposes of the project whereby between one and three staff from each of the pilot agencies, as well as one member from the research team (henceforth called Local Trainers ), attended a four day training delivered by Chestnut Health Systems in Illinois, U.S.. The training included presentations; small group work with hands on practice administering the GAIN Initial (GAIN I); a workshop to practice identifying and clarifying inconsistent responses; information about the Administration Quality Assurance (A QA) process; an overview of the GAIN generated clinical reports; discussions regarding using the GAIN to guide diagnosis and treatment planning; a demonstration of the GAIN ABS online version of the instrument, data management information; and an opportunity for trainees to make their first submissions toward GAIN Administration Certification. Local Trainers were then expected to return to their agencies and submit taped sessions of their administration of the GAIN I with clients until they demonstrated the requirements for Administrative Certification. The next phase, Local Trainer certification, required Local Trainers to train up to five staff at their agencies to administer the GAIN I, provide feedback and recommendations, conduct ongoing quality assurance, and recommend them for Administration Certification. The Local Trainers received continued support from the A QA Team for a total of six months to achieve both Administration and Local Trainer Certification. Once certified to train on the GAIN I, Local Trainers are then considered certified to also train on the shorter assessments, including the GAIN Q3, which they did for all clinical staff at the pilot agency, and for all follow up workers through a half day workshop. The DTFP research team Local Trainer also provided supplementary training to the agency pilot sites and follow up workers on the GAIN ABS system via a one hour demonstrative webinar, modeled on the monthly trainings provided by Chestnut Health Systems. DATIS issued passwords to all agency 39

staff and follow up workers in order to provide access to the ABS system. As with the larger Catalyst system, agencies were only permitted access to view data from GAIN assessments administered at their agency. The follow up team also held regular sessions with the Local Trainer to review challenges and issues that arose as they became familiar with the administration of the instrument. 4.2 DEVELOPING A RECOVERY MONITORING SYSTEM 4.2.1 Overview of the Follow Up Protocol The follow up team was comprised of three research analysts and a research coordinator who monitored and oversaw the entire process. During the pilot, clients were contacted by the follow up team at various time points including project engagement, verification of locating information, three months and six months interviews upon completion of the GAIN Q3 MI. The follow team was trained to locate clients over the phone and to administer the data collection tool during the recovery monitoring interviews. The implemented model was based on the approach developed by Dr. Christy Scott (2004). However, it was adapted to meet the needs of our provincial project and to account for contextual/cultural differences between Canada and the U.S. A toll free 1 800 number was established for this project. Processes were put in place to ensure client privacy through a non displayable identifier; voicemail was also available. Follow up workers were directed to introduce their affiliation as a Health Research Group, so as not to identify any clients as having received addiction treatment services. Follow up workers also kept a log detailing contacts with all clients, including failed attempts. More details are provided in the following sections. 40

Follow Up Protocol The protocol of the follow up process in the study was developed largely based on the follow up services established at the Lighthouse Institute of Chestnut Health Systems. The major components of the follow up process included engagement, verification of locating information, contact maintenance, and quarterly check up. Engagement The research team encouraged clinicians to connect recruited clients with the follow up team on the phone following the study consent process. These engagement calls provided the follow up team an opportunity to welcome the client to the recovery monitoring process and introduce themselves to the client. The calls also allowed the team to explain the purpose and the process of recovery monitoring in detail to the clients and set up a verification call appointment with them. Although the engagement calls were highly recommended, only 20% of the participants had an engagement call due to challenges with the availability of phones at the pilot sites. Given the circumstance, the activities originally designed to be in the engagement calls were completed during verification calls in this project. Verification The study leads at the pilot sites faxed consent and the Locator Forms for recruited clients through the project team s confidential fax line at the end of the day during the study period. Upon receiving the fax (or package sent via courier), a followup worker reviewed the Locator Form and outlined any missing or illegible locating information that needed clarification from clients. The follow up worker also verified each piece of contact information in the form by comparing it to registrations on public directories, such as 411.ca. Any unmatched information was noted in the client file to be clarified during the verification call. 41

The verification calls were made within 24 72 hours upon receiving the study documents from the pilot sites. During a verification call, the follow up team described the follow up procedures in detail using a structured script to the client and reviewed the Locator Form with him/her to correct any erroneous information and replace missing data. The team also informed clients of the upcoming mail or phone contacts during the call. Maintenance It is important to maintain contact with clients between the quarterly follow up interviews to build a relationship and remind them of upcoming appointments. To that end the follow up team designed and created an introduction pamphlet containing the details of the follow up process and the follow up team. This pamphlet was distributed to clients at the time of study recruitment. An appointment card with the follow up team s contact information was also given to the clients along with the pamphlet for recording appointment dates. Following the verification call, a thank you card was sent out to the primary mailing address provided by clients as a token of appreciation for their participation in the study and to verify their address. In the case of an email address having been provided, an e copy of the thank you card was also sent to the client. The follow up team scheduled the subsequent three and six month interviews on the telephone with clients after the completion of the baseline interview (i.e., GAIN Q3 MI). The team also sent out greeting cards to clients reminding them of the upcoming interview appointments six weeks prior to their follow up interview dates and called or emailed them a day or two before the appointments. About a third of the recruited clients did not go on to complete a baseline assessment during the study. In these cases, the follow up team still tried to maintain 42

contact with these clients and complete the Return to Treatment Protocol quarterly after recruitment to facilitate connection to referral services. Three and Six month Follow Up The three and six months follow up interviews were scheduled up to one month before or after the 90 and 180 day mark since the baseline interview. This was in accordance with the recommended data collection window of the GAIN Q3 tools. Clients, who were not reached for their first scheduled follow up appointment on the telephone, would be placed in the tracking list of the follow up team. Once the client was in tracking, the follow up team focused its effort in reaching the client using their primary telephone and email contact information within the first two days of a missed scheduled appointment. After the two days, the calls would expand to other contacts listed in the Locator Form. For some clients, who were difficult to reach, the interviews were done up to two months after the follow up due date. This was still within the accepted data collection window of the GAIN Q3. Any subsequent follow up interview dates were adjusted based on the completion date of the latest interview. Follow Up Team Training In addition to receiving training on the GAIN Q3 as mentioned in an earlier section (4.1.3), the team was also trained on follow up procedures by Dr. Christy Scott. Dr. Christy Scott, who specializes in recovery monitoring, supported the Recovery Monitoring Project as a consultant and delivered a 3 day training session to the followup team staff before the launch of the project. The content of the training session included: generalizability and effectiveness of the follow up model, impact of different follow up rates on client outcomes, strategies for negotiating agency access, protocol components: engagement, verification, maintenance and confirmation, and follow up process. The training provided the followup team opportunities to review and practice scripts used for different types of calls in 43

the follow up process. These scripts were developed based on the ones used by Dr. Christy Scott s team at the Lighthouse Institute and were revised subsequently to better fit the context of Ontario. The project team also invited the manager of the Back on Track Program, Rita Thomas to deliver a half day phone etiquette and communication techniques training to the follow up team. Monitoring and Tracking of Follow Up All the follow up activities were documented in a centralized database created in Microsoft Access. This database kept track of all sending and receiving activities of study documents, such as the consent forms, Locator Forms, and assessment reports. Locating information was also entered into the database to facilitate information sharing among the follow up team. The details of all the calls made to clients were logged into the database. Appointment dates and daily tasks such as sending out reminder cards and making reminder calls were all entered and organized in a master calendar. The research coordinator monitored all of the data collected by follow up workers. This included checking the quality of locator information (i.e., completeness of records), the call and study document logs of follow up workers, and follow up rates. The team had weekly meetings to discuss cases in tracking and case assignments in the following week. The coordinator also examined GAIN Q3 assessment validity reports, and conducted audits on selected files periodically to ensure that procedures were being followed appropriately. Privacy and Confidentiality The follow up team followed strict procedures established by the various ethics boards that reviewed the project; the procedures respected the privacy and 44

confidentiality of our clients. All team members completed and received certificates for the Tri Council Policy Statement 2 (TCPS 2) and Good Clinical Practice (GCP) training. Study documents were only delivered through confidential posts or a password protected fax line, to which the follow up team had sole access. The central database in Microsoft Access, the master calendar, and study documents delivered between sites were all password protected. The paper client files, which contained the consent forms, Locator Forms, and other study documents, were stored in a locked cabinet. Only the follow up team had access to the key. The team also strictly followed scripts on the phone and used only the study alias when talking with other contacts of clients, such as family and friends. A private P.O. Box for returned mail and a private email address for all outgoing email messages were also set up to prevent divulging the nature of the study. Before any follow up assessment or communication related to study data was discussed with the client, the team would go through an identity verification process with the client. Self Care Plan The clients in our study often had experienced severe trauma and/or were part of a marginalized community. The follow up assessments sometimes triggered these traumatic memories and caused frustrations, sadness, and even anger for these clients during an interview. The follow up team recognized the challenge of remaining compassionate and professional while managing the emotional turbulence shown by the participants during an assessment. As a mitigating measure of compassion fatigue, the follow up team participated in a two day workshop on reducing compassion fatigue, burnout and traumatic stress in caregivers. The workshop raised awareness of compassion fatigue among the follow up workers, emphasized the importance of self care, and provided concrete guidance on 45

developing a self care plan. The team also had regular confidential debrief meetings to talk about any interaction with difficult clients or other contacts of clients. These meetings provided a safe and comfortable platform for workers to ventilate their frustrations and to draw support from their network. 4.2.2 Changes to the protocol In the original protocol, we had planned to use community trackers to find clients in the community in the event that the follow up team was unable to locate individuals over the telephone. This outreach component has been shown to be quite effective in increasing follow up rates in the U.S. (Scott, 2004). However, it was not piloted due to challenges in recruiting such trackers as project staff or contracting with local community agencies. The other major hurdle was the lack of information that would be available to the community trackers. Specifically, we did not have photos of clients recruited into the study and we had limited information available through publicly accessible databases (e.g., Nexus/Lexus, credit reporting bureaus, etc.). Similarly, we did not pilot the Request for Locating Information Form, which asks for clients permission to contact relevant organizations (e.g., homeless shelters, law enforcement agencies, etc.) to release information pertaining to their possible whereabouts to the research team. The components that were dropped from our study are key aspects of Dr. Christy Scott s follow up model in the U.S.; however, we were unable to implement them in Ontario given time constraints and important differences in privacy regulations, cultural context, and agency buy in. Feedback from the pilot sites and the Program Advisory Committee members were particularly helpful in this regard. Additionally, Dr. Christy Scott s follow up model included remuneration for clients, engagements calls with each individual recruited into the study, and an option to complete the follow up interviews in person. In our pilot project, clients were not reimbursed for their participation (see section 2.7); engagement calls were completed only if sufficient time was available; and follow up interviews were done over the 46

telephone due to resource limitations. In general, these key differences challenged us in achieving the follow up rates reported by Scott (2004), typically over 90%. 5.0 EVALUATION STRATEGY The main goal of the evaluation strategy was to collect feedback from all participating staff at the pilot sites and other key stakeholders in order to assess the feasibility and utility of our tools and procedures and to determine strengths and limitations or issues going forward in terms of eventual province wide implementation. The particular evaluation questions and indicators of success that guided this process are outlined in Table 3. Informal feedback was provided regularly by the study leads, GAIN Trainers and Executive Directors via email and/or telephone. Relevant comments were collected and maintained in a Microsoft Excel file that was monitored by a member of the research team. Front line clinical staff also provided feedback using a logbook in which they responded to three open ended questions, typically on a weeklybasis during the pilot phase. The purpose was to obtain the clinicians and their clients (indirect) perceptions of the project tools and procedures. Feedback from the follow up team was also collected using logbooks consisting of similar types of questions. Structured feedback was collected from all agency staff involved in the study and the follow up team via an online questionnaire, which consisted of approximately 60 questions, including items pertinent to the GAIN Q3 MI, GAIN ABS, OWTOM A, and follow up (GAIN Q3 Standard and RTT protocol). Of 48 invitations sent to staff at the five pilot sites, 38 submitted a completed questionnaire (79% response rate). Skips were built into the online software program (Fluid Surveys, www.fluidsurveys.com), thus respondents were rarely required to answer all of the questions. For example, if a respondent had not administered the OWTOM A, then they would skip all of the related questions in that section. The questionnaire was administered to pilot sites during three weeks in November, 2012; however, staff at Rideauwood completed the online 47

questionnaire in January 2013, given that recruitment at this site had just wrapped up in December, 2012. In addition, the project lead, project scientist, and program manager visited all of the pilot sites in January and February, 2013 to conduct focus groups. During the oneday visit, a summary of the findings was presented to all of the staff at the sites. Specifically, updates were provided on the numbers of clients recruited and tools completed. The analysis of study feedback was also presented and preliminary results were validated by asking staff members to comment on the degree to which the findings matched their own experiences. Furthermore, the proposed revisions to the GAIN tools and preliminary recommendations to the MoHLTC were discussed. For the final component of this visit, key informant interviews were held with the study leads and Executive Directors. Unstructured and open ended questions related to the next phase of piloting of recovery monitoring procedures were used to guide the interviews. Specific topic areas included the perceived value add of the OWTOM A; alternative modes for conducting follow up interviews; sharing of follow up results with treatment agencies; and the addition of community trackers to locate clients we were unable to reach via telephone. All discussions from the pilot site visits were audio taped and later transcribed. Additional feedback by email or telephone was also encouraged after these de briefing sessions at the pilot agencies. The project team also requested a meeting with a standing committee comprised of Mental Health and Addiction Leads from each of Ontario s Local Health Integration Networks (LHINs). The project lead presented the goals and work completed to date for three of the provincial DTFP projects, including the screening and assessment project, as well as key implications for the addiction system based on preliminary pilot results. Committee members were asked to discuss these implications further with LHIN colleagues who were unable to attend the meeting and to provide a summary of feedback at a later date. 48

Finally, this project served as a case study for the separate evaluation of the Ontario DTFP Systems Enhancement Program, conducted by the Evaluation Centre for Complex Health Interventions from St. Michael s Hospital in Toronto. As part of this case study, this larger DTFP evaluation team conducted site visits and interviews with Executive Directors from two of the project s pilot sites (the Addictions Centre and ADSTV) and requested feedback regarding the tools and processes piloted, as well as regarding the support provided by the research team. Feedback from this process was used to triangulate the results from all evaluation strategies described above. Table 3. Evaluation questions and indicators of success Objective To examine the feasibility of implementing common withintreatment outcome measures and the potential usefulness of the results among decision makers at the clinical, program and system planning levels Evaluation Questions What are the perceived benefits and costs of the within treatment recovery monitoring process? What are the key lessons learned regarding withintreatment recovery monitoring that impact scalability and sustainability? Data collection Strategy and Indicators of Success Technical indicators: Clinician fidelity to the withintreatment recovery monitoring component completion of each screening tool Changes in the within treatment recovery measure over the course of care Qualitative indicators of success: Perceived value add of the within treatment recovery monitoring staged approach to screening Positive feedback on adaptability of tools for specific programs, services contexts and subpopulations (e.g., group versus individual work; mandated versus voluntary clients) Positive feedback on utility of the information for quality improvement and accountability 49

Feedback gathered via interviews or focus groups of (a) pilot site clinicians and clinical directors; (b) program managers including managers of agencies receiving referrals and forwarded assessment information. Key output: Specifications for provincial implementation and sustainability of within treatment recovery monitoring: Leadership and sector support Training requirements and cost IT infrastructure and cost Fidelity assessment To assess the feasibility of a comprehensive system of client recovery monitoring and the potential use of the results among decision makers at the clinical, program and system planning levels What are the perceived benefits and costs of the recovery monitoring process? What are the key lessons learned regarding recovery monitoring that impact scalability and sustainability? Technical indicators: % of clients agreeing to the follow up procedures (without incentive) and reasons for refusal % of clients recruited that are located Incremental % increase in clients located via different elements of the follow up protocol (e.g., telephone) Number and percentage of clients returned to treatment via the follow up protocol Sensitivity of the measures to change over time three and six month changes in the followup measures over the two follow up interviews o individual GAIN Q3 subscales o multi morbidity index across the sub scales 50

Qualitative indicators of success: Perceived value add of the recovery monitoring approach and resulting information for clinical work, quality improvement and accountability Feedback gathered via interviews or focus groups of (a) pilot site clinicians and clinical directors; (b) program managers including managers of agencies receiving referrals and forwarded assessment information Key output: Specifications for provincial implementation and sustainability of recovery monitoring: Leadership and sector support Training requirements and cost IT infrastructure and cost Fidelity assessment 5.1 QUALITATIVE DATA ANALYSIS To support achievement of the pilot objectives (pg 18) the overall qualitative analysis of pilot data was guided by three key questions: 1. What are the perceived strengths and challenges of the actual administration/utilization of the new suite of tools? 2. What are the perceived value add or perceived value loss of the new suite of tools over current practice? 3. What are the strengths and limitations/issues going forward in terms of wider implementation of the new suite of tools? Qualitative data from the different sources of formal data collection tools (e.g., online survey, logbooks, focus groups, feedback from LHINS) were first analyzed separately as distinct sets of data. All data from one source were exported into 51

Microsoft Excel files and were organized first into broad categories (e.g., type of tool, feedback regarding the within treatment tool). Then, within each category, the analyst reviewed all responses and identified a provisional set of qualitatively distinct points and assigned numeric codes with corresponding labels. Where a number of points (and corresponding numeric codes) clustered within a given thematic area, overarching thematic categories were assigned. Thematic categories across all sources of data were then grouped and analyzed to validate the findings and to identify any qualitative outliers (i.e., feedback that was not consistent with any particular theme). Finally, informal feedback collected during the pilot site visits were also consulted to determine whether there were any discrepancies with the identified themes. 5.2 QUANTITATIVE DATA ANALYSIS AND DESCRIPTION OF STUDY SAMPLE The screening process for the study is outlined in Figure 2 below. Out of the 773 clients approached to participate in the study, 292 were not interested in continuing after the introduction of the study. The main reasons for the lack of interest in the study included the extra time commitments associated with the new assessment process introduced by the study 10, attending services for legal reasons only, and not presenting for substance use services. The remaining 481 clients were screened for the study based on the inclusion and exclusion criteria. Two hundred and forty seven of these clients were excluded from the study: 44 did not present for services for a substance use problem at the pilot agencies; 6 were not proficient in English; 37 were not clinically stable at the time of recruitment; 2 were cognitively impaired; and 160 did not consent to the study. The additional assessment time required by the study process and the perception that it would divert focus from treatment were the main reasons clients declined to participate in the study during the consent process. In the end of the screening process, 234 clients were enrolled into the study. The breakdown by site is as follows: 10 Participants were required to complete the standard process (ADAT) plus the new tools being piloted in the project. 52

170 from Addiction Services of Thames Valley (London) 47 from Fourcast (Peterborough) 6 from Manitoulin Community Withdrawal Management Services (Little Current) 4 from Addictions Centre (Belleville) 7 from Rideauwood Addiction and Family Services (Ottawa) Figure 2. An Overview of the Recruitment Process 53

The research team requested basic socio demographic information for the 234 clients from DATIS; however, data were available only for 217 of these clients. The other 17 clients did not have any admission program information available in DATIS. As shown previously, the number of participants from Manitoulin Community Withdrawal Management Services, Addictions Centre, and the Rideauwood Addiction and Family Services was very small. In addition, each of these three sites represented a special client population within Ontario addiction services: Aboriginal peoples, youth and mandated treatment populations. After careful consideration, the research team decided to analyze the data from these three sites independent from the rest of the study sample to highlight the uniqueness of their client populations. All the analyses conducted for this report are based on a final sample containing 200 clients (from ADSTV and Fourcast). The representativeness of our study sample was determined by comparing them on key characteristics to the substance use treatment population across all publiclyfunded agencies in Ontario (see Table 4). Pertinent data were obtained from DATIS. Information from residential treatment programs were excluded as only non residential programs had participated in our study. The sample of 200 clients was compared to two populations: (1) all clients who presented for community substance use services at Addiction Services of Thames Valley (London) and at Fourcast (Peterborough); and (2) all clients who presented for community substance use services in all the publicly funded, non residential addiction agencies in Ontario. Chi square statistical tests were conducted to compare the distributions of socio demographic characteristics of the study sample to the two populations (Table 4) to examine the generalizability of our findings. Descriptive statistics were also generated for the administration times (Table 5) of all the tools used in this study. Changes in administration time over the course of the study were analyzed using the independent sample and pair sample t tests. Among the 54

200 clients, 148 completed a baseline tool (GAIN Q3 MI). The remaining 52 clients did not complete a baseline tool, mainly due to missing several appointments following study recruitment. Follow up rates were calculated separately for clients who had completed a baseline tool and those who had not. Furthermore, chi square statistical tests were conducted to compare the characteristics of clients who completed at least part of both their three and six month follow up interviews to those who did not (Table 6). In total, 16 clients withdrew from the study at various stages. Most commonly, clients were in a state of crisis, and were thus unable to continue with the study. Some indicated that their needs were already being met at the agency; as such, they were no longer interested in participating in the study. Less frequent reasons for withdrawal were related to the fact that some clients had moved out of the province and in a few cases, there were difficulties in finding a convenient time to talk on the phone. For analyses, these clients were included in the group who missed at least one interview during the follow up period. 6.0 FINDINGS AND INTERPRETATIONS 6.1 FEASIBILITY AND IMPLEMENTATION ISSUES: QUANTITATIVE RESULTS 6.1.1 Representativeness of sample Compared to the total treatment populations from ADSTV and Fourcast (see Table 4), the study sample was representative in distributions of many key characteristics, such as gender, preferred language, housing, employment, presenting substances at admission, frequency of substance use, and mental health. However, younger clients were somewhat under represented in our sample: 27% were younger than 25 years old in the treatment population, whereas only 16% were less than 25 in our sample (χ 2 (4)=13.49, p<0.05). Clients in our sample also tended to have higher education levels (χ 2 (3)=23.21, p<0.001) and were less likely to be involved in the court 55

system. More specifically, the proportion of clients who had no legal problems were 60% in the treatment population and 67% in the sample (χ 2 (3)=9.75, p<0.05). This was also reflected in the fact that presenting for services solely for court reasons was one of the main reasons for declining study participation. Compared to the treatment population across all Ontario publicly funded community treatment services (see Table 4), our study sample was representative in terms of gender, age group, employment and mental health. However, our study clients were more likely to be in a stable relationship, have at least a high school education, and be housed (fixed address). All our study clients preferred receiving services in English, whereas 5% of all clients in the treatment system preferred receiving services in languages other than English. This finding is, however, not surprising as only those who were able to speak or understand English were eligible to participate in our study. In terms of substance use, over 40% of our study sample sought services related to a drinking problem only. This was significantly higher than the 30% observed in the treatment population (χ 2 (2)=18.28, p<0.001). In addition, daily substance users represented 61% of the treatment population, but only 45% of the sample (χ 2 (4)=21.61, p<0.001). 56

Table 4. Comparison of Study Sample to Treatment Population in Ontario Treatment population in Ontario N=12858 Treatment population in London and Peterborough Sites N=744 Study Sample n=200 Gender (n) % X 2 (n) % X 2 (n) % Statistics Statistics Female 4888 38.04% 0.09 280 37.63% 0.04 74 37.00% Male 7963 61.96% 464 62.37% 126 63.00% Age categories <= 24 years 3077 23.93% 7.42 204 27.42% 13.49* 32 16.00% 25 34 years 3542 27.55% 194 26.08% 63 31.50% 35 44 years 2654 20.64% 141 18.95% 45 22.50% 45 54 years 2337 18.18% 127 17.07% 37 18.50% 55 + years 1248 9.71% 78 10.48% 23 11.50% Preferred Language English 12317 95.9% <0.001 1 743 99.9% 0.816 1 200 100.0% Other 531 4.1% 1 0.1% 0 0.0% Relationship status Married/partnered/ common law 3257 25.7% 7.32* 196 26.5% 5.94 66 33.5% Single (never married) 7066 55.8% 407 55.1% 93 47.2% Separated or divorced/ widowed Employment status 2330 18.4% 136 18.4% 38 19.3% Employed full time 3222 25.5% 7.91 182 24.6% 4.72 57 28.5% Employed part time 847 6.7% 79 10.7% 22 11.0% Unemployed (looking 4645 36.8% 230 31.1% 68 34.0% for work) Other 3908 31.0% 248 33.6% 53 26.3% Education < High School 4775 38.6% 15.17** 286 38.8% 23.21*** 51 25.5% Completed secondary 3023 24.4% 231 31.3% 64 32.0% or High School Some post secondary 1888 15.2% 73 9.9% 35 17.5% Completed College or University Legal status 2699 21.8% 148 20.1% 50 25.0% No problem 8053 63.9% 6.79 444 60.1% 9.75* 137 68.5% Awaiting trial or 1645 13.1% 106 14.3% 31 15.5% sentencing Probation 2195 17.4% 158 21.4% 27 13.5% Other 703 5.6% 31 4.2% 5 2.5% 57

Fixed address (postal code) Treatment population in Ontario N=12858 (n) % X 2 Statistics Treatment population in London and Peterborough Sites N=744 (n) % X 2 Statistics Study Sample n=200 (n) % NFA 874 7.0% 3.60* 22 3.0% 0.17 7 3.5% Fixed address 11628 93.0% 709 97.0% 191 96.5% Presenting problem substance Alcohol only 3536 30.4% 18.28*** 287 40.3% 3.83 83 42.3% Other substance(s) only + no alcohol Alcohol and other substance(s) Frequency of substance use 3592 30.9% 274 38.4% 63 32.1% 4515 38.8% 152 21.3% 50 25.5% 1 3 times monthly 784 7.3% 21.61*** 50 9.1% 5.54 10 6.4% 1 2 times weekly 959 9.0% 77 14.0% 24 15.3% 3 6 times weekly 1484 13.9% 97 17.7% 33 21.0% Daily 6478 60.6% 225 41.0% 70 44.6% Binge 985 9.2% 100 18.2% 20 12.7% Mental health diagnosis in the past 12 months No 9699 78.7% 1.74 531 73.2% 0.30 147 75.0% Yes 2624 21.3% 194 26.8% 49 25.0% Receiving mental health support currently No 9121 73.6% 0.19 532 72.8% 0.03 143 72.2% Yes 3274 26.4% 199 27.2% 55 27.8% Source of income Employment 3600 28.7% 37.34*** 227 30.7% 16.69** 66 33.0% EI 507 4.0% 42 5.7% 16 8.0% ODSP 1788 14.3% 82 11.1% 15 7.5% Insurance 422 3.4% 42 5.7% 12 6.0% Ontario Works 2734 21.8% 166 22.4% 60 30.0% Other 2219 17.7% 113 15.3% 17 8.5% None 1265 10.1% 68 9.2% 14 7.0% Treatment mandated None 8674 68.6% 1.87 445 60.5% 12.92** 145 72.9% Legal 1863 14.7% 153 20.8% 27 13.6% Other 2110 16.7% 138 18.8% 27 13.6% * (p<0.05), ** (p<0.01), *** (p<0.001) 1 p value based on Z statistics 58

In summary, our results indicate that our sample agreeing to participate is reasonably representative of the demographics of clients at the main participating sites and the overall Ontario substance use treatment system. There is a general trend, however, for the clients engaged in the project to be somewhat more stable, for example, compared to other clients in the participating agencies, those consenting were older and somewhat less likely to have legal problems, Compared to the overall treatment population in community treatment services our participants tended to: be married/partnered, have at least a high school degree, and present only with an alcohol use problem (i.e., less involvement of other drugs). Clients in our sample were also less likely to use substances on a daily basis. These results are not unexpected, as our protocol was quite time consuming and burdensome (as with any research study). More discussion regarding this issue is provided in the later sections. A description of recruited clients from the remaining three pilot sites in the study is provided in the following sections. Addictions Centre (Belleville) Four clients were recruited from Addictions Centre in Belleville. Three of these clients were male, under 35 of years of age and on probation. Two clients presented for services for non alcohol substances, one for alcohol only, and the other for alcohol and other substances. Manitoulin Community Withdrawal Management Services (Little Current) Six clients were recruited from Manitoulin Community Withdrawal Management Services. Five of these clients were female and four identified as Aboriginal. Although all six clients were above 20 years of age, only one had completed high school. Two clients presented for services for non alcohol substances, two for alcohol only, and the other two for alcohol and other substances. 59

Rideauwood Addiction and Family Services (Ottawa) There were seven clients recruited from the School based Substance Abuse and the Teen Substance Abuse programs at Rideauwood Addiction and Family Services. Five of these clients completed a Locator Form and were included in the follow up process. All of these clients were under 24 years of age and did not complete high school. They all presented for services for alcohol and other substances. Two of them were on probation, and three of the five clients were female. 6.1.2 Administration of Tools The administration times for all of the tools are listed in Table 5. On average, the OWTOM A was completed less than four minutes. The majority of the clients (81%) selfadministered the tool. Eighty seven of the 200 clients in the sample completed at least one OWTOM A. Nineteen completed the tool twice; 21 completed the OWTOM A three times; and 22 completed the tool four times. The follow up team completed 172 GAIN Q3 Standard administrations. This included computer and paper and pencil administrations at both the three and sixmonth follow up periods. Of these 172 administrations, 140 were completed electronically. This is the preferred mode as ABS automates navigation between the survey sections based on conditional responses, and it includes a data validation process during administration. On average, it took 44 minutes to complete a follow up administration via ABS. The other 32 administrations were done using paper and pencil. This was reported by the follow up team to save time when compared to electronic administrations as time it took to load the screen for the latter was often quite lengthy. Thus, paper and pencil administrations helped to minimize wait times between questions, enhanced the flow of administration and allowed for shorter times for completion overall. Nonetheless, the amount of time spent on data entry after paperand pencil administration offset the time savings for the project team (but not the respondent). 60

Table 5. Completion Times for Recovery Monitoring Tools Instrument (No. administered) Locator Form (n=189) Global Appraisal of Individual Needs Quick3 (GAIN Q3) MI (n=128) (baseline) Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) (n=205) 2 Global Appraisal of Individual Needs Quick3 (GAIN Q3) Standard at 3 months (n=85) Global Appraisal of Individual Needs Quick3 (GAIN Q3) Standard at 6 months (n=55) Return to Treatment (RTT) Protocol (n=226) 3 Avg. Time to Complete (min) 1 15.2 mins (min=3.0, max=80.0) 60.5 mins (min=5.0, max=140.0) mean=3.7 mins (min=1, max=38) 44.2 mins 4 (min=17.0, max=87.0) 37.2 mins 4 (min=21.0, max=67.0) 3.9 mins (min=1, max=16) 1 All means have been 5% trimmed. 2 This number reflects total number of administrations of the OWTOMAs. The same client can have multiple administrations. 3 This number reflects total number of administrations of the RTTs. The same client can have multiple RTTs. 4 Time calculated based on ABS administrations only. Administration times at three months were compared to those at six months; on average, we found that administration times at six months were significantly lower than at three months (t(138)=3.56, p<0.001). There are a couple of reasons that may explain the differences in administration times between the two follow up periods. First, the follow up team, with practice, was more familiar and comfortable with administering the tool at six months. Second, participants themselves may have been more familiar with the tool at six months compared to three months. The difference in completion time was further tested on the 65 participants who had completed both the three and six month follow up interviews via ABS. We found that the three month administration time was significantly longer than the completion time at six months (t(64)=6.44, p<0.001). 61

The follow up team completed 226 Return to Treatment (RTT) protocols. On average, the RTT took about four minutes to complete. At three months, 90 (74%) of the 121 clients who completed an RTT reported that they were still receiving services at their recruiting agency. Eighty (89%) of these clients who were still receiving services found treatment helpful and 67 (74%) either indicated a need for additional services or had thoughts about getting additional support. Among those who were no longer at the agency at three months, 14 (45%) thought the treatment they had received was helpful and 16 (52%) indicated a need for additional services or had thought about getting additional support. Furthermore, less than half of the clients who were no longer at the recruiting agency but had expressed a need for services indicated a preference for being connected back to their recruiting agencies. The most common reason for not wanting to return to the recruiting agency was the clients perceptions that their needs were not being met. Some had found other services and other preferred support from their network of friends and family. At six months, 62 (59%) of the 105 clients who completed an RTT reported that they were still receiving services at their recruiting agency. Of these clients who were still receiving services, 58 (93%) found treatment helpful and 38 (61%) either indicated a need for additional services or had thought about getting additional support. Among those who were no longer at the agency at six months, 32 (43%) thought the treatment they had received was helpful and 19 (44%) indicated a need for additional services or had thought about getting additional help. Similar to the clients completing a three month RTT Protocol, less than half of those who were no longer at their recruiting agency, but expressed an interest in further services indicated a preference to return to their recruiting agency for services. The reasons reported for not wanting to return to their recruiting agency were the same as those reported for three months listed above. 62

6.1.3 Follow Up Rates In order to effectively examine changes in client functioning and outcomes, a standardized questionnaire was administered both at baseline and during follow up interviews. In our pilot project, a number of participants (n= 52) did not complete the GAIN Q3 MI, that is the baseline tool. The main reasons were related to a lack of time or clients not showing up for their appointments. While changes in client outcomes cannot be assessed for these clients, we were still able to contact many of them at threemonths in order to administer the Return to Treatment protocol (see Flow Chart for more details). Nonetheless, in comparing clients who had completed the GAIN Q3 MI to those who had not (Table 6), we found that clients who had completed a baseline were older, more likely to be married and employed. They were also more likely to present for services for alcohol only. More details on this are available in the DTFP Best Practice Screening and Assessment Project. Figure 3A and Figure 3B outline the basic follow up process with clients and the follow up rates at three and six months. Of the 200 clients in our study sample, 148 of them completed a baseline assessment. The follow up team verified locating information with 90% of these 148 clients. The team also administered the follow up tool to 99 clients at three months, which was close to 70% of those who completed a baseline assessment. As shown in the right hand column of Figure 3A about 44% of the clients who did not complete a baseline assessment were located and completed a Return to Treatment Protocol at three months. At six months (Figure 3B), the follow up team was able to complete a follow up interview with 89 clients out of 129 (69%) who had a baseline interview and who were due for a six month interview (19 were not scheduled for a six month interview as they had been recruited late in the study period). In comparison, only 43% of the clients with no baseline interview were located in the same period. Combining the three and sixmonths data, the follow up team was able to complete a follow up interview at either 63

time period with 102 clients, which was about 80% of all 129 clients. Seventy two of 129 clients completed interviews at both time points (56%). These clients were later on compared with the clients who did not complete interviews at both time points. The findings are outlined in Table 6. The follow up rates in the remaining three pilot sites are discussed below. Addictions Centre (Belleville) Two of the four recruited clients completed a baseline assessment. The followup team was able to verify locating information with one of them, but was unable to reach any of the clients at the three or six month follow up periods. Manitoulin Community Withdrawal Management Services (Little Current) Five of the six recruited clients completed a baseline assessment. The follow up team was able to verify locating information with four out of the five clients. At threemonths, the follow team completed a follow up assessment with all clients for whom locator information had been verified; however, we were unsuccessful in locating any of them at six months. Rideauwood Addiction and Family Services (Ottawa) Due to the late start in recruitment at this pilot site, only one client had a threemonth follow up period. The follow up team was able to complete the interview with this client, and the team also verified information with four of the five clients. 64

Figure 3A. Follow up data at 3 months 65

Figure3B Follow up data at 6 months 66

As mentioned earlier, all the follow up rates were calculated under the consideration of study withdrawals as loss to follow up. Given that many of these withdrawals took place before the clients follow up due dates, the follow up team did not attempt locating these clients. Therefore, the proportions of clients located among those the follow up team tried locating are higher than the follow up rates reported above. Compared to clients who did not complete a follow up assessment at both three and six months, clients who participated at both times were more likely to be older (i.e., at least 35 years of age, χ 2 (4)=15.07, p<0.01), married (χ 2 (2)=5.80, p=0.05), employed (χ 2 (3)=19.49, p<0.001), more educated (χ 2 (3)=10.66, p=0.01), and seeking services for drinking problems only (χ 2 (2)=10.75, p<0.01). Please see Table 6 for further details. This pattern is consistent with the one observed between the group of participants who completed a baseline assessment and those who did not, as mentioned earlier in the section. Preliminary multivariate analyses were also performed to examine the association between these characteristics when considered together and the success of completing a follow up assessment with clients. Results of the logistic regression (not shown) indicated that higher education level was independently associated with successfully completing a follow up interview. In particular, the odds of completing a follow up assessment with a client was three times higher for those who had a post secondary degree compared to those who had not finished high school (Exp(b)=3.4, p=0.029). In addition, clients presenting for services for alcohol and other drug problems were more difficult to locate than those presenting only for alcohol problems (Exp(b)=0.284, p=0.005). 67

Table 6. Characteristics of clients (from London and Peterborough) who completed both 3 month and 6 month interviews VS those lost to follow up (among clients with GAIN Q3 MI) Gender Completed interview at both 3 and 6 months (n=72 1 ) Did not complete interview at both 3 and 6 months (n= 44) % (n) % (n) X 2 Statistics Female 38.89% 28 36.84% 21 0.057 Male 61.11% 44 63.16% 36 Age categories <= 24 years 8.33% 6 7.02% 4 15.07** 25 34 years 19.44% 14 47.37% 27 35 44 years 25.00% 18 26.32% 15 45 54 years 27.78% 20 12.28% 7 55 + years 19.44% 14 7.02% 4 Ethnic groups Canadian 97.2% 70 94.7% 54 0.53 Non Canadian 2.8% 2 5.3% 3 Relationship status Married/partnered/ common law 43.7% 31 34.5% 19 5.8 Single (never married) 32.4% 23 52.7% 29 Separated or divorced 23.9% 17 12.7% 7 Employment status Employed full time 38.9% 28 26.3% 15 4.31 Employed part time 12.5% 9 7.0% 4 Unemployed 23.6% 17 33.3% 19 Other 28.5% 18 20.0% 19 Education < High School 12.5% 9 35.1% 20 12.42** Completed secondary or High School 34.7% 25 29.8% 17 Some post secondary 16.7% 12 19.3% 11 Completed College or University Legal status 36.1% 26 15.8% 9 68

Completed interview at both 3 and 6 months (n=72 1 ) Did not complete interview at both 3 and 6 months (n= 44) % (n) % (n) X 2 Statistics No problem 70.8% 51 64.9% 37 5.9 Awaiting trial or sentencing 19.4% 14 15.8% 9 Probation 6.9% 5 19.3% 11 Other 2.8% 2 0.0% 0 Fixed address (postal code) NFA 1.4% 1 5.3% 3 1.51 Fixed address 98.6% 69 94.7% 54 Presenting problem substance Alcohol only 62.0% 44 32.7% 18 13.23*** Other substance(s) only + no alcohol Alcohol and other substance(s) Frequency of substance use 16.9% 12 43.6% 24 21.1% 15 23.6% 13 Did not use 21.1% 15 22.2% 12 1.56 1 3 times monthly 4.2% 3 7.4% 4 1 2 times weekly 14.1% 10 11.1% 6 3 6 times weekly 16.9% 12 18.5% 10 Daily 33.8% 24 35.2% 19 Binge 9.9% 7 5.6% 3 Mental health diagnosis in the past 12 months No 78.3% 54 73.2% 41 0.43 Yes 21.7% 15 26.8% 15 Receiving mental health support currently No 69.0% 49 75.0% 42 0.55 Yes 31.0% 22 25.0% 14 Source of income Employment 44.4% 32 28.1% 16 19.49*** ODSP 2.8% 2 14.0% 8 Ontario Works 13.9% 10 38.6% 22 None 8.3% 6 5.3% 3 Other 30.6% 22 14.0% 8 69

Treatment mandated Completed interview at both 3 and 6 months (n=72 1 ) Did not complete interview at both 3 and 6 months (n= 44) % (n) % (n) X 2 Statistics None 75.0% 54 75.0% 42 1.05 Legal 9.7% 7 14.3% 8 Other 15.3% 11 10.7% 6 * (p<0.05), **(p<0.01), ***(p<0.001) 1 This number includes 2 partially completed GAIN Q3 s. 6.1.4 Qualitative Feedback The qualitative feedback from the online questionnaire, logbooks and follow up team revealed important findings related to the recovery monitoring tools and procedures. Where relevant, results from the focus groups are also presented to highlight key discrepancies or areas of consensus, and to expand on the main findings. Feedback from our pilot sites were also collected by the Evaluation Centre for Complex Health Interventions (led by Dr. Sanjeev Sridharan); pertinent results are provided below. In the below sections, respondents refer to the agency staff who completed the online questionnaire; 38 agency staff responded to the questionnaire in total. OWTOM A Perceived strengths and value add All but one respondent noted the ability to monitor progress over time (and one logbook entry), for both the clinician and the client, as a particular strength of the OWTOM A. Two respondents valued the ability to monitor satisfaction with services/strength of the therapeutic relationship. Two respondents valued the fact that the OWTOM A is quick/easy to administer and one noted that the OWTOM A allows the client to assess their commitment to recovery. Three respondents reported that the OWTOM A facilitated the clinical process in different ways: by helping to engage clients in the process, by identifying progress and the need for more information and services, and by identifying strategies and goals in relation to 70

concerns uncovered by the OWTOM A. Another respondent noted that the OWTOM A encouraged a client centred approach by checking in from their perspective regarding progress. During the focus groups, one large agency stated that they would be continuing to use the OWTOM A beyond the pilot period, as it was helpful for guiding the treatment sessions with clients. The same agency also indicated that monitoring client progress within treatment was useful from a clinical and program planning perspective. Finally, Dr. Sridharan s evaluation indicated that staff members who had completed an OWTOM A with clients appreciated the information it provided, particularly with respect to being able to see changes in client responses at a glance, and being able to share this information with clients. Perceived challenges Two respondents, noted some challenges related to administering the OWTOM A within the timeframes directed in the study protocol. Two also voiced concerns that clients may not feel comfortable being candid regarding their therapeutic relationship (this was echoed in four logbook entries; clinicians questioned whether the OWTOM A should be self administered). A similar concern was voiced by staff in one site during the focus group sessions. Additionally, in the online questionnaire we asked specifically about concerns related to the length of the OWTOM A; only one respondent suggested that more questions could be added to the tool. Three respondents noted that the OWTOM A did not facilitate the clinical process either because it was administered at the end of the session (1), because no new information was uncovered (1), or because there was no significant change noted between administrations (2). Locator Form Perceived strengths and value add Seven respondents commented that they saw utility in the Locator Form for clinical connections with clients, whether that be for following, ensuring that clients receive the care they need, or reaching out to clients (specific examples of use include outreach programs, services to transient clients and supportive housing programs). One respondent liked the idea of it if used purposefully and one respondent noted that the form could be lengthened. During 71

the focus groups, one site indicated that they would be continuing to use the Locator Form as part of their intake procedures. Staff noted its usefulness in terms of facilitating contact with vulnerable or high risk clients who drop out of treatment. Perceived challenges In terms of challenges, three respondents noted some duplication with protocols already in place to collect this type of locator information. One respondent noted feedback from a client that the title of the form should be changed to make it less invasive. In addition, four logbook entries pertained to clients being frustrated about the length and detail of the information requested. In addition, Dr. Sridharan s evaluation indicated that staff at some agencies felt their clients were concerned about privacy and confidentiality. Finally, staff at one agency suggested revising the name of the form to Reconnection Form. This was based on a concern that Locator Form may have been a little alienating to some clients who were already hesitant about providing personal information to agencies. GAIN Q3 Standard Perceived strengths and challenges In response to the question on how data from the GAIN Q3 Standard being employed for follow up purposes was used, only one example was provided: explore recommendations from the GAIN Q3 Standard in comparison to client s identified treatment goals. However, many concerns were raised by clients during the follow up GAIN Q3 assessments. Among the most notable concerns were the length of the instrument, questions that were not applicable, phrasing of sensitive topics, the instrument not being personalized, and the repetitiveness of questions. Administration of assessments over the phone was an added financial burden to clients, especially when follow up interviews were an hour long. Clients expressed difficulty accessing phones, paying for minutes on cell phones, and privacy concerns with respect to using household lines. Finally, the majority of the survey respondents from the pilot agencies reported not using the results shared from the follow up assessment. However, this was likely 72

due to technical issues with ABS, which resulted in delays to the automated report generating function; the issues have now been resolved. Return to Treatment (RTT) protocol Perceived strengths and challenges We received limited feedback on the RTT protocol from survey respondents; nonetheless, one participant felt that the information provided was duplicative of discussions already held with the client while receiving services. On the other hand, the strengths and value add of our approach were extolled by the follow up team. Members stated that the follow up process appeared to serve as an opportunity for clients to discuss their progress, recognize changes in their behaviour, and allowed clients to be re integrated into addictions services if further treatment was required. The follow up team reminded and encouraged clients to make and keep appointments with addictions treatment agencies, which facilitated communication between clients and treatment agencies. Some clients expressed appreciation of the efforts of the follow up team in keeping them connected to available services and motivating them to stay in treatment. These findings were supported by staff at the agencies, who indicated during focus groups that the follow up calls facilitated re engagement with their clients. Furthermore, staff felt that the RTT protocol was useful as it allowed them to learn why clients felt disconnected to the agencies or why they had decided not to go back for more services. General follow up procedures Perceived strengths and value add The follow up team provided the bulk of the feedback pertaining to the follow up process, given their extensive experience in administering the tools and having direct contact with clients. In general, the team indicated that the follow up process provided clients with a sense of ongoing support, which included additional approaches to ensuring successful treatment outcomes (e.g., thank you cards, reminder cards, and phone calls). Several clients indicated that the follow up process allowed them to feel connected and extend their support 73

network. Some clients also expressed gratitude and a sense of increased well being from receiving the cards and calls. Furthermore, several clients reported that participating in the follow up process provided a safe forum for them to provide feedback about services they had received and their experiences in the addiction system. Feedback provided by clients may be of interest to the participating agencies in the study as they may be able to monitor the quality of services and recognize areas for improvement. Perceived challenges During the focus groups, staff at one agency indicated that clients felt the six month follow up period was too intrusive; although it was recognized as being necessary to obtaining useful outcome data. In addition, Dr. Sridharan s evaluation highlighted that some staff were concerned about the challenges of implementing recovery monitoring processes in rural settings. Specifically, issues around isolation, and lack of access to telephones and health or social services more generally, were cited as possible impediments to the success of the followup protocol. Feedback on considerations going forward in terms of wider implementation During the focus groups, staff members were asked to comment on how, if at all, the community trackers could be incorporated into the next phase of the recovery monitoring project. One agency felt strongly that community tracking should be a function of the agencies rather than an external team (i.e., a FTE should be devoted to this role). Additionally, all of the agencies voiced concerns regarding the potential issues related to locating clients in the community. Specifically, this component was seen as being intrusive, and staff members were concerned about the impacts on confidentiality and potential issues with liability (i.e., potential risks of field work). In other respects, the recovery monitoring process was recognized by at least two pilot sites as an important indicator of the shift to a chronic or continuing care model; it was seen as changing the way we engage on a longer term (staff member from Peterborough). The agencies spoke about the potential for educating clients and the public about the chronic 74

nature of addictions, and thereby reducing associated stigma. Furthermore, most agencies agreed that monitoring outcomes within treatment has great value, and one site commented on the potential utility of incorporating the tool in an electronic interface (e.g., Catalyst), along with its reporting functions. Finally, the follow up team suggested that reimbursing client expenses to cover phone or travel costs, or providing access to a project phone would be helpful for increasing client engagement in the treatment process. Agencies also discussed exploring other modes of contact during follow up, including using mobile phones, being able to add/top up time to participants cell phone, texts and other mobile devices. 6.2 LESSONS LEARNED AND CHALLENGES There are several key lessons learned regarding recovery monitoring that may impact scalability and sustainability. We did not include a residential program in our pilot project. In Ontario, approximately 22% of all publicly funded addiction programs offer residential services, including residential withdrawal management (Pascoe, Rush, & Rotondi, under review). While our results may not be generalizeable to these specific programs, it is generally accepted that the within treatment monitoring of outcomes is most appropriate for clients receiving community treatment services (McLellan et al., 2005). In addition, all of our pilot sites were located outside of Metropolitan Toronto. This region is distinct from the rest of Ontario given its size (consisting of the largest population in the province), ethnic diversity, and availability of specialized addiction services. Our findings regarding the utility and feasibility of the recovery monitoring system may therefore not be generalizeable to Metropolitan Toronto. That said, we have no reason to believe that our protocol would be any less effective in Toronto. Our model was based on the work of Dr. Christy Scott (2004), who developed and refined her approach using participants from large, urban centres (e.g., Chicago). Another issue impacting scalability is the fact that our follow up process seemed to be most effective in locating relatively stable clients; i.e., those who were older, employed, married, and presenting for less complex substance use problems. Moving forward, we will need to explore strategies for engaging diverse clients with more severe substance use problems, and determine how well our protocol 75

works in locating different client populations. Concrete next steps are provided in Section 7.3 below. Feedback from the pilot sites also indicated that some staff and clients were concerned about maintaining privacy and confidentiality during the follow up process. Our project received approval from two research ethics boards, and we took all necessary precautions in order to protect the personal information of our participants. While this was conveyed to all involved, it is apparent that agency staff required additional training or information about the follow up process. In this way, we may have been able to dispel their remaining concerns around maintaining privacy and confidentiality, as well as those of their clients. An underlying issue is the fact that the team needed more time to engage with the pilot sites and agency staff. For many, this was the first time that they had heard of, or participated in, a recovery monitoring approach, thus building support, engagement and agency buy in required much more time and resources than we had available. Furthermore, in terms of the RTT protocol, the project coordinator would pass on the names of all clients, who were interested in returning to their recruiting agencies to the study leads immediately after the completion of the RTT. The study leads then called the clients to set up an appointment. Although this process was designed to connect clients back to services in a timely manner, it was challenging for the study leads to connect with the clients to schedule an appointment following the completion of the RTT. Without a dedicated resource for locating clients, such as the follow up team, study leads would themselves try to contact clients by phone a few times per day within the first week or two after receiving notice from the coordinator. If they were unsuccessful in reaching the client during this time, the chance of reconnecting was considered lost. Based on the follow up team s experience, clients who were disconnected from services were harder to reach. It is therefore important to take full advantage of the small window of opportunity for reconnecting clients directly to services while they are on the phone (e.g., make appointments for them, etc.). 76

Finally, all clients who completed a RTT were offered ConnexOntario s number or a chance to be connected to ConnexOntario directly. Only one person in the study agreed to be connected with ConnexOntario on the phone immediately following the interview. All other clients declined the offer due to the reluctance to repeat their stories again to a ConnexOntario agent after a 40 min follow up interview. Half of the clients made note of ConnexOntario s numbers for future reference, though a few clients reached at six months by the follow up team reported having used the numbers. Most clients who did not want ConnexOntario s number thought they had enough support at the time. An important consideration going forward is how we can maximize the use of ConnexOntario. On the other hand, we must explore whether there is even a need for this resource to be incorporated into the RTT protocol and if not, perhaps consider other sources of support and linkages to services. 7.0 RECOMMENDATIONS, POTENTIAL IMPACTS AND NEXT STEPS 7.1 RECOMMENDATIONS Our pilot project has demonstrated that recovery monitoring is feasible; specifically, about 90% of participants were contacted to verify locating information, 80% at either three or six month (70% were located at three months, close to 70% at six months), and 55% at both three and six months. Overall, agency staff, clients and the follow up team found great value in the recovery monitoring process. It was useful from a clinical perspective, provided more opportunities for building rapport with clients, and allowed for re engagement with the treatment system. Nonetheless, several issues were identified, and given the resource intensive nature of this work, we feel that our follow up protocol is not ready to be disseminated province wide. Specifically, we recommend more piloting so that we may refine the procedures and test components of the follow up model that were not incorporated during the initial phase of piloting. We are also concerned that the procedures did not successfully engage less stable, more severe and potentially more marginalized clients. Concrete and actionable next steps are outlined in section 7.3 below. The Project team, however, encourages uptake at the individual agency and LHIN levels if adequate resources and administrative support can be 77

provided. The results from these more local implementation efforts would then serve to support plans for broader implementation in other parts of the province. 7.2 POTENTIAL FOR IMPROVEMENTS IN MEASURING CLIENT OUTCOMES AND PERFORMANCE INDICATORS IN THE SUBSTANCE USE TREATMENT SYSTEM While more work is needed, our pilot study has demonstrated that there is immense potential for advancements in measuring client outcomes across Ontario s provincial substance use services. More importantly, our recovery monitoring model may eventually be able to provide answers to a key systems question: Is there evidence of improvement in clients receiving services from Ontario substance use treatment programs? Currently, there is no standard, structured or evidence based system in place to measure and monitor client outcomes in Ontario. A variety of tools are being used across the system, many of them homegrown and not suitable for tracking outcomes over time. Furthermore, there is currently no capacity to integrate, document or report on the client outcomes. Given these constraints, some agencies have turned to DATIS, which provides data on client characteristics and service use through a centralized information system. However, DATIS is an administrative system, which is not designed to provide valid and reliable measures of outcomes, or to assess change over time. The OWTOM A is the first such within treatment, quantitiative monitoring tool in Canada; if scaled up across Ontario, it will allow for standardized tracking of client outcomes over time. The tool is brief, ideal for situations where there is an ongoing relationship between the client and the clinician, and has been shown to greatly facilitate the clinical process. In the future, the OWTOM A may also be linked to DATIS/Catalyst, thereby providing additional information on client characteristics and treatment episodes. This will allow for the profiling of change over time (e.g., by sex, age, etc.), which would be of high value for system and program level improvements. 78

In the future, we can expect concrete follow up data on key outcome measures and performance indicators. A parallel DTFP project tested a new package of screening and assessment tools to replace the ADAT package and has recommended provincial implementation of the GAIN Q3 MI, the sister tool for the GAIN Q3 Standard tested here for follow up purposes. With the provincial implementation of the GAIN Q3 MI and the use of the GAIN Q3 Standard for follow up purposes with the infrastructure of the follow up team and staff for data analysis, a system would be in place to measure clinically significant change over time on a large scale in the Ontario substance use treatment system. This is important for determining the effectiveness of treatment interventions in several key domains of functioning (e.g., substance use, mental health, risk behaviours). Importantly we will be able to link outcome data to information on the cost of services if changes are made to current accounting and reporting systems as recommended in another of the provincial DTFP projects (DTFP Development and Implementation of a Province Wide Program to Assess and Benchmark the Cost of Addictions Treatment Services). We could also potentially link outcome data to the different types of treatment received by clients in order to ascertain which interventions work best and for whom and at what cost. We will also be able to compare effectiveness and costeffectiveness of innovative approaches to accessing and delivering treatment and benchmark/evaluate quality improvement activities on the basis of client outcomes. To determine the latter, we will need to profile change over time by examining the characteristics of the treatment population, using data from the GAIN Q3 Standard and by linking to DATIS. In this way, we may determine whether there is evidence of improvement in all clients, regardless of age, sex, ethnicity, region of residence, etc. Consequently, we will be able to support the development of targeted services that adequately address the needs of Ontarians. Of utmost value is the potential for an integrated, common and centralized recovery monitoring system in Ontario. The infrastructure has already begun to take shape, as evidenced by the Assessment Building System (ABS) developed for the province (with support from Chestnut Health Systems) and hosted in Catalyst. Our project has demonstrated the opportunities for utilizing and enhancing existing provincial resources. Ultimately, information 79

compiled at the LHIN level or the province as a whole will contribute to system level performance measurement and benchmarking. Specifically, outcome data from the GAIN Q3 Standard could be compared across jurisdictions in Ontario, and thereby allow for the evaluation of program effectiveness according to industry standards. Regarding the Return to Treatment (RTT) protocol, we have shown its potential in reconnecting clients with the addictions treatment system. This process, in conjunction with information collected from the GAIN Q3 Standard, will further facilitate the quality improvement efforts across the provincial treatment system. For example, we will not only be able to measure the levels of treatment retention and engagement (Garnick, Lee, Horgan, Acevedo, & the Washington Circle Public Sector Workgroup, 2009), but we expect an associated positive impact (i.e., increase in the levels of these key performance indicators) due to our follow up procedures. Given evidence linking service intensity and duration with better outcomes (Garnick et al., 2007; Harris, Humphreys, Bowe, Tiet, & Finney, 2010), we expect that our RTT protocol will address important gaps in system performance. In the future, these efforts may contribute to reductions in service utilization and costs. Specifically, it is well known that early attrition carries high costs to programs, in terms of resources dedicated to assessment and treatment planning in the early stages of treatment (Simpson et al., 1997; Walker, 2009). If increased retention and engagement lead to improvements in client outcomes (as measured using the GAIN Q3 Standard), then there is the potential for savings in the cost of reduced re admission to treatment, as well as reduced ER use, criminal justice, unemployment or other related costs (Garnick et al., 2009). 7.3 NEXT STEPS AND IMPLICATIONS Addiction agencies in Ontario lack the resources to provide comprehensive follow up services. Our recovery monitoring model has the potential to address this gap by providing an outreach service that could play an important role in the continuum of care. By recommending further piloting we are committing to improving the follow up process. Specifically, we would like to test our model in sites located in Toronto and programs that provide residential services. 80

Furthermore, we will strive to implement strategies that may help to maximize follow up rates. With additional funding we hope to explore : 1. strategies for targeting more diverse clients, especially those with severe substance use issues (alcohol and other drugs, frequent users, etc.), clients who are single, of younger age, and/or unemployed; 2. options for reducing the costs incurred by clients when completing follow up interviews (e.g., providing access to a project phone); 3. alternative modes of follow up and tool administration (e.g., in person interviews, mobile devices); 4. other approaches to reconnecting clients to services, such as booking an appointment with a counsellor while they are on the phone; 5. the optimal use of ConnexOntario in the Return to Treatment process; 6. electronic administration of recovery monitoring tools, especially the OWTOM A; and 7. use of community trackers. Work will also be needed on linking the outcome data to case mix profiles, as well as client satisfaction and other administrative type data. We are well positioned to continue further piloting, as we have built capacity through our follow up team and infrastructure, and local and system change agents (e.g., Program Advisory Committee, pilot sites). However, concrete next steps are needed. First, we must secure funding in order to carry out the next phase of pilot work. If Health Canada/MoHLTC funding is not available, then we will seek out external grants through the Canadian Institutes of Health Research and National Institutes of Health. Second, we will continue to work closely with our pilot sites, Program Advisory Committee, and LHIN and Ministry partners to refine our follow up protocol and develop strategies to improve project implementation across a variety of agencies in Ontario. During this process, we will also engage persons with lived experience, including family members. Third, we will ensure that our findings and plans going forward are shared with the addictions treatment community through EENet webinars, plain language summaries, and other KTE strategies will be employed. 81

Finally, support from all levels of the addiction treatment system (LHINs, Ministry, treatment agencies) will be needed to ensure the continuation of this work. While monetary support may not be feasible, stewardship, collaboration and a commitment to building a recovery monitoring system will at least be necessary for moving forward. The implications of not proceeding with further piloting are clear at the system level: an inability to report on how our clients (and services) are doing within the provincial treatment system. This in turn limits our ability to support ongoing investment of public resources in this critically needed system of services. 82

APPENDIX 1 GLOSSARY AND LIST OF ACRONYMS ABS ADAT ADSTV Assessment Baseline CAMH Catalyst Chestnut Health Systems ConnexOntario Community Trackers DATIS DHQ DTFP Evidence Informed Practice EENet Follow up Assessment Building System A web based application that allows the GAIN family of instruments to be administered and summarized by a computer. Admission and Discharge Criteria and Assessment Tools The ADAT is a MoHLTC package of eight tools mandated for all publicly funded addiction service programs. Addiction Services of Thames Valley An extensive and individualized identification of mental health and substance use strengths and needs of people whose screening results warrant future investigation. A measure against which future outcomes can be determined. For DTFP projects: Screening and Assessment and Recovery Monitoring, the baseline tool used was the GAIN Q3 MI. Centre for Addiction and Mental Health A browser based computer application, managed by DATIS, which houses clientlevel clinical and administrative data for MoHLTC funded addictions agencies in Ontario. A U.S. based organization that founded the GAIN family of instruments. A provincial registry that informs professionals and the general public about the availability of drug and/or alcohol services in Ontario. An outreach team dedicated to finding clients in the community, when traditional follow up methods (i.e. telephone calls) are unsuccessful in locating clients. Drug and Alcohol Treatment Information System A program dedicated to the on going development and maintenance of a comprehensive, province wide client information system. Drug History Questionnaire A 14 item tool designed to obtain detailed history of the client's use of alcohol and other drugs in the past 90 days. Drug Treatment Funding Program Interventions that effectively integrate the best research evidence with clinical expertise, cultural competence and the values of the persons receiving the services. These interventions have evidence showing improved outcomes for families, clients, and/or communities. Evidence Exchange Network for Mental Health and Addictions (EENet) A mental health and addictions knowledge exchange network that connects stakeholders across Ontario, Canada. A set of processes and procedures for follow up activities that was specifically 83

Protocol Follow up Team Fourcast GAIN GAIN CC GAIN SS CAMH Modified GRRS Hybrid Model LHIN Locator Form MoHLTC National Anti Drug Strategy Outcome Monitoring developed for the DTFP Recovery Monitoring project and implemented by the follow up team. This protocol was largely based on the follow up services established by the Lighthouse Institute of Chestnut Health Systems. A four person unit assembled to locate clients quarterly on the telephone and conduct follow up interviews in the DTFP Recovery Monitoring project. Four Counties Addiction Services Team Global Appraisal of Individual Needs A family of instruments designed to assist clinicians with diagnosis, placement and treatment planning of adolescents and adults in various treatment programs. The GAIN I is the full standardized clinical assessment. Subsets of the GAIN I include: the GAIN Q3 (both the MI and Standard), a short multipurpose assessment tool and the GAIN SS, a brief screening tool. In the DTFP Recovery Monitoring project the GAIN Q3 MI was used as a baseline tool, and the GAIN Q3 Standard was used as an outcome measure tool. The Gain Coordinating Centre A division of the Lighthouse Institute, a part of the Chestnut Health Systems, that is dedicated to providing services to current and prospective users of the GAIN family of instruments. A modified version of the GAIN SS that was developed by CAMH and includes seven additional items that cover eating disorders (2), traumatic exerperiences (1), psychotic symptoms (2) and problem gambling (2). However, these items have not been validated as an index. Global Appraisal of Individual Needs Recommendation Referral Summary A detailed and clinical report automatically generated by ABS upon completion of an assessment that can be edited by clinicians. A recovery monitoring model that integrates outcome determination directly into treatment, rather than the traditional pre post follow up strategy. Local Health Integration Network Created by the Ontario government, 14 not for profit corporations that work with local health providers and community members to determine the health service priorities of their designated regions. A form used to locate clients for the follow up phase of the DTFP Recovery Monitoring project that requests a wide range of client information. This form was adapted from the one used by the follow up services established by the Lighthouse Institute of Chestnut Health Systems Ministry of Health and Long Term Care A Canadian federal government strategy focused on three action plans (prevention, treatment, combating the production and distribution of illegal drugs) aimed at reducing the supply of and demand for illicit drugs, as well as addressing the crime associated with illegal drugs. Procedures and tools used to assess client changes in their symptoms, behaviour and function over time that can be attributed to their participant in treatment. This traditional model focuses on measuring client outcomes post treatment. 84

OWTOM A Program Advisory Committee Recovery Monitoring RTT Protocol Screening Staged Approach Withintreatment monitoring Working Group The Ontario Within Treatment Outcome Measure for Addictions (OWTOM A) A general short well being assessment tool developed for the DTFP Recovery Monitoring project. A committee of a broad range of stakeholders, who were engaged in the planning stages of the DTFP Recovery Monitoring Project. Similar to outcome monitoring, procedures and tools used to assess client outcomes over time, but with a specific emphasis on continuous monitoring that includes within treatment outcome measures. Return to Treatment Protocol. A set of processes and procedures developed for the DTFP Recovery Monitoring project, implemented during the follow up phase to determine client s needs for and interests in further treatment or support and provides linkage between clients and services accordingly The use of evidence based procedures and tools to identify individuals with problems, or those who are at risk for developing problems. It is intended to be an efficient way of raising a red flag about the possibility of a particular disorder or problem area. A progressive and efficient use of screening and assessment resources to guide treatment planning and eventually, recovery monitoring. Screening involves 2 stages: case finding and case definition. Assessment also consists of a 2 staged process: information gathering and placement and diagnosis and treatment planning. Procedures and tools used to assess client outcomes over time, during treatment, with the intent of assisting clinicians in planning treatment and providing feedback to clients about their treatment progress. A subgroup of the Program Advisory Committee that informed key decisions regarding many aspects of the study project including (but not limited to) the selection of tools, costing, and procedures for implementation. 85

APPENDIX 2 RECOVERY MONITORING PROJECT TOOLS 1. LOCATOR FORM 2. OWTOM A 3. GAIN Q3 STANDARD 4. RTT PROTOCOL 86

Follow-up Locator Form Revised Version 1.2 To be filled out by the interviewer DATE: (mm/dd/yyyy) START TIME: FOLLOW-UP LOCATOR FORM (Ontario DTFP Version 1.1) COPYRIGHT NOTICES This form is a shortened version of another with the same name, copyrighted and owned by The Lighthouse Institute and/or Dr. Christy Scott, Chicago, Illinois, USA. For permission to use this modified version please contact CAMH at (416) 535-8501 ext. 6681. PURPOSE AND INTRODUCTION Although it will seem that we are asking a lot of questions that may not be applicable to you, the questions on this form have been shown to be effective in locating people from many different walks of life. Also, keep in mind that although some of the questions may not seem relevant now, the information may be very appropriate at the time we are trying to reach you for your checkup. If we need to contact any of the individuals or organizations listed in this form, we will tell them only that we are trying to find you because you are participating in a survey. We will ask these individuals only for information on how to contact you and may leave a voicemail message unless you indicate otherwise on the form. We will not identify ourselves as being CAMH Confidential Health Systems and Health Equity Research Group 2012 86

Follow-up Locator Form Revised Version 1.2 General Contact Information Please fill out the following to the best of your ability. 1. What is your name? (Last name) (First name) (Middle name) 1a. Is this your married name? No Yes (Maiden name) 1b. What other name(s) are you known by? (alias/street name) 1c. What is your mother s maiden name? 1d. What is your date of birth? / / dd mm yyyy 1e. What is your most recent address? No Fixed Address: Street: Apt./Room/Floor: City: Province : Postal Code: 1. What is the name on the mailbox (Does mail need to be in care of someone else?) 2. Whose place is it? Name: Relationship: 3. What is the phone number at this location? ( ) - No voicemails How long will you be at the this place? 1f. What is your phone number? ( ) - Cell phone No voicemails 1. Is this the telephone number where you want to be called for your follow up interview? Yes No (If no), what number should we call? ( ) - No voicemails 2. What are the best days of the week to call you? 3. What are the best times to call you? 5. Do you have an email address? @. 1h. If something were to happen with your current living arrangements where is the best place to find you in 6 months to do the follow up interview? Street: Apt./Room/Floor: City: Province : Postal Code: 1. What is the name on the mailbox (Does mail need to be in care of someone else?) 2. Whose place is it? Name: Relationship: 3. What is the phone number at this location? ( ) - No voicemails CAMH Confidential Health Systems and Health Equity Research Group 2012 87

Follow-up Locator Form Revised Version 1.2 2a. Parent/s, (if parents deceased or no parents, ask: Is there someone older that you feel close to or feel that you could call for support)? Last Name: First Name: Address: Apt./Room/Floor: City: Province: Postal Code: 1. Whose place is it? Name: Relationship: 2. What is the telephone number at this location? ( ) - No voicemails Cell number? ( ) - No voicemails 2b. What relatives do you see or talk to (do not repeat previously given contact)? (If none ask, what acquaintances do you see or talk with?) Relationship: Last Name: First Name: Address: Apt./Room/Floor: City: Province: Postal Code: 1. Whose place is it? Name: Relationship: 2. What is the telephone number at this location? ( ) - No voicemails Cell number? ( ) - No voicemails 2c. Adult Children (do not repeat previously given contact)? Relationship: Last Name: First Name: Address: Apt./Room/Floor: City: Province: Postal Code: 1. Whose place is it? Name: Relationship: 2. What is the telephone number at this location? ( ) - No voicemails Cell number? ( ) - No voicemails 2d. Is there a friend that you see or talk to? (do not repeat previously given contact) (If none, ask what acquaintances do you see or talk with; e.g. neighbor) Last Name: First Name: Address: Apt./Room/Floor: City: Province: Postal Code: Name on phone listing if different than above: 1. Whose place is it? Name: Relationship: 2. What is the telephone number at this location? ( ) - No voicemails Cell number? ( ) - No voicemails CAMH Confidential Health Systems and Health Equity Research Group 2012 88

Follow-up Locator Form Revised Version 1.2 2e. Is there anyone else you see or talk to regularly? (do not repeat previously given contact) Relationship: Last Name: First Name: Address: Apt./Room/Floor: City: Province: Postal Code: 1. Whose place is it? Name: Relationship: 2. What is the telephone number at this location? ( ) - No voicemails Cell number? ( ) - No voicemails 3. Are you currently working or attending school outside of the home? Work School Both Neither (If neither, skip to next page) Please describe your employer or school (if you have both, please describe your job): 3a. Name: 3b. Street: 6c. Room: 3d. City 3e. Province 3f. Postal Code 3g. Telephone No. ( ) - Employment / School Contacts Please fill out the following to the best of your ability. 3h. Contact person (1) Contact person (2) 3j. What time do you usually start work or school? : am pm 3k. May we contact you there to confirm your follow-up interview if we cannot find you at home? No Yes 3m. Is there a co-worker or fellow student who usually knows how to reach you? No Yes Relationship: Last Name: First Name: Address: Apt./Room/Floor: City: Province: Postal Code: What is the telephone number at this location? ( ) - CAMH Confidential Health Systems and Health Equity Research Group 2012 89

Follow-up Locator Form Revised Version 1.2 Medical Contacts Please fill out the following to the best of your ability. 4a. Is there a hospital that you go to regularly or in case of an emergency? Address: Apt./Room/Floor: City: Province: Postal Code 4b. Is there a clinic that you go to regularly or in case of an emergency? Address: Apt./Room/Floor: City: Province: Postal Code 4c. Do you use prescribed medications on a regular basis? YES NO Where do you pick up your meds? Address: Apt./Room/Floor: City: Province: Postal Code Day of the Month: Time of Day: : 1 am 2 pm 4d. Have you been under care or treatment at a community agency or rehabilitation facility? Yes No Facility Name Address: Apt./Room/Floor: City: Province: Postal Code Contact Name 5. Please provide the names and addresses of health care or drug treatment professionals who may know where you are in the next 3 months: 5a. Counselor or other health provider (describe: ) Name: Organization/Agency: Address: Apt./Room/Floor: CAMH Confidential Health Systems and Health Equity Research Group 2012 90

Follow-up Locator Form Revised Version 1.2 City: Province: Postal Code Telephone Number: ( ) - 5b. Doctor/Nurse or other health provider (describe: ) Name: Organization/Agency Address: Apt./Room/Floor: City: Province: Postal Code Telephone Number: ( ) - 5c. Pharmacist, Dentist or other health provider (describe: ) Name: Organization/Agency Address: Apt./Room/Floor: City: Province: Postal Code Telephone Number: ( ) - 5d. Social Worker, Caseworker or other professional (describe: ) Name: Organization/Agency (optional) Address: Apt./Room/Floor: City: Province: Postal Code Telephone Number: ( ) - CAMH Confidential Health Systems and Health Equity Research Group 2012 91

Follow-up Locator Form Revised Version 1.2 Mobility Please fill out the following to the best of your ability. 6a. Do you go to any shelters, community centers, religious organizations, or healthcare clinics? Contact person: Organization/Agency: Address: City: Province: Postal Code: Telephone number: 6b. Where do you typically get your food? (i.e. soup kitchens, restaurants, churches, stores) Organization/Agency/Store: Address: City: Province: Postal Code: What days of the week are you there? What times of the day? (Breakfast, lunch, dinner) 6c. If you sleep outside, where do you go (i.e. hallways, building front, viaduct)? Name of place 1: Location: Markers: (Park bench, etc) Name of place 2: Location: Markers: (Park bench, etc) 6d. Are there any places that you regularly hang out with your friends (i.e. parks, corners)? Name or type of place: Location/Intersection: What times of the day are you there? (Morning, afternoon, evening) What days of the week? 6e. Other Locating Information: (hang outs, parks, where eat, shower, etc.): CAMH Confidential Health Systems and Health Equity Research Group 2012 92

Follow-up Locator Form Revised Version 1.2 Identifying Features Please fill out the following to the best of your ability. 7. In trying to locate you for a follow up interview and to make sure we find the right person, can you tell us approximately?: 7a. How tall are you? (Feet/Inches) 7b. How much do you weigh? (Pounds) 7c. How would you describe your eye color? Black... Blue/Gray... Brown... Green... Hazel... Other (Please describe)... v. 7d. How would you describe your hair color? Black... Blonde... Brown/Brunette... Red... Gray... Other (Please describe)... v. 7e. How would you describe your skin color? Light... Medium... Dark... Other (Please describe)... v. 7g.What is your age today?....................... years months 7h. How would you describe you gender identity? 1. Male 2. Female 3. Other 7i. What is you main language?.. 7j. Do you speak any other language?.. 7i. Do you have any distinguishing features (e.g., scars, tattoos, piercing, other?) Thank you. These are all the questions we have. END TIME: To be filled out by the interviewer INTERVIEWER INITIALS: CAMH Confidential Health Systems and Health Equity Research Group 2012 93

To be filled out by the interviewer DATE: (mm/dd/yyyy) START TIME: CLIENT NAME: Ontario Within-Treatment Outcome Measure for Addictions Section A: The next questions are about how satisfied you feel with different parts of your life. Please indicate how satisfied you currently feel by responding Very Satisfied, Satisfied, Mixed, Dissatisfied, or Very Dissatisfied. Please place an X in the box that corresponds with your answer. Currently, how satisfied are you with A1. The level of intimate relationships (e.g. marital, partner, sexual)? A2. Your family relationships? A3. Your general level of happiness? A4. Where you are living? A5. How your life is going so far? A6. Your school or work situation? Very Satisfied Satisfied Mixed Dissatisfied Very Dissatisfied Not Applicable /Don t know (1) (2) (3) (4) (5) (9) Section B: Supplementary Questions Please place an X in the box that corresponds with your answer. B1. "How confident are you that you will achieve your substance use goals for the next month? 1. Very Confident 2. Quite Confident 3. Somewhat Confident 4. Not At All Confident 9. Not Applicable/Can t Really Say 94

B2. "The content of my treatment sessions is well suited to my needs?" 1. Strongly Agree 2. Agree 3. Unsure 4. Disagree 5. Strongly Disagree 9. Not Applicable/Can t Really Say B3. How important today is it for you to make changes related to your treatment goals? 1. Very Important 2. Important 3. Somewhat Important 4. Not Important 9. Not Applicable/Can t Really Say END TIME: To be filled out by the interviewer COMPLETED BY: CLIENT CLINICIAN CLINICIAN INITIALS: 95

Global Appraisal of Individual Needs - Q3 (GAIN-Q3) Version [GVER]: 3.2.3 ONT Standard Copyright 2000-2012 by Chestnut Health Systems Site ID [XSITE]:... Staff ID [XSID]:... Part. ID [XPID]:... Observation [XOBS]:... Edit Staff ID [XEDSID]:... Data Entry Staff ID [XDESID]:... Local Site ID [XSITEa]:... Staff Initials [XSIN]:... Last Name [XPNAM]:... First Name: v. Edit Date [XEDDT]:... Key Date [XDEDT]:... / / M.I.: / 20 / 20 For Staff Use Only A1. Administrative Information A1a. Time:... : HH:MM... A1b. (AM/PM) A1c. Today's Date [XOBSDT]:... / / 20 (MM/DD/YYYY) Introduction Purpose: The purpose of this assessment is to provide a summary of how things have been going in your life. The information collected will be used only to identify and address problems that you may want assistance with and to help us evaluate our own services. Format: This assessment has questions about what has been going on in your life across a wide range of areas, including your physical and mental health, stress and risk behaviours, and life satisfaction. You will be able to say you do not know or refuse to answer any question that you do NOT want to answer. Length: Depending on how much has been going on in your life, it will take about 20-45 minutes to complete. You will be able to take a break if you need to. Privacy and Confidentiality: Your answers are private. All research and clinical staff who have access to your answers understand this restriction and have agreed not to share your specific answers without your prior written consent. This includes giving information to family members, other individuals, other treatment agencies, social work agencies, criminal justice agencies and other agencies. There are, however, two exceptions. First, we are mandated to report child abuse or if you are presently a danger to yourself or others. Second, officials from the federal government have the right to audit us to check to make sure we have protected your safety and accurately reported what we have done. Any questions? GQ 3.2.3 ONT Standard 08/24/2012

GAIN-Q3 A3. Timeframe Anchoring Several questions will ask you about things that have happened during the past 90 days. To help you remember this time period, please look at the calendar. First, let's find today's date and circle it. Next, count back 13 weeks to about 90 days ago and circle that date. Do you recall anything that was going on about (DATE 90 DAYS AGO)? (PROBE FOR SPECIFIC EVENT. IF UNABLE TO RECALL: Do you remember any birthdays, holidays, sporting or other special events that happened around (DATE 90 DAYS AGO)? Did anything change in terms of where you were living, who you were with, whether you were in treatment, work, school or jail? Where were you living then? Were you in treatment, working, in school, or involved with the law then?) A3a1. Record anchor for 90 days: v. When we talk about things happening to you during the past 90 days, we are talking about things that have happened since about (NAME 90-DAY ANCHOR). Please keep this calendar handy and use it as we go through the interview to help you remember when different things happened. Additional Administration Instructions As we go through the questionnaire, I will read the questions and record your answers. It is important that you try to answer each question if you can and are willing to. You may not always know the exact answer, but I would like you to give me your best guess if you can. You can also tell me if you simply do not know or if you do not want to or refuse to answer any questions. I also have some cards here that we will use to help answer some of the questions. Do you have any questions before we begin? GQ 3.2.3 ONT Standard 08/24/2012

A4a_c. In a few words, can you tell me why you are here today? (What is your main reason for coming to treatment?) (Do not ask, "Any others?") v1. (Clarify and code) Drug availability (difficulties obtaining drugs or "good" drugs)... 1 Financial (can't afford to stay on drugs, lost an income source)... 2 General personal motive ("habit out of control," "tired," "want to change," "improve lifestyle," "save self")... 3 Health reasons (too ill to continue; drugs or related diseases are hurting or threatening own health, unborn baby, to live)... 4 Pressure from family (parent, spouse, partner)... 5 Parenting issues (get or keep custody or become better parent)... 6 Pressure from criminal justice system (court mandate, probation officer, parole officer, attorney, etc.)... 7 Pressure from school teacher, minister, coach, etc.... 9 Desire for services (want housing or other benefit)... 10 School or job (to get, keep or improve situation)... 11 Pressure from Child Welfare Services... 13 Other (Please describe in A4a_cv1)... 99 A4b. What is the name of the person who referred you to come here? v. A4c. What is this person's relationship to you? v. GQ 3.2.3 ONT Standard 08/24/2012

A4d_c. Referral Sources (check up to two categories from below)... 01 Self 02 Family/Friends 03 Education/Training Programs/ Services 04 Initial Assessment Treatment Planning (Agency) 05 Residential Withdrawal Management Service Level 1, 2, 3 06 Community Withdrawal Management Service Level 1, 2, 3 07 Residential Treatment Service 08 Residential Supportive Housing Service Level 1 & 2 09 Community Treatment 10 Community Day/Evening Treatment Services 11 Residential Medical/Psychiatric Services 12 Community Medical/Psychiatric Services 13 Case Management 14 Psychiatric Services/Hospital Individuals, Professionals and Agencies 15 Private Psychiatrist/Psychologist 16 Medical Services - Hospital 17 Medical Services - Private 18 Community Health Centre 19 Physician/Private Practitioner 20 Public Health Unit/Nursing Services 21 Community Mental Health Agency - Adult Program 22 Community Mental Health Agency - Child Program 23 Social Service Agency - Adult Program 24 Social Service Agency - Child Program 25 Other Community Institution/ Residential Program 26 Housing Programs/Services 27 Self Help Groups (e.g. Alcoholics Anonymous) 28 EAP - Employee Assistance Program 29 Police 30 Other Legal System, Excluding Police 31 DART - Drug and Alcohol Registry of Treatment 32 Other 33 Traditional Healer/Elders 34 Women's/Men's Shelters 35 Alternative Health Therapies 36 Native Treatment Services 37 Housing- Native/Non-Profit 38 Other Native Services 39 Toronto WMS Central Access (For Toronto WMS ONLY) 40 OPGH - Ontario Problem Gambling Helpline 41 MHSIO - Mental Health Services Information Ontario 42 Responsible Gaming Information Centres 88 Unknown GQ 3.2.3 ONT Standard 08/24/2012

B. Background Information In this first section, I am going to ask you some very basic questions about yourself. B1_c. What is your gender? Male... 01 Female... 02 Other (Please describe)... 99 v. BAC B1d. About how tall are you in feet and inches?... Feet Inches B1e. About how much do you weigh without shoes?... Pounds B2. What is your date of birth?... / / Month Day Year B2a. How old are you today?... Age [IF 18 OR OVER, GO TO B3a_c] B2b_c. Who currently has legal custody of you? (Would you say...) v. (Clarify and code) Parents living together... 1 Parents who are separated but share custody... 2 One parent (even if living with stepparent)... 3 Other family members... 4 Legally emancipated minor living on your own... 5 Runaway/on own (without legal emancipation)... 6 Crown Ward (foster home or protective services)... 7 Juvenile or correctional institution... 8 Other (Please describe in B2b_cv)... 99 GQ 3.2.3 ONT Standard 08/24/2012

Please answer the next questions using yes or no. PAI B2. During the past 12 months, have you done any of the following things with your (biological, foster, adopted or step) parents? Yes No e. Spent 30 minutes or more playing or doing fun things with them... 1 0 f. Gone with them to an organized activity or event... 1 0 g. Had them read to you, or talked to them about a book, magazine or newspaper... 1 0 h. Gotten help from them with your homework (reading, writing or math)... 1 0 j. Had them meet with a teacher, social worker, lawyer, court official or police officer about you... 1 0 B3a_c. Which population group best describes you?(select one) White... 1 First Nations/Aboriginal Ancestry... 2 Asian... 3 Black... 4 Middle Eastern... 5 Latin American... 6 Multiple or mixed... 7 B3a_c1. If your population group is White, which of the following best describes your background?(select one) North European (Danish, Norwegian, Swedish, etc.)... 1 Central Western European (English, Scottish, Irish, Welsh, German, Dutch, Czech, Slovak, etc.)... 2 South European (Italian, Spanish, Portuguese, Greek, French, Turkish, etc.)... 3 East European (Bulgarian, Ukrainian, Polish, Romanian, Russian, Slovenian, Serbian, Croat, etc.)... 4 North American (Canadian, American, etc.)... 5 B3a_c2. If your population group is First Nations/Aboriginal Ancestry, which of the following best describes your background?(select one) Aboriginal Status... 1 Aboriginal Non-Status... 2 Métis... 3 Inuit... 4 Other (Please describe)... 99 v. GQ 3.2.3 ONT Standard 08/24/2012

B3a_c3. If your population group is Asian, which of the following best describes your background?(select one) East Asian (e.g., Chinese, Japanese, Korean)... 1 South Asian (e.g., Indian, Pakistani, Afghani, Sri-Lankan)... 2 South East Asian (e.g., Filipino, Malaysian)... 3 B3a_c4. If your population group is Black, which of the following best describes your background?(select one) Black African (e.g., Ghanaian, Somali, Kenyan, Ethiopian)... 1 Black Caribbean (e.g., Trinidadian, Jamaican)... 2 Black American... 3 Other (Please describe)... 99 v. B3a_c5. If your population group is Middle Eastern, which of the following best describes your background?(select one) Arabic (e.g., Saudi Arabia, Jordan)... 1 Northern African (e.g., Egyptian, Libyan)... 2 West Asian (e.g., Syrian, Lebanese, Iranian, Iraqi)... 3 Israeli... 4 Other (Please describe)... 99 v. B3a_c6. If your population group is Latin American, which of the following best describes your background?(select one) South American (e.g., Argentinean, Chilean, Peruvian)... 1 Central American (e.g., Mexican, Costa Rican)... 2 Caribbean... 3 Other (Please describe)... 99 v. B3a_c7. If you are from multiple or mixed population groups, please describe: v. GQ 3.2.3 ONT Standard 08/24/2012

B12. What is the last grade or year that you completed in school? (NOTE: Use 12 for high school, 14 for 2 year college program, 16 for a BA/BS, and 17 for graduate school or more than 4 years of university). B13_c. What is your highest level of education? Grade (Select one) No formal schooling... 1 Some primary school... 2 Primary school... 3 Some secondary or high school... 4 Completed secondary or high school... 5 Some community college, technical college, CEGEP... 6 Completed community college, technical college, CEGEP... 7 Some university (not completed)... 8 University degree (completed) Bachelors, Masters, PHD... 9 Unknown... 88 B14_c. Which of the following best describes your sexual orientation? (Select one) Asexual or non-sexual... 1 Bisexual... 2 Gay... 3 Heterosexual or straight... 4 Lesbian... 5 Not sure or Questioning... 6 Queer... 7 Two-spirited or another indigenous sexual identity... 8 Other or no option applies (Please describe)... 99 v. B15. What is your current marital status? (Clarify and code) Married... 1 Remarried... 2 Living with someone as married... 3 Married but living apart... 4 Divorced... 5 Legally separated... 6 Widowed... 7 Never married and not living as married... 8 GQ 3.2.3 ONT Standard 08/24/2012

B15a. What kind of housing do you currently live in? (Clarify and code) A house, apartment or room you, your spouse, your partner, or your parents rent or own... 1 A foster home... 2 A public housing or rent-subsidized apartment or house registered in your or your family's name... 3 A friend or relative's house, apartment or room... 4 An unsupervised dormitory or quarters, such as at college, religious or military quarters or agriculture or other workers' quarters... 5 A nursing home or any other kind of group home... 6 Any kind of hospital, inpatient or residential facility for medical, mental, alcohol or drug-related problems... 7 A jail, detention center, correctional halfway house or other correctional institution... 8 Temporary or emergency shelter for people who are homeless, runaways, neglected or abused... 9 Vacant buildings, public or commercial facilities, parks, cars or on the street because you do not have a place to stay... 10 Any other housing situation (Please describe)... 99 v. B15b. How long have you been living there?... + + + Years Months Weeks Days B15c. During the past 90 days, on how many days did you live someplace where you were not free to come and go as you please - such as jail, an inpatient program, or a hospital?... Days [IF UNDER 17, SELECT 0 AND GO TO B17] B16_c. Have you ever been in the Canadian Forces or armed forces in another country? (Select one) No, never served in any armed forces... 0 [GO TO B17] Yes, served in the Canadian Forces... 1 Yes, served in the armed forces or military of another country (Which country?)... 99 v. Yes No B16a. Were you ever in a combat zone?... 1 0 Where? v. GQ 3.2.3 ONT Standard 08/24/2012

B16b. What was your highest rank in the military? v. B16c. What is your military status or type of discharge? (Clarify and select all that apply) v. MENTIONED 1. On active duty in the armed forces... Yes 1 No 0 2. In a selected or other reserve component that drills regularly... 1 0 3c. In the Canadian Army... 1 0 4c. In the Canadian Army Reserve... 1 0 5. Retired... 1 0 6. Honorably discharged(are you currently in any kind of reserve?)... 1 0 7. Generally discharged or entry-level separation... 1 0 8. Other than honorably discharged... 1 0 9. Bad conduct or other administrative discharge or dismissal... 1 0 10. Dishonorably discharged or dismissal after court martial... 1 0 99. Other (Please describe in B16cv)... 1 0 [IF B16c6-99 = 0, GO TO B17] Yes No B16d. Was your discharge related to any physical, medical, mental, alcohol, drug or other problems?... 1 0 [IF NO, GO TO B17] B16d. What were the problems? (Please record and select all that apply) v. MENTIONED 1. Physical... Yes 1 No 0 2. Medical... 1 0 3. Mental... 1 0 4. Alcohol... 1 0 5. Drug... 1 0 99. Other problem (Please describe in B16dv)... 1 0 [IF MALE, GO TO SP1] Yes No B17. Are you currently pregnant?... 1 0 GQ 3.2.3 ONT Standard 08/24/2012

SP. School Problems The next questions are about being in any kind of school or training program. Using Card Q and responding "in the past month," "2 to 3 months ago," "4 to 12 months ago," "1 or more years ago," or "never"... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 SPScr/ SP1. QOLI SP1f. When was the last time you... a. came in late or left early from school or training?... 4 3 2 1 0 b. skipped or cut school or training just because you didn't want to be there?.. 4 3 2 1 0 c. got bad grades or had your grades drop at school or training?... 4 3 2 1 0 d. got sick at school or training?... 4 3 2 1 0 e. went to any kind of school or training?... 4 3 2 1 0 When was the last time, if ever, you received any kind of help dealing with school problems (for example, talking to a school counselor about problems at school, working with a tutor, attending a social skills group at school)?... 4 3 2 1 0 [IF SP1e IS LESS THAN 3, GO TO SP1f1] Please answer the next questions using the number of days. QCS SP1e. During the past 90 days, on how many days... 1. were you absent from school or training for a full day?... Days 2. did you go to any kind of school or training?... Days [IF SP1f IS LESS THAN 3, GO TO WP1a] Please answer the next questions using the number of days. SP1f1. During the past 90 days, on how many days have you received any kind of help dealing with school problems?... Days GQ 3.2.3 ONT Standard 08/24/2012

WP. Work Problems The next questions are about working at a job. For these items, a job includes a full or part-time job that you are paid for doing, including military service. If you have never worked, please answer "never". Using Card Q... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 WPScr/ WP1. QOLI When was the last time you... a. came in late or left early from work?... 4 3 2 1 0 b. skipped or cut work just because you didn't want to be there?... 4 3 2 1 0 c. did badly at work or did worse at work?... 4 3 2 1 0 d. got sick at work?... 4 3 2 1 0 e. went to work?... 4 3 2 1 0 WP1f. When was the last time, if ever, you received any kind of help dealing with work problems (for example, talking to a counselor about problems at work, using the services of an employee assistance program, participating in mediation for dispute resolution)?... 4 3 2 1 0 [IF WP1e IS LESS THAN 3, GO TO WP1f1] Please answer the next questions using the number of days. QCS WP1e. During the past 90 days, on how many days... 1. were you absent from work for a full day?... 2. did you work for money at a job or in a business?... Days Days [IF WP1f IS LESS THAN 3, GO TO PH1a] Please answer the next questions using the number of days. WP1f1. During the past 90 days, on how many days have you received any kind of help dealing with work problems?... Days GQ 3.2.3 ONT Standard 08/24/2012

PH. Physical Health The next questions are about your physical health. Using Card Q... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 HPScr/ PH1. QOLI When was the last time you... a. gained 10 or more pounds when you were not trying to?... 4 3 2 1 0 b. lost 10 or more pounds when you were not trying to?... 4 3 2 1 0 c. were worried about your health?... 4 3 2 1 0 d. had a lot of physical pain or discomfort?... 4 3 2 1 0 e. had health problems that kept you from meeting your responsibilities at work, school or home?... 4 3 2 1 0 f. saw a doctor or nurse about a health problem or took prescribed medication for one?... 4 3 2 1 0 Please answer the next questions using the number of times, nights or days. PH1e1. During the past 90 days, on how many days did you have an injury where any part of your body was hurt?... Days [IF PH1f IS LESS THAN 3, GO TO PH2a] QCS PH1f. During the past 90 days, how many... 1. times have you had to go to the emergency room for a health problem?... Times 2. nights total did you spend in the hospital for a health problem?... Nights 3. times did you see a doctor or nurse in an office or outpatient clinic for a health problem?... Times 4. times did you have an outpatient surgical procedure for a health problem?... Times 5. days did you take prescribed medication for a health problem?... Days GQ 3.2.3 ONT Standard 08/24/2012

PPI PH2. During the past 90 days, on how many days... a. have you been bothered by any health or medical problems?... [IF 0, GO TO PH2c] Days b. have health problems kept you from meeting your responsibilities at work, school or home?... Days c. have you smoked or used any kind of tobacco?... Days d. have you exercised for at least 20 minutes per day?... Days GQ 3.2.3 ONT Standard 08/24/2012

SS. Sources of Stress The next questions are about stress in your life. Using Card Q... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 SSScr/ QOLI SS1. When was the last time you were under stress for any of the following reasons? a. Death of a family member or close friend.... 4 3 2 1 0 b. Health problem of a family member or close friend.... 4 3 2 1 0 c. Fights with boss, teacher, coworkers or classmates.... 4 3 2 1 0 d. Major change in relationships for you or your family (e.g., marriage, divorce, separations).... 4 3 2 1 0 e. Something you saw or that happened to someone close to you. (Please describe)... 4 3 2 1 0 v. f. New job, position or school.... 4 3 2 1 0 g. You didn't have enough money to pay all your bills on time.... 4 3 2 1 0 SS1g1. When was the last time, if ever, that you considered yourself to be homeless?... 4 3 2 1 0 SS1h. When was the last time, if ever, you received any kind of help dealing with your stress (for example, talking to a counselor about ways to manage stress, participating in classes to learn to better manage stress)?... 4 3 2 1 0 [IF SS1h IS LESS THAN 3, GO TO SS2a] Please answer the next questions using the number of days. SS1h1. During the past 90 days, on how many days have you received any kind of help dealing with your stress?... Days PPI SS2. During the past 90 days, on how many days have you... a. felt stressed by events or situations in your life?... Days b. had any money problems, including arguing about money or not having enough for food or housing?... Days GQ 3.2.3 ONT Standard 08/24/2012

RB. Risk Behaviours and Trauma The next questions are about experiences of trauma and/or behaviours that put you at risk for getting and spreading infectious diseases, including HIV. These experiences or behaviours may be things you have done or that others have done to you. Please remember that all your answers are strictly confidential. Using Card Q... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 RBScr/ RB1. When was the last time you... QOLI a. had two or more different sex partners during the same time period?... 4 3 2 1 0 b. had sex without using any kind of condom, dental dam or other barrier to protect you and your partner from diseases or pregnancy?... 4 3 2 1 0 c. had sex while you or your partner was high on alcohol or other drugs?... 4 3 2 1 0 d. used a needle to inject drugs like heroin, cocaine or amphetamines?... 4 3 2 1 0 Trauma g. were attacked with a weapon, including a gun, knife, stick, bottle or other weapon?... 4 3 2 1 0 h. were physically abused, where someone hurt you by striking or beating you to the point that you had bruises, cuts or broken bones?... 4 3 2 1 0 j. were sexually abused, where someone pressured or forced you to participate in sexual acts against your will, including your regular sex partner, a family member or friend?... 4 3 2 1 0 k. were emotionally abused, where someone did or said things to make you feel very bad about yourself or your life?... 4 3 2 1 0 [IF ALL RB1g-k = 0, GO TO RB1n] RBScr RB1. When was the last time you... m1. were abused several times or over a long period of time?... 4 3 2 1 0 m2. were afraid for your life or that you might be seriously injured by the abuse?... 4 3 2 1 0 RB1n. When was the last time, if ever, you received any kind of help to reduce your risk behaviours (for example, participating in a needle exchange program, being instructed in safe sex practices, moving to a shelter for domestic violence victims)?... 4 3 2 1 0 [IF RB1n IS LESS THAN 3, GO TO RB2a] RB1n1. During the past 90 days, on how many days did you receive any kind of intervention to reduce your risk behaviours?... Days GQ 3.2.3 ONT Standard 08/24/2012

Please answer the next questions using the number of times or days. If something does not apply, please answer zero (0). PPI RB2. During the past 90 days, how many... a. times have you had unprotected sex (sex without using any kind of condom, dental dam or other barrier to protect you and your partner from disease or pregnancy)?... Times b. days have you used a needle to inject any kind of drug or medication?... Days c. days have you been attacked with a weapon, beaten, sexually abused or emotionally abused?... Days d. days have you gone without eating or thrown up much of what you did eat?... Days GQ 3.2.3 ONT Standard 08/24/2012

MH. Mental Health The next questions are about common psychological, behavioural and emotional problems. These problems are considered significant when you have them for two or more weeks, when they keep coming back, when they keep you from meeting your responsibilities, or when they make you feel like you can't go on. Using Card Q... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 IDScr6/ MH1. QOLI When was the last time you had significant problems with... a. feeling very trapped, lonely, sad, blue, depressed or hopeless about the future?... 4 3 2 1 0 b. sleep trouble, such as bad dreams, sleeping restlessly or falling asleep during the day?... 4 3 2 1 0 c. feeling very anxious, nervous, tense, fearful, scared, panicked or like something bad was going to happen?... 4 3 2 1 0 d. becoming very distressed and upset when something reminded you of the past?... 4 3 2 1 0 e. thinking about ending your life or committing suicide?... 4 3 2 1 0 f. seeing or hearing things that no one else could see or hear or feeling that someone else could read or control your thoughts?... 4 3 2 1 0 Using Card Q... EDScr6 MH2. QOLI When was the last time you did the following things two or more times? a. Lied or conned to get things you wanted or to avoid having to do something.... 4 3 2 1 0 b. Had a hard time paying attention at school, work or home.... 4 3 2 1 0 c. Had a hard time listening to instructions at school, work or home.... 4 3 2 1 0 d. Had a hard time waiting for your turn.... 4 3 2 1 0 e. Were a bully or threatened other people... 4 3 2 1 0 f. Started physical fights with other people.... 4 3 2 1 0 g. Tried to win back your gambling losses by going back another day.... 4 3 2 1 0 MH2h. When was the last time, if ever, you were treated for a mental, emotional, behavioural or psychological problem by a mental health specialist or in an emergency room, hospital or outpatient mental health facility, or with prescribed medication?... 4 3 2 1 0 GQ 3.2.3 ONT Standard 08/24/2012

[IF MH2h IS LESS THAN 3, GO TO MH3a] Please answer the next questions using the number of times, nights or days. QCS PPI MH2h. During the past 90 days, how many... 1. times have you had to go to an emergency room for mental, emotional, behavioural or psychological problems?... Times 2. nights total did you spend in the hospital for mental, emotional, behavioural or psychological problems?... Nights 3. times did you see a mental health doctor in an office or outpatient clinic for mental, emotional, behavioural or psychological problems?... Times 4. days did you take prescribed medication for mental, emotional, behavioural or psychological problems?... Days MH3. During the past 90 days, on how many days... a. were you bothered by any nerve, mental or psychological problems?... [IF 0, GO TO MH3c] Days b. did these problems keep you from meeting your responsibilities at work, school or home, or make you feel like you could not go on?... Days c. have you been disturbed by memories of things from the past that you did, saw or had happen to you?... Days d. have you had any problems paying attention, controlling your behaviour, or broken rules you were supposed to follow?... Days GQ 3.2.3 ONT Standard 08/24/2012

SU. Substance Use The next questions are about your use of alcohol and other drugs. Alcohol includes beer, wine, whiskey, gin, scotch, tequila, rum or mixed drinks. "Other drugs" include a) marijuana, b) other street drugs like crack, heroin, PCP, or poppers, c) inhalants like glue or gasoline and d) any non-medical use of prescription-type drugs. Please do not include any prescription drugs you used only as instructed by a doctor. Using Card Q... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 SDScr/ SU1. QOLI When was the last time... a. you used alcohol or other drugs weekly or more often?... 4 3 2 1 0 b. you spent a lot of time either getting alcohol or other drugs, using alcohol or other drugs, or recovering from the effects of alcohol or other drugs (e.g., feeling sick)?... 4 3 2 1 0 c. you kept using alcohol or other drugs even though it was causing social problems, leading to fights, or getting you into trouble with other people?... 4 3 2 1 0 d. your use of alcohol or other drugs caused you to give up or reduce your involvement in activities at work, school, home or social events?... 4 3 2 1 0 e. you had withdrawal problems from alcohol or other drugs like shaky hands, throwing up, having trouble sitting still or sleeping, or you used any alcohol or other drugs to stop being sick or avoid withdrawal problems?... 4 3 2 1 0 f. you received treatment, counseling, medication, case management or aftercare for your use of alcohol or any other drug? Please do not include any emergency room visits, detoxification, self-help or recovery programs. 4 3 2 1 0 [IF SU1f IS LESS THAN 3, GO TO SU2a] Please answer the next questions using the number of times, nights or days. QCS SU1f. During the past 90 days, how many... 2. nights were you in a halfway house, residential, inpatient, or hospital program for your alcohol or other drug use problems?... Nights 3. days were you in an intensive outpatient or day program for your alcohol or other drug use problems?... Days 4. times did you go to a regular (1-8 hours per week) outpatient program for your alcohol or other drug use problems?... Times 5. days did you take medication like methadone or Antabuse to help with withdrawal or cravings?... Days 99. days did you go to any other kind of treatment program or work with some other kind of case manager for your alcohol or other drug use problems? (Please describe)... Days v. GQ 3.2.3 ONT Standard 08/24/2012

QCS SU2. During the past 90 days, how many... a. days have you been in a detoxification program to help you through withdrawal?... Days b. days have you attended one or more self-help group meetings (such as Alcoholics Anonymous, Narcotics Anonymous, Cocaine Anonymous or Al-Anon) for your own or another person's alcohol or other drug use?... Days c. times have you been given a breathalyzer or urine test to check for your alcohol or other drug use? (Do not count any today)... Times d. times did you go to an emergency room for your alcohol or other drug use problems?... Times Please answer the next questions using the number of days. PPI SU3. During the past 90 days... a. on how many days did you go without using any alcohol, marijuana or other drugs?... [IF 90, GO TO SU5] Days b. on how many days did you get drunk at all or were you high for most of the day?... Days c. on how many days did alcohol or other drug use problems keep you from meeting your responsibilities at work, school or home?... Days PPI SU4. During the past 90 days, on how many days have you... a. used any kind of alcohol (beer, gin, rum, scotch, tequila, whiskey, wine or mixed drinks)?... [IF 0, GO TO SU4c] Days b. gotten drunk or had 5 or more drinks?... Days c. used marijuana, hashish, blunts or other forms of THC (herb, reefer, weed)?... Days d. used cocaine, opioids, methamphetamine or any other drug, including a prescription medication that was not prescribed to you, or one that you took more of than you were supposed to?... [IF 0, GO TO SU5] Days GQ 3.2.3 ONT Standard 08/24/2012

SU4. During the past 90 days, on how many days have you... e. used crack, smoked rock or freebase?... Days f. used other forms of cocaine?... Days g. used inhalants or huffed (such as correction fluid, gasoline, glue, lighters, spray paints or paint thinner)?... Days h. used heroin or heroin mixed with other drugs?... Days j. used nonprescription or street methadone?... Days k. used painkillers, opiates, or other analgesics (such as codeine, Darvocet, Darvon, Demerol, Dilaudid, Karachi, OxyContin, Oxys, Percocet, Propoxyphene, morphine, opium, Talwin or Tylenol with codeine)?... Days m. used PCP or angel dust (phencyclidine)?... Days n. used LSD (acid), ketamine, special K, mushrooms, or other hallucinogens (such as K2, mescaline, peyote, psilocybin, shrooms or spice)?... Days p. used anti-anxiety drugs or tranquilizers (such as Ativan, Deprol, Equanil, Diazepam, Klonopin, Meprobamate, Librium, Miltown, Serax, Valium or Xanax)?... Days qa. used methamphetamine, crystal, ice, glass, or other forms of methedrine (such as Desoxyn)?... Days qb. used speed, uppers, amphetamines, ecstasy, MDMA or other stimulants (such as Adderall, Biphetamine, Benzedrine, Concerta, Dexedrine, Methylphenidate, Mixed Salt Amphetamine or Ritalin)? Days r. used downers, sleeping pills, barbiturates or other sedatives (such as Dalmane, Donnatal, Doriden, Flurazepam, GHB, Halcion, liquid ecstasy, methaqualone, Placidyl, Quaalude, Secobarbital, Seconal, Rohypnol or Tuinal)?... Days s. used any other drug that has not been mentioned (such as amyl nitrate, cough syrup, nitrous oxide, NyQuil, poppers, Robitussin or steroids)? (Please describe)... Days v. GQ 3.2.3 ONT Standard 08/24/2012

SU5. During the past 90 days, on how many days have you been in a jail, hospital or other place where you could not use alcohol, marijuana or other drugs? (Use 0 for none)... [IF 0-12, GO TO SU10] Days To help you remember the time period for the next set of questions, let's get out the calendar like we did earlier and mark out the last 90 days when you spent fewer than 13 days in a jail, hospital or other place where you could not use alcohol, marijuana or other drugs. Do you recall anything that was going on about (DATE 90 DAYS BEFORE PARTICIPANT ENTERED CONTROLLED ENVIRONMENT)? (PROBE FOR SPECIFIC EVENT AS BEFORE) Record anchor: v. When we talk about things happening to you during "those 90 days in the community," we are talking about things that have happened from about (PRE-CONTROLLED ENVIRONMENT ANCHOR) to the time you entered the controlled environment. Please answer the next questions using the number of days. (Use 0 for none) SU5. In those 90 days in the community... a. on how many days did you go without using any alcohol, marijuana or other drugs?... [IF 90, GO TO SU10] Days b. on how many days did you get drunk at all or were you high for most of the day?... Days c. on how many days did alcohol or other drug use problems keep you from meeting your responsibilities at work, school or home?... Days For the following questions, please do not count people just because they are in the same building (e.g., jail, detention, school), or because you only see them a few times. Using number of people... SU10. During the past 12 months, how many people would you say you have regularly lived with, including your parents and family?... [IF 0, GO TO SU11] Using Card C... People LRI/ ERS SU10. Of the people you have regularly lived with, would you say that none, a few, some, most or all of them... None A Few Some Most All a. were employed or in school or training full time?.. 4 3 2 1 0 b. were involved in illegal activity?... 0 1 2 3 4 c. weekly got drunk or had 5 or more drinks in a day? 0 1 2 3 4 d. used any drugs during the past 90 days?... 0 1 2 3 4 e. shout, argue and fight most weeks?... 0 1 2 3 4 f. have ever been in drug or alcohol treatment?... 4 3 2 1 0 g. would describe themselves as being in recovery?... 4 3 2 1 0 GQ 3.2.3 ONT Standard 08/24/2012

Using number of people... SU11. During the past 12 months, how many people would you say you spend most of your time with at work, a training program or school?... [IF 0, GO TO SU12] Using Card C... People VRI/ ERS SU11. Of the people you have regularly worked or gone to school with, would you say that none, a few, some, most or all of them... None A Few Some Most All a. were employed or in school or training full time?.. 4 3 2 1 0 b. were involved in illegal activity?... 0 1 2 3 4 c. weekly got drunk or had 5 or more drinks in a day? 0 1 2 3 4 d. used any drugs during the past 90 days?... 0 1 2 3 4 e. shout, argue and fight most weeks?... 0 1 2 3 4 f. have ever been in drug or alcohol treatment?... 4 3 2 1 0 g. would describe themselves as being in recovery?... 4 3 2 1 0 Using number of people... SU12. During the past 12 months, how many people would you say you spend most of your free time with or hang out with?... [IF 0, GO TO CV1a] Using Card C... People SRI/ ERS SU12. Of the people you have regularly socialized with, would you say that none, a few, some, most or all of them... None A Few Some Most All a. were employed or in school or training full time?.. 4 3 2 1 0 b. were involved in illegal activity?... 0 1 2 3 4 c. weekly got drunk or had 5 or more drinks in a day? 0 1 2 3 4 d. used any drugs during the past 90 days?... 0 1 2 3 4 e. shout, argue and fight most weeks?... 0 1 2 3 4 f. have ever been in drug or alcohol treatment?... 4 3 2 1 0 g. would describe themselves as being in recovery?... 4 3 2 1 0 GQ 3.2.3 ONT Standard 08/24/2012

CV. Crime and Violence The next questions are about crime and violent behaviour. Using Card Q... Past Month 2 to 3 Months Ago 4 to 12 Months Ago 1+ Years Ago Never 4 3 2 1 0 CVScr/ CV1. QOLI When was the last time you... a. had a disagreement in which you pushed, grabbed or shoved someone?... 4 3 2 1 0 b. took something from a store without paying for it?... 4 3 2 1 0 c. sold, distributed or helped to make illegal drugs?... 4 3 2 1 0 d. drove a vehicle while under the influence of alcohol or illegal drugs?... 4 3 2 1 0 e. purposely damaged or destroyed property that did not belong to you?... 4 3 2 1 0 f. were involved in the criminal justice system, such as jail or prison, detention, probation, parole, house arrest or electronic monitoring?... 4 3 2 1 0 [IF CV1f IS LESS THAN 3, GO TO CV3] Please answer the next questions using the number of days. QCS CV2. During the past 90 days, on how many days have you been... a. on probation?... Days b. on parole?... Days c1. in juvenile detention?... Days c2. in jail or prison?... Days d. on house arrest?... Days e. on electronic monitoring?... Days PPI CV3. During the past 90 days, on how many days did you have an argument with someone else in which you swore, cursed, threatened them, threw something, or pushed or hit them in any way?... Days GQ 3.2.3 ONT Standard 08/24/2012

PPI CV4. During the past 90 days, on how many days were you involved in any activities you thought might get you into trouble or be against the law, besides drug use?... [IF 0, GO TO CV4b] Days PPI CV4a. On how many of these days were you involved in these activities (you thought might get you into trouble or be against the law)... 1. in order to support yourself financially?... Days 2. in order to obtain alcohol or other drugs?... Days 3. while you were high or drunk?... Days Please answer the next question using the number of times. QCS CV4b. During the past 90 days, how many times have you been arrested and charged with breaking a law? (Please do not count minor traffic violations.)... Times GQ 3.2.3 ONT Standard 08/24/2012

LS. Life Satisfaction The next questions are about how satisfied you feel with different parts of your life. After you hear each question, please tell me how satisfied you currently feel by using Card I and responding "very satisfied," "satisfied," "mixed," "dissatisfied," or "very dissatisfied." Very Satisfied Satisfied Mixed Dissatisfied Very Dissatisfied 5 4 3 2 1 LSI LS1. Currently, how satisfied are you with... g. the level of intimate relationships (e.g. marital, partner, sexual)?... 5 4 3 2 1 h. your family relationships?... 5 4 3 2 1 j. your general level of happiness?... 5 4 3 2 1 k. where you are living?... 5 4 3 2 1 m. how your life is going so far?... 5 4 3 2 1 n. your school or work situation?... 5 4 3 2 1 GQ 3.2.3 ONT Standard 08/24/2012

Z. End Thank you! That is all of the questions we have for you at this time. (Please enter the current time in Z1. If you went straight through, we will figure out how many minutes you took. If you took any breaks, please make sure that you record about how many minutes total it took you to do the assessment without including the time for the breaks. If continuing interview on another day, record the time for the first day in Z1d and record the total time in XADMh1a-d.) Z1. What time is it now?... : Time (HH:MM) b. Is it AM or PM?... AM/PM c. How many breaks did you take today?... Breaks d. Not counting breaks, how long did it take you to finish this?... Z2. Are there any other special issues we need to know about to help you (or help you come to treatment)? Do you have any additional comments or questions? v1. Minutes GQ 3.2.3 ONT Standard 08/24/2012

XADM.Administration For Staff Use Only Please document the following aspects of how the interview was administered. If there are more detailed comments elsewhere in the document, please be sure to summarize them in the additional comments section in XADMj or at least say where we can find them. a1. How were the questions administered? Yes No a. Self-Administered... 1 0 b. Orally Administered by staff... 1 0 c. Orally Administered by others... 1 0 z. Other (Please describe)... 1 0 a2. b. v. What was the mode of administration? Yes No a. Done with Pen and Paper... 1 0 b. Done on Computer... 1 0 c. Done on Telephone... 1 0 z. Other (Please describe)... 1 0 v. What was the primary language in which it was conducted? English using the English GAIN... 1 French using the English GAIN... 2 Other using the English GAIN (Please describe)... 99 v. c. Were there any indications that the client might have learning disabilities that would interfere with his or her ability to respond or participate in treatment or, in general, indications of developmental disabilities? No/none... 0 Minimal... 1 Moderate... 2 Major... 3 e. Was there any evidence of the following observed participant behaviours? Yes No 1. Depressed or withdrawn... 1 0 2. Violent or hostile... 1 0 3. Anxious or nervous... 1 0 4. Bored or impatient... 1 0 5. Intoxicated or high... 1 0 6. In withdrawal... 1 0 7. Distracted... 1 0 8. Cooperative... 1 0 GQ 3.2.3 ONT Standard 08/24/2012

For Staff Use Only g. What was the participant's location during the assessment? Treatment unit... 1 Specialized intake unit... 2 Correctional setting... 3 School... 4 Employment or work setting... 5 Home... 6 Probation or Parole Office... 7 Welfare or Child Protection Agency... 8 Research Office or Setting... 11 Other (Please describe)... 99 v. g1-5. Were there any problems providing a quiet, private environment? Yes No 1. Noise or other frequent distractions... 1 0 2. Divided attention or frequent interruptions... 1 0 3. Other people present or within earshot... 1 0 4. Police, guards, social workers or other officials present... 1 0 5. Speaker or telephone call monitoring... 1 0 h1. Was administration done over multiple days?... 1 0 [IF NO, GO TO XADMj] a. What is the final revision date (mm/dd/yyyy)?... / / 20 Month Day Year b. What is the total number of breaks across all sessions and days? (Include "1" for break in between multiple sessions.)... c. What is the total number of minutes spent doing the interview across all sessions and days?... d. What is the Staff ID [XSID] of the person finishing the interview?. j. Do you have any additional comments about the administration of the assessment or things that should be considered in interpreting this assessment? Be sure to document any critical collateral information that you think should be considered during interpretation (or cross-reference where it is documented). v1. GQ 3.2.3 ONT Standard 08/24/2012

Ontario Feasibility Study for Evidence-Based Practice of Screening, Assessment and Client Recovery Monitoring for Addiction Services Script for the Follow Up Interviews at 3 and 6 months You need: 1. The client s file (signed Letter of Information and Consent Form, Locator Form, etc.) 2. Follow Up Instruments (i.e. GAIN Q3 Standard walkthrough, 2 year calendar, calculator, and a blank paper copy of the GAIN Q3 Standard, and Return to Treatment Protocol) 3. ConnexOntario description/contact information 4. Suicide prevention guide 5. Project Protocol a) Before contacting client, check the calendar to determine if the client completed the GAIN-Q3-MI at baseline. If the calendar suggests the client has not completed the GAIN-Q3-MI, check with Nancy. b) Have the participant s research file with you, including his/her Letter of Information and Consent to Participate Form. After greeting the client on the phone, do the following: o Remind the participant that you are calling from the Health Research Group and that you are the person who is going to conduct the follow-up interviews (use first names only) o Remind the client about the follow-up process and what is entailed in terms of participation in the study. o Inform client that they can ask questions if they wish about the data collection process or about the study in general. c) Before beginning the interview make sure you ask the client about whether any Locator Form information has changed so that the client can be contacted for further treatment. If there are changes, please record them in notes section of the cover sheet. d) If the client DID complete the GAIN-Q3-MI, please go to (e) If the client DID NOT complete the GAIN-Q3-MI, please ask the following: I see here that you did not complete the GAIN-Q3-MI questionnaire with your counsellor. We need to figure out if this assessment tool would be useful for all clients in addiction agencies across Ontario. To help us with our decision-making process, can you tell me why you did not complete the questionnaire? (Please

take notes here) Go on to Return-to-Treatment Protocol e) Before administering tool: o Make sure the client is in a comfortable, quiet place where he/she is at ease to conduct the interview. o Ask the client if it is a good time to speak and if they would like to get some water and sit comfortably as the interview is going to take approximately 45 minutes to complete. Proceed with the interview by administering the GAIN-Q3 Standard; remember to ask the client where they are so that you can record it in the administration section of the questionnaire. You may have to allow for brief breaks if requested by the client. Also, be vigilant about the client s levels of alertness and tiredness. f) For the 6 Month Interview: After asking last question: That s the last question I have to ask you and this is actually our final interview. I just want to thank you very much for your help with this study and let you know that we really appreciate the time you gave us to answer these questions. We won t be calling you again, but if you have any questions about the study, please feel free to call or email us. You can also contact (Recruiting Agency Name) or ConnexOntario if you find you need additional help. Thank you again for your time.

g) Return-to-Treatment Protocol After administering the GAIN-Q3 Standard, ask the following questions: e.1: Are you still involved with (name of agency/program)? Yes No Please describe what type of services client is still involved with at the agency (if applicable): e.2: Do you feel that the treatment you have received has been helpful? Yes No e.3: Do you feel the need for additional assessment or treatment? Yes No Please describe (if applicable): e.4: Have you given any thought to getting additional support? Yes No Please describe (if applicable): e.5: How important is it for you to get additional support right now? Please describe: Interviewer Check: e1 e3 e4 Part Y Y Y H Y Y N H Y N Y H Y N N F N Y Y G N Y N G N N Y G N N N End interview

Part F f.1: The information you shared with us during this interview may help the agency better address your needs and in a timely manner. May we send your responses from the follow-up interview to the agency? Y/N f.2: Explain the purpose and function of ConnexOntario. Ask the client if they would like the toll-free telephone number. (1-800-565-8603) This is the end of the interview. Thank the client for his/her time. End ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Part G g.1: Some people find it helpful to go back to the same program. Would you like to return to the same program/agency? Y/N If Y, proceed to g.2 If N, skip to g.4 g.2: Is it OK for someone from the agency to contact you to make an appointment? Y/N If N, ask if they would prefer to contact the agency directly, and give them the phone number or any other contact details. g.3: The information you shared with us during this interview may help the agency better address your needs and in a timely manner. May we send your responses from the follow-up interview to the agency? Y/N Tell the client that someone from the Agency will contact them for an appointment. Thank the client for his/her time. End of interview. g.4: Can I facilitate contact with an organization that helps people find treatment options? [Explain the purpose and functions of ConnexOntario]. Y/N If Y, interviewer will keep client on the line, connect with ConnexOntario, and directly link the client to ConnexOntario. If N, ask the client if they would like the number for ConnexOntario (1-800-565-8603). This is the end of the interview. Thank the client for his/her time.

Part H h.1: Have you talked to your counsellor at about needing or wanting additional treatment or support? Y/N Please describe (if applicable): h.2: Recommend the client have a conversation with the programming staff/treating clinician h.3: The information you shared with us during this interview may help the agency better address your needs and in a timely manner. May we send your responses from the follow-up interview to the agency? Y/N h.4: (Use your judgment to see if the client would benefit from getting connected with Connex. If yes, please proceed, if no see bolded text below) Can I facilitate contact with an organization that helps people find treatment options? [Explain the purpose and functions of ConnexOntario]. Y/N Interview is complete. Thank client for his/her time. If Y, interviewer will keep client on the line, connect with ConnexOntario, and directly link the client to ConnexOntario. If N, ask the client if they would like the number for ConnexOntario. (1-800-565-8603) End ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Notes: Phone #s: Thames Valley: 519-673-3242 Fourcast: 705-876-1292 Belleville: 613-969-0077 Manitoulin: 705-368-0058 Rideauwood: 613-724-4881 Connex: 1-800-565-8603 Completed paper questionnaires should be kept in a locked cabinet.

References Canadian Centre on Substance Abuse. (2009). Substance abuse in Canada: Concurrent disorders. Ottawa ON: Canadian Centre on Substance Abuse. Centre for Addiction and Mental Health. (2008). Navigating screening options for concurrent disorders. Toronto ON: Centre for Addiction and Mental Health. Chestnut Health Systems. (2013). GAIN Q3. Retrieved from http://www.gaincc.org/gainq3 Cottler, L.B., Compton, W.M., Ben-Abdallah, A., Horne, M., & Claverie, D. (1996). Achieving a 96.6 percent follow up rate in a longitudinal study of drug abusers. Drug and Alcohol Dependence, 41(3), 209 217. Dennis, M., Scott, C.K., & Funk, R. (2003). An experimental evaluation of recovery management checkups (RMC) for people with chronic substance use disorders. Evaluation and Program Planning, 26, 339 352. Dennis, M. L., Titus, J. C., Diamond, G., Godley, S.H., TIms, F.M., Webb, C., Scott, C. K. (2002). The cannabis youth treatment (CYT) experiment: rationale, study design, and analysis plans. Addiction, 97, S16 S34. Garnick, D. W., Horgan, C. M., Lee, M. T., Panas, L., Ritter, G. A., Davis, S., Leeper, T., Moore, R., & Reynolds, M. (2007). Are Washington circle performance measures associated with decreased criminal activity following treatment? Journal of Substance Abuse Treatment, 33 (4), 341 352.

Garnick, D. W., Lee, M. T., Horgan, C. M., Acevedo, A., & the Washington Circle Public Sector Workgroup. (2009). Adapting Washington circle performance measures for public sector substance abuse treatment systems. Journal of Substance Abuse Treatment, 36(3), 265 277. Gilmore, D. (2012). Testing a model of participant retention in longitudinal substance abuse research: The moderating role of participant characteristics. (Master s thesis), Georgia State University, Georgia, United States. Godley, M. D., Godley, S.H., Dennis, M.L., Funk, R., & Passetti, L. (2002). Preliminary outcomes from the assertive continuing care experiment for adolescents discharged from residential treatment. Journal of Substance Abuse Treatment, 23, 21 32. Harris, A. H., Humphreys, K., Bowe, T., Tiet, Q., & Finney, J. W. (2010). Does meeting the HEDIS substance abuse treatment engagement criterion predict patient outcomes? Journal of Behavioral Health Services and Research, 37(1), 25 39. Health Canada. (2001). Best practice for concurrent mental health and substance use disorders. (Cat. No. H39 599/2001 2E/ISBN: 0 662 31388 7). Ottawa, ON: Health Canada. Health Canada. (2008). Drug Treatment Funding Program (DTFP) Framework. Retrieved from www.hc sc.gc.ga/hc ps/drugs drogues/dtfp pftt/framework cadre eng.php Katzman, R., Brown, T., Fuld, P., Peck, A., Schechter, R., & Schimmel, H. (1983). Validation of a short orientation memory concentration test of cognitive impairment. American Journal of Psychiatry, 140(6), 734 739.

Lennox, R. D., Dennis, M. L., Scott, C. K., & Funk. R, R. (2006). Combining psychometric and biometric measures of substance use. Drug and Alcohol Dependence, 83, 95 103. Lighthouse Institute. (2011). GAIN Q overview handout. Chestnut Health Systems. Retrieved October 24, 2011, from http://www.chestnut.org/li/gain/gain_q/index.html. McLellan, A.T., Chalk, M., & Bartlett, J. (2007). Outcomes, performance, and quality what s the difference? Journal of Substance Abuse Treatment, 32, 331 340. McLellan, A.T., McKay, J.R., Forman, R., Cacciola, J., & Kemp, J. (2005). Reconsidering the evaluation of addiction treatment: From retrospective follow up to concurrent recovery monitoring. Addiction, 100, 447 458. Pascoe, R., Rush, B., & Rotondi, N. K. (2013). Wait times for publicly funded addiction and problem gambling treatment agencies in Ontario, Canada. To be submitted to BMC Health Services Research. Rush, B.R., Castel, S, Brands, B., Toneatto, T. & Veldhuizen, S. Validation and comparison of diagnostic accuracy of four screening tools for mental disorders in people with substance use disorders. Manuscript submitted for publication. Rush, B.R., Castel., S., Somers, J., Duncan, D., & Brown, D. (2009). Systematic review and research synthesis of screening tools appropriate for children and adolescents with mental and substance use disorders. Toronto, ON: Centre for Addiction and Mental Health.

Rush, B.R. & Castel, S. (2011). Screening for mental and substance use disorders. In D. Cooper (Ed.), Care in mental health substance use (pp. 89 105). Mental Health Substance Use Book Series: Oxford, UK: Radcliffe Publishing Ltd. Rush, B., Martin, G., & Corea, L. (2008). Models of outcome monitoring for addictions services. Toronto, ON: Ministry of Health and Long Term Care. Rush, B., Martin, G., Corea, L., & Rotondi, N. K. (2012). Engaging stakeholders in review and recommendations for models of outcome monitoring for substance abuse treatment. Substance Use & Misuse, 47(12), 1293 1302. Scott, K. C. (2004). A replicable model for achieving over 90% follow up rates in longitudinal studies of substance abusers. Drug and Alcohol Dependence, 74, 21 36. Scott, C.K., Dennis, M.L., & Foss, M.A. (2005). Utilizing recovery management checkups to shorten the cycle of relapse, treatment reentry, and recovery. Drug and Alcohol Dependence, 78(3), 325 338. Shane, P., Jasiukaitis, P., & Green, R. S. (2003). Treatment outcomes among adolescents with substance abuse problems: The relationship between comorbidities and post treatment substance involvement. Evaluation and Program Planning, 26, 393 402. Simpson, D. D., Joe, G. W., Broome, K. M., Hiller, M. L., Knight, K., & Rowan Szal, G. A. (1997). Program Diversity and Treatment Retention Rates in the Drug Abuse Treatment Outcome Study (DATOS). Psychology of Addictive Behaviors, 11(4), 279 293.

Walker, R. (2009). Retention in treatment indicator or illusion: An essay. Substance Use and Misuse, 44(1), 18 22