How To Understand The Atop

Transcription

1 The Australian Treatment Outcomes Profile (ATOP): Clinical Outcome Monitoring for the Drug & Alcohol Sector Jennifer Holmes and Kristie Mammen Drug and Alcohol Services, South Eastern Sydney Local Health District AMHOIC Conference Brighton Beach, Sydney June 2013

2 Acknowledgements ATOP Project Team: A/Prof Nicholas Lintzeris; A/Prof Adrian Dunlop; Vi Hunt; Anni Ryan (South Eastern Sydney & Hunter New England Local Health Districts) Patients and staff of the services involved in the initial validation study: Jacaranda House, the Langton Centre and Newcastle OPT Clinics. Patients and staff of the services participating in Phase II: the pilot Implementation and Evaluation. MHDAO, Ministry of Health, D&A Research Grant 2009/2010 MHDAO, Ministry of Health, D&A Research Grant 2011/2012 UK National Treatment Agency, NHS

3 Overview Background Development and validation NSW pilot implementation and evaluation of the ATOP Where does the ATOP fit in an Outcome Based Framework for Drug and Alcohol Services?

4 THE AUSTRALIAN TREATMENT OUTCOMES PROFILE (ATOP): CLINICAL OUTCOME MONITORING FOR THE DRUG & ALCOHOL SECTOR Background

5 Implementation barriers Few examples of successful implementation in D&A services Why is implementation so difficult? length of the instruments + inadequate attention to training + excessive data entry requirements + poor feedback for patients, clinicians and administrators + Perception of outcome monitoring as a bureaucratic process = limited utility and resistance Lawrinson, et al, 2009; Smith et al, 1998; Teruya et al, 2006; Tobin & Hickie, 1998

6 Treatment Outcomes Profile (TOP) - UK Treatment Outcomes Profile (TOP) a validated tool introduced by the NTA in the UK in 2007 (Marsden et al., 2008) introduced across all NHS funded D&A services demonstrating the effectiveness of drug treatment for heroin and cocaine users (Marsden et al., 2009) comparing the relative effectiveness of treatment services (Marsden et al., 2012).

7 Developing the ATOP Modified for Australian conditions = ATOP Reflecting substances commonly used in Australia Made more intuitive to complete Validated with Australian treatment sample Emphasis on balancing clinical utility with service evaluation and research potential: systematic, documented and ongoing client assessment feedback to clients on progress care planning and motivation clinical handover service evaluation - do our clients get better? quality assurance and research tool

8 What is the ATOP? A clinician administered 1- page validated tool Self-report measures across 2 key domains in the preceding 4 weeks Substance use Quantity & frequency of use IDU Health & well being Global ratings (0-10) of physical & mental health, quality of life Housing, employment & study, arrests, violence, child protection

9 THE AUSTRALIAN TREATMENT OUTCOMES PROFILE (ATOP): CLINICAL OUTCOME MONITORING FOR THE DRUG & ALCOHOL SECTOR Validation and Feasibility Study Ryan, A., Holmes, J., Hunt, V., Dunlop, A., Mammen, K., Holland, R., Sutton, Y., Sindhusake, D., Rivas, G., and Lintzeris, N. (Submitted). Validation and implementation of the Australian Treatment Outcomes Profile in specialist Drug and Alcohol settings.

10 Method Concurrent validity and inter-rater reliability: ATOP administered as part of routine care by clinic staff at 3 month intervals Research interview within 72 hrs of last ATOP: gold standard instruments + repeat ATOP Implementation and feasibility issues (incl. data management): clinician, service manager and patient perspectives via satisfaction surveys and focus groups

11 Results Age and gender of sample were reflective of treatment population Inter-rater reliability (n=103) Good or excellent agreement between researcher and clinician ratings on all continuous items except standard drinks (ICC). Most dichotomous items had good or excellent agreement (Kappa). Exceptions 1 crime item, 1 IDU item combined in new item. Concurrent validity (n=131) Excellent agreement between ATOP and Gold star instrument ratings on all dichotomous items (Kappa). Strong agreement was detected between all continuous items (PPC).

12 Results: Clinician Ratings of clinical utility easy to administer: 80% agreed appropriate for my client population: 85% agreed format & style easy to understand: 85% agreed length appropriate for routine practice: 70% agreed appropriate for my setting: 65% agreed, 30% unsure useful in developing a case plan: 65% agreed, 25% unsure useful for identifying important problems 45% agreed, 35% ambivalent, 20% disagree happy to use as part of regular client reviews 45% agreed, 30% ambivalent, 25% disagree *** Disliked crime questions***

13 Results: Client ratings of usefulness ATOP questions were easy to understand - 93% agreed Helpful way of looking at how well treatment is working for me - 85% agreed Helpful to have this same review every few months - 85% agreed Length of the ATOP was about right - 90% agreed

14 Summary ATOP is a valid instrument for measuring treatment outcomes in an Australian opioid maintenance treatment population ATOP is compatible with routine clinical practice ATOP can feasibly be implemented as part of routine clinical practice in public OPT Clinics Investigate feasibility of ATOP in other treatment settings

15 THE AUSTRALIAN TREATMENT OUTCOMES PROFILE (ATOP): CLINICAL OUTCOME MONITORING FOR THE DRUG & ALCOHOL SECTOR Pilot Implementation

16 Phase II: Pilot Implementation & Evaluation Overview Feasibility, utility and acceptability of ATOP, particularly in services beyond Opioid Pharmacotherapy Units Establishing business rules for different service types Initial database (MS Access) with individual and service reports Training Package Train the trainer workshops Presentations and user manual

17 Participants 12 NSW Local Health Districts

18 Instrument Feedback Ongoing refinement of ATOP Survey participants at training workshops Consultation teleconferences and site visits Examples of questions refined Drug quantities included for all drugs Child protection Injecting Homelessness

19 Clinical Utility and Acceptability

20 Clinical Utility and Acceptability

21 Implementation Feedback Dovetails existing case management review schedules, assisting implementation Provides structure to case management reviews TLFB method challenging initially, but easier overtime Ensures routine comprehensive reviews of clients Counselling services had more varied feedback depending on the therapeutic framework used by the counsellor and the processes more challenging cf OPT (eg. still working on followup frequency; reminders for follow-up due) Data entry challenging for a number of services who/when Burden of multiple measures Inclusion of quantities helpful

22 Training Developed Train-the-trainer workshops and resources >60 participants at each 2 workshops Positive feedback from attendees on training and resources Following the training I feel prepared to: 100% Strongly Agree 90% 80% 70% Somewhat agree 60% 50% Neither agree nor 40% 30% disagree 20% Somewhat Disagree 10% 0% Strongly Disagree

23 THE AUSTRALIAN TREATMENT OUTCOMES PROFILE (ATOP): CLINICAL OUTCOME MONITORING FOR THE DRUG & ALCOHOL SECTOR Outcome based framework for Drug and Alcohol Services

24 What is a performance framework? Framework for measuring the performance of health services Three broad components: Governance / quality framework: accredited health services, appropriately skilled & credentialed workforce, efficiently delivering treatment according to good practice standards / guidelines / model of are Measures of throughput: how many services provided (volume) Outcome framework: measures the clinical outcomes associated with treatment Historically poorly done for DA services

25 Why so difficult to develop outcome framework in Drug and Alcohol? Chronic relapsing condition & patients usually require multiple types of services in order to achieve long term outcomes e.g. detox followed by counselling, ± medications, self help etc... to which service do you ascribe outcomes? Different types of services have different objectives & outcomes e.g. detox outcomes OTP outcomes Different objectives & expected outcomes within service types counselling the abstinent patient to prevent relapse counselling active user Different complexity of patients Broad range of concurrent conditions (medical, psychiatric, social, cultural) that impact upon resources required & outcomes achieved by DA treatment DA treatment may only have minimal immediate impact upon secondary outcomes (e.g. employment), or outcomes may be more related to other treatment / services (e.g. surgery, housing support, mental health services) How to factor not only problems, but patient s resources to address problem (social supports, existing treatment or social services for other problems)

26 Key Principles An outcome framework should: 1. Describe client characteristics: who is being treated? 2. Are treatment services delivered well 3. Are treatment services achieving good outcomes for clients

27 3. Are services achieving good outcomes for clients Can we globally rate whether each service episode had a good, indifferent, or poor outcome for the client Treatment outcomes need to reflect a combination of Process outcomes (e.g. treatment completion, drop out, adverse events/complications) Clinical outcomes - for D&A treatment need to reflect Primary outcomes: measure of primary substance use (frequency / quantity recent use) Secondary outcomes: broader range of measures Other substance use Measures of physical and mental health, social functioning (employment, education, violence), QOL The mix of process and clinical outcomes will vary for each service episode type

28 Clinical Outcome Measurement ATOP ATOP substance use domain provides data for comparing substance use frequency over time ATOP health and wellbeing domain provides data for comparing changes over time

29 Heavy Drug Use: Primary Drug used more than 50% of days in last 4 weeks Primary Domain Poor clinical outcome Indeterminate clinical outcome Good clinical outcome Primary Drug use Increase * No increase or a reduction * No increase or no reduction * No increase or no reduction * Reduction * Secondary Domain And And And Secondary drug use, Physical, Psychological, Quality of life, Social Significant deterioration in 3 out of 5 of the secondary domains No major change in 3 out 5 domains Major improvement in 3 out of 5 secondary domains * in days used in last 28 days by the agreed change ratio

30 Abstinent or Light Drug Use: Primary Drug used less than 50% of days in last 4 weeks Indeterminate clinical outcome Primary Domain Poor clinical outcome Good clinical outcome Primary Drug use Use on more than 14 out of 28 days Use on less than 14 out of 28 days No use on more than 14 out of 28 days No use on more than 14 out of 28 days Secondary Domain And And And Secondary drug use, Physical, Psychological, Quality of life, Social Deteriorated in 3 out 5 secondary domain No major change in 3 out 5 domains Improvement in 3 out of 5 secondary domain

31 Next Steps Build data items into clinical information system Further testing of model with large data sets Define a composite measure for each treatment type based on process, client complexity and clinical outcome

32 Questions and Comments

33 For more information: contact Kristie Mammen, ATOP Project Manager Jennifer Holmes, Project Manager Drug and Alcohol Information Systems A/Prof Nicholas Lintzeris, Chief Addiction Medicine Specialist, MHDAO, NSW Ministry of Health