A Phelps, J Cooper and K Densley Australian Centre for Posttraumatic Mental Health, University of Melbourne.

Transcription

1 Ms Andrea PHELPS A Phelps, J Cooper and K Densley Australian Centre for Posttraumatic Mental Health, University of Melbourne. Using Mental Health Outcome Measures to Support Quality Assurance of DVA-Funded PTSD Programs The Australian Centre for Posttraumatic Mental health has been responsible for the quality assurance of DVA funded treatment programs for veterans with Posttraumatic Stress Disorder (PTSD) since This presentation will explain the nature of these programs including their history and purpose, the mental health outcome measures that are routinely used by the programs, the treatment outcomes achieved nationally, and the way in which these outcome measures, combined with a number of other sources of information, are used to support quality assurance of the programs. Recent revisions to the accreditation process in anticipation of a broader range of treatment models arising from DVA s move towards more community based treatment for PTSD, will also be described. DVA funded PTSD programs Background and history The diagnosis of Posttraumatic Stress Disorder entered the psychiatric nomenclature in 1980 with impetus from the increasing presentations of posttraumatic stress symptoms in U.S. Vietnam veterans returning from the war in the 1970 s and the women s movement raising awareness of posttraumatic stress problems in women subject to domestic violence and sexual assault. Group treatment programs were established across the U.S. for Vietnam veterans with posttraumatic mental health problems in the late 1970 s but similar moves were not made in Australia until the early 1990 s. In 1994 representatives from the Australian Department of Veterans Affairs, ex-service organisations and treatment providers visited the U.S. programs before establishing the first treatment programs in Australia in in Melbourne and Sydney. In 1995 ACPMH was established (then called the National Centre for War-related PTSD) with responsibility for supporting the development of programs throughout Australia. The programs were set up to meet the expressed need of Vietnam era veterans who were presenting to mental health treatment services with posttraumatic mental health problems, as well as the anticipated needs of other Vietnam veterans who had not yet sought treatment. On the basis of an Australian epidemiological study (O Toole et al., 1996) the lifetime prevalence of PTSD in Vietnam era veterans could be expected to be around 20%; or of the Australian servicemen who served. As programs became established, ACPMH had an ongoing role in instituting quality assurance processes to ensure best practice care was provided. To this end, accreditation guidelines and evaluation protocols were developed and formal program accreditation began in Development of the Australian model The U.S. PTSD programs involved long stay inpatient treatment which placed great importance on the therapeutic milieu. Although popular with both veterans and staff, these programs were unfortunately associated with rather poor treatment outcomes (Johnson et al.,1996). A second existing treatment model was the Koach project in Israel, established to treat the chronic PTSD of veterans of the Lebanon War (Solomon et al., 1992; Solomon, Shalev et al., 1992). In this model, the inpatient stay was relatively brief (four weeks) followed by indefinite outpatient follow-up.

2 In line with the U.S. programs, the Koach model emphasised the group milieu but in addition, incorporated cognitive behavioural approaches to addressing the veterans avoidance behaviour. Unfortunately, also in line with the U.S. experience, the Koach program was perceived as very positive by veterans and staff but objectively had poor outcomes (Solomon, Shalev et al., 1992). The initial treatment model adopted by Australian programs comprised a 4 week inpatient stay followed by an outpatient phase of 1 day per week for 8 weeks. This was designed to provide the opportunity for skills transfer into veterans normal environment and thus to facilitate veterans re-integration into the community (Creamer et al., 1996). To address the prevailing problems of veterans with PTSD and common co-morbidities using best practice treatment, the programs were required to have: a multidisciplinary team a cognitive behavioural orientation a group program with core components including o psychoeducation about PTSD and its treatment o trauma focussed work o symptom management in areas such as anxiety and depression o anger management groups o substance abuse and addictive behaviours o interpersonal, problem-solving and communication skills; o physical health and lifestyle issues o relapse prevention o education and support to veterans partners individual therapy attention to discharge planning and appropriate follow-up Program outcome measures Dimensions rated Program outcome measures assess change in PTSD and common co-morbid mental health problems, marital adjustment and quality of life. The instruments used include: PTSD checklist (PCL; Weathers et al,. 1993), Hospital Anxiety and Depression Scale (HADS; Zigmond & Snaith, 1983), Dimensions of Anger Reaction (DAR; Novaco, 1975), Alcohol Use Disorders Identification Test (AUDIT; Saunders et al. 1993), Abbreviated Dyadic Adjustment Scale (ADAS; Sharpley & Rogers 1984; Spanier 1976) and Brief World Health Organisation Quality of Life Instrument (WHOQoL-Bref; WHOQOL Group, 1998). Time points Participants are asked to complete a battery of self-report instruments at intake, discharge, 3 month follow-up and 9 month follow-up. Use of effect size Effect size, the standardised difference between two means, is used as the measure of change in preference to statistical significance, because it is independent of sample size and provides a stronger assessment of the clinical importance of the change. The data base contains over 4000 cases and so even small differences across time would likely be identified as statistically significant. 2

3 An effect size of 0.2 is considered small, 0.5 moderate and 0.8 strong. Interpretation of effect size needs to take account of condition severity and end state functioning. Where symptoms are less severe at intake, the scope for improvement is less and smaller effect sizes are to be expected. In these circumstances, the end state scores are an important indication of the benefit veterans have derived from the program in terms of their end state functioning compared to other participants of other programs across the country. An early treatment outcome study (Creamer et al., 1999) was promising, with results from 419 program participants indicating modest improvements in core PTSD symptoms, anxiety, depression, alcohol abuse and anger on self report measures, from intake to 3 and 9 month follow-up (see Table 1). In most cases, treatment gains were apparent at 3 but maintained at the 9 month follow-up point. Table 1: Outcome data for PTSD programs reported in Creamer et al Intake 3 9 Measure Mean SD Mean SD Mean SD PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) Anger (range 0-8) Effect size (ES) Intake to 9 mths Over time a range of treatment models have evolved including residential, day program and regional outreach and variations in number of treatment days between 20 and 30 days. Importantly, a treatment outcome study comparing inpatient and day programs (Creamer et al., 2002) found comparable effects with different models of treatment. Program participants have also changed over time with the programs initially targeting Vietnam era veterans but progressively including older (WWII and Korean) veterans as well as younger veterans and peacekeepers. According to our data (see Table 2), older veterans tend to have less severe PTSD and comorbid symptoms at intake, notably alcohol use, and smaller treatment effects compared to Vietnam era veterans. Importantly, with lower intake scores older veterans leave the programs with lower end state scores on all measures and the programs are of clinical benefit. 3

4 Table 2: Intake scores and treatment outcomes for older veterans compared to Vietnam veterans Measure Older Veterans Vietnam veterans Mean score Effect Size Mean score Effect Size Intake 9 mth intake to 9 Intake 9 mth intake to 9 Clinician Administered 66.2 N/A N/A 82.2 N/A N/A PTSD Scale (CAPS) PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) DAR (range 0-56) An early investigation of treatment outcomes for peacekeepers (Forbes et al., 2005) based on 3 month follow up data for 66 peacekeepers (deployments including Timor, Cambodia, Rwanda and Somalia) compared to 63 Vietnam veterans undertaking programs in the same facilities, indicated treatment gains similar to those seen for Vietnam veterans on anxiety depression and alcohol use, but weaker treatment effects on PTSD symptoms and anger. This data is presented in Table 3. Table 3: Outcome data for peacekeepers reported in Forbes et al Measure Peacekeepers Vietnam veterans Mean score Effect Size Mean score Effect Size Intake 3 mth intake to 3 Intake 3 mth intake to 3 PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) War Stress Inventory (Anger) More recent data that includes 318 peacekeepers combined with veterans of the war in Iraq and Afghanistan (see Table 4) confirms this trend with peacekeepers and younger veterans presenting with more severe PTSD (clinician rated) and anger (self-rated) but otherwise a similar symptom profile to Vietnam veterans. In terms of treatment outcomes, while peacekeepers and younger veterans derive clinically important improvements from the programs the magnitude of change on measures of PTSD, anger and depression are smaller than those seen with Vietnam veterans. 4

5 One may have expected better results for peacekeepers and younger veterans given the (relatively) earlier intervention and so these outcomes are somewhat disappointing. The treatment of younger veterans and peacekeepers has been identified as one of the significant challenges facing programs. Table 4: Outcome data for peacekeepers and younger veterans compared to Vietnam veterans Measure Peacekeepers and younger veterans Vietnam veterans Mean score Effect Size Mean score Effect Size Intake 9 mth intake to 9 Intake 9 mth intake to 9 Clinician 88.1 N/A N/A 82.2 N/A N/A Administered PTSD Scale (CAPS) PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) DAR (range 0-56) Use of treatment outcome data in the quality assurance process The outcome data collected from programs across Australia is used in the quality assurance for individual programs in three ways: an individual program s outcomes are compared to the national averages as a measure of accountability; an individual program s outcomes are compared to their previous outcomes as a means of providing feedback on service initiatives or other changes; and disparities between a program s outcomes and national outcomes are used as service development opportunities. At the time of accreditation, and at other times if requested, programs receive reports on their outcome data against national benchmarks. The data presented includes the national and program means on all measures at intake, discharge, 3 and 9 month follow-ups, along with the magnitude of change between time points expressed as an effect size. A sample report is presented in Table 5. 5

6 Table 5: Sample report to programs Measure National Mean Effect size (Intake Intake 3mths 9mths to 9mths) N Program Mean Intak e 3mths 9mths PCL TOTAL Intrusion Avoidance Arousal AUDIT TOTAL Hazardous Dependency Harmful HADS Anxiety Depression ADAS WHOQoL Physical Psychological Social Environment Effect size (Intake to 9mths) N Importantly, in the program accreditation process, outcome data is used as one source of information about quality, with any issues or concerns arising, checked against information from other sources. These other sources include program staff s description of their treatment model and practices, feedback from past participants and key stakeholders such as the Veterans and Veterans Families Counselling Service (VVCS). For example, where there is lower than average reduction in avoidance symptoms the program s approach to exposure therapy will be specifically examined; or if a program has high scores on alcohol use at intake and follow-up, an important focus of accreditation will be alcohol assessment, preparation and intervention. As an outcome of accreditation, specific recommendations for modifications to programs will be made with a view to promotion of ongoing service improvement. At subsequent accreditations, the specific outcome data pertaining to a service improvement recommendation will be examined along with staff reports and stakeholder feedback, to monitor changes made and any effect on treatment outcomes. If, during the examination of all sources of information in the course of accreditation, concern remains about the program s performance, a meeting is arranged to discuss the concerns. Again in this process, the outcome data is not used alone as an indicator of quality but triangulated with information from program staff and key stakeholders including veterans, family members, VVCS and DVA. 6

7 Maintaining quality during a time of change There are a number of challenges in ensuring the quality of PTSD treatment for veterans into the future. The two main issues of concern facing the existing PTSD programs appear to be maintaining quality during a period of declining demand for treatment and addressing the different needs of the younger veterans coming through. In addition, as DVA moves to purchasing PTSD treatment from a broader range of community based providers, there rises a challenge in ensuring the quality assurance of those treatments. Declining numbers Figure 1 shows the participation rate in programs over time Number of participants treated by financial year and state ACT NSW Qld SA Tas Vic WA Total Participants /96 96/97 97/98 98/99 99/00 00/ Figure 1: Participation rate in PTSD programs by financial year Clearly, the demand for PTSD programs has passed through a peak in 2000/2001 and has declined since that time, with some suggestion of a levelling off in that decline in recent years. The declining demand places pressure on program resourcing, and in some instances, brings the viability of programs into question. The number of programs in Australia reached a peak of 19 in 2002 with programs available in all states and territories except Northern Territory. With declining demand, there are currently 12 programs. Potential threats to quality arising from this declining demand include the pressure to fill a cohort with one or more veterans who are not be ready or suitable for intensive programmatic treatment, and the loss of a coherent and experienced staff team arising from staff being recruited from other hospital programs or teams for the purpose of running a PTSD program, and then being dispersed back to their core positions. 7

8 Younger veterans with different needs The key issues regarding the younger group of peacekeepers and veterans appear to be the need to increase accessibility of treatment and the need to tailor treatment to meet their particular needs. The 2004 Pathways to care study (Hawthorne et al., 2004) showed that 30% of veterans recently compensated for a mental health disability were not receiving treatment and that of those who had sought mental health care, 20% were dissatisfied. Peacekeepers were overrepresented in this group and indeed 42% of peacekeepers reported that they had stopped treatment. Barriers to health care included difficulty accepting they had a health problem, previous unsuccessful treatment, uncertainly about what was available and services not available locally. Anecdotal reports from PTSD programs involved in treating peacekeepers and younger veterans indicate that they generally present with more anger, social disruption and substance use issues, than their Vietnam veteran counterparts. A small number of these programs have begun to develop innovative approaches in an attempt to better meet needs. There would be benefit and efficiencies if this was approached on a national level as was done in the establishment of the early PTSD programs. In this way, the clinical experience of Australian programs could be combined with international experience in treating younger veterans and the limited available research, to develop best practice treatment guidelines for programs for younger veterans. Move towards community based treatment In addition to specialised PTSD treatment programs, veterans with PTSD currently receive care from a range of community based psychiatrists, psychologists, counsellors and general practitioners. The quality assurance mechanisms applied to the PTSD treatment programs do not currently extend to these other service providers. However as DVA moves towards purchasing more of its specialist treatment for veterans with PTSD and other mental health conditions from a broad range of community based providers, quality assurance processes will ideally be in place. In anticipation of this, ACPMH has reduced the intensity of its PTSD program accreditation processes, moving from the previous method of site visits, to a written self-assessment process. Importantly, the same sources of information are used; that is, program outcome data, staff reports on program content and operational management issues and feedback from veterans, partners, VVCS and DVA. The revised accreditation model incorporates the following components: access and targeting; quality of service; clinical processes of intake, assessment, case management and discharge; treatment; operational management; and outcome monitoring. The most recent accreditation guidelines and self-assessment documentation have been developed to apply to the current PTSD programs but be applicable across a broader range of community based treatment providers. 8

9 References Creamer, M., Morris, P., Biddle, D. & Elliot, P. (1999) Treatment outcome in Australian veterans with combtrelated posttraumatic stress disorder: A cause for cautious optimism? Journal of Traumatic Stress 12 (4), Creamer, M., Forbes, D., Biddle, D. & Elliot, P. (2002) Inpatient versus day hospital treatment for chronic, combat-related posttraumatic stress disorder: A naturalistic comparison. The Journal of Nervous and Mental Disease 190 (3), Forbes, D., Bennett, N., Biddle, D., Crompton, D., McHugh, A., Elliot, P. & Creamer, M. (2005) Clinical presentations and treatment outcomes of peacekeeper veterans with PTSD: Preliminary findings. American Journal of Psychiatry 162, Hawthorne, G., Hayes, L., Kelly, C. & Creamer, M. (2004) Pathways to care in veterans recently compensated for a mental health condition. Australian Centre for Posttraumatic Mental Health report commissioned by the Department of Veterans Affairs. Johnson, D. R., Rosenheck, R., Fontana, A., Lubin, H., Charney, D., & Southwick, S. (1996). Outcome of intensive inpatient treatment for combat-related posttraumatic stress disorder. American Journal of Psychiatry, 153, Novaco, R. (1975). Dimensions of anger reactions. Irvine, CA: University of California. O'Toole, B. I., Marshall, R. P., Grayson, D. A., Schureck, R. J., Dobson, M., Ffrench, M., Pulvertaft, B., Meldrum, L., Bolton, J., & Vennard, J. (1996). The Australian Vietnam veterans health study: III. Psychological health of Australian Vietnam veterans and its relationship to combat. International Journal of Epidemiology, 25, Saunders, J.B., Aasland, O.G., Babor, T.F., De la Fuente, J.R. & Grant, M. (1993) Development of the Alcohol Use Disorders Identification Test (AUDIT: WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption. Addiction, 88(6), Sharpley, C. F. & Rogers, H.J. (1984) Preliminary validation of the Abbreviated Spanier Dyadic Adjustment Scale: Some psychometric data regarding a screening test of marital adjustment. Educational and Psychological Measurement, 44, (4), Spanier, G.B. (1976) Measuring dyadic adjustment new scales for assessing quality of marriage and similar dyads. Journal of Marriage and the Family. 38(1), Weathers, F. W., Litz, B. T, Herman, D. S., Huska, J. A., & Keane, T, M. (1993). The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility. Paper presented at 9th Annual Conference of the International Society for Traumatic Stress Studies, San Antonio, TX. WHOQoL Group (1998) Development of the World Health Organization WHOQOL-BREF Quality of Life Assessment. Psychological Medicine 28, Zigmond, A., & Snaith, R. (1983). The Hospital Anxiety and Depression Scale. Ada Psychiatrica Scandinavia, 67,

10 Tables and Figures Table 6: Outcome data for PTSD programs reported in Creamer et al Intake 3 9 Measure Mean SD Mean SD Mean SD PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) Anger (range 0-8) Effect size (ES) Intake to 9 mths Table 7: Intake scores and treatment outcomes for older veterans compared to Vietnam veterans Measure Older Veterans Vietnam veterans Mean score Effect Size Mean score Effect Size Intake 9 mth intake to 9 Intake 9 mth intake to 9 Clinician Administered 66.2 N/A N/A 82.2 N/A N/A PTSD Scale (CAPS) PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) DAR (range 0-56) Table 8: Outcome data for peacekeepers reported in Forbes et al Measure Peacekeepers Vietnam veterans Mean score Effect Size Mean score Effect Size Intake 3 mth intake to 3 Intake 3 mth intake to 3 PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) War Stress Inventory (Anger)

11 Table 9: Outcome data for peacekeepers and younger veterans compared to Vietnam veterans Measure Peacekeepers and younger veterans Vietnam veterans Mean score Effect Size Mean score Effect Size Intake 9 mth intake to 9 Intake 9 mth intake to 9 Clinician 88.1 N/A N/A 82.2 N/A N/A Administered PTSD Scale (CAPS) PCL (range 17-85) HADS Anxiety HADS Depression AUDIT (range 0-40) DAR (range 0-56) Table 10: Sample report to programs Measure National Mean Effect size (Intake Intake 3mths 9mths to 9mths) N Program Mean Intak e 3mths 9mths PCL TOTAL Intrusion Avoidance Arousal AUDIT TOTAL Hazardous Dependency Harmful HADS Anxiety Depression ADAS WHOQoL Physical Psychological Social Environment Effect size (Intake to 9mths) N 11

12 Number of participants treated by financial year and state ACT NSW Qld SA Tas Vic WA Total Participants /96 96/97 97/98 98/99 99/00 00/ Figure 2: Participation rate in PTSD programs by financial year 12