Military Veterans Improving Access to Psychological Therapies (MV IAPT) Services

Transcription

1 PSSRU Military Veterans Improving Access to Psychological Therapies (MV IAPT) Services Final Report of an Independent Evaluation to Pennine Care NHS Foundation Trust Paul Clarkson, Clarissa Marie Giebel and David Challis Discussion Paper M277 August 2013 Not publicly available PERSONAL SOCIAL SERVICES RESEARCH UNIT The University of Manchester Dover Street Building University of Manchester Oxford Road Manchester M13 9PL Tel: The University of Kent at Canterbury Cornwallis Building University of Kent at Canterbury Canterbury Kent CT2 7NF Tel: / London School of Economics London School of Economics Houghton Street London WC2A 2AE Tel: Website:

2

3 MILITARY VETERANS IMPROVING ACCESS TO PSYCHOLOGICAL THERAPIES (MV IAPT) SERVICES FINAL REPORT OF AN INDEPENDENT EVALUATION TO PENNINE CARE NHS FOUNDATION TRUST August 2013 Authors Dr Paul Clarkson Research Fellow PSSRU, University of Manchester Clarissa Marie Giebel Project Assistant PSSRU, University of Manchester Professor David Challis Director PSSRU and Professor of Community Care Research University of Manchester PSSRU at the University of Manchester

4 PSSRU at the University of Manchester

5 MILITARY VETERANS IMPROVING ACCESS TO PSYCHOLOGICAL THERAPIES (MV IAPT) SERVICES FINAL REPORT OF AN INDEPENDENT EVALUATION TO PENNINE CARE NHS FOUNDATION TRUST CONTENTS Acknowledgements... 7 Executive summary... 9 Chapter 1: Introduction Background to the services Aims of the evaluation Structure of this report Chapter 2: Methodology A purposive literature review Characteristics and outcomes of the Military Veterans services Costs and costeffectiveness of the Military Veterans services Cost and outcome relationships Interviews with key service providers The analyses in this report Chapter 3: Literature review Characteristics and needs of military veterans Service models providing psychological or social assistance to veterans Previous nonveteran IAPT services The costs and costeffectiveness of IAPT services Summary of the literature and directions for this study Chapter 4: Findings I Characteristics and outcomes Findings from the clinical veterans IAPT service Findings from the social Live at Ease service Summary of evidence relating to characteristics and outcomes Chapter 5: Findings II Costs and costeffectiveness The costs of the clinical veterans IAPT service The costs of the social LAE service Costeffectiveness Summary of costeffectivess evidence Chapter 6: Findings III Cost/outcome relationships Predictors of outcomes Predictors of costs Summary of predictors of cost/outcome relationships PSSRU at the University of Manchester

6 Chapter 7: Qualitative service evaluation Staff s service experiences Veterans service experiences Summary of views and experiences Chapter 8: Summary and conclusions Summary of the costs and effectiveness of the services Interpretation of these findings Overall conclusions References Appendix 1: Modelling of the time commitment of staff for the live at ease service Appendix 2: Semi structured interview schedule to explore the MV IAPT design and initial implementation/setup of the service Boxes LIST OF BOXES, FIGURES AND TABLES 3.1 Key IAPT reference documents 42 Figures 4.1 Patient pathways through the clinical service Therapies received (A) by those who completed treatment and (B) by all veterans Referrals per month to the clinical service Overall outcomes for the clinical service, pre and posttreatment Distribution of costs of the MV IAPT clinical service, therapy contacts Distribution of costs of the Live at ease caseworkers in providing the service to particular case types (average hours per case) Costeffectiveness plane incremental costeffectiveness ratios based on 1000 bootstrap replicates for total costs of the clinical service Costeffectiveness plane incremental costeffectiveness ratios based on 1000 bootstrap replicates for running costs of the clinical service Costeffectiveness acceptability for total and running costs of the clinical service Costeffectiveness plane incremental costeffectiveness ratios based on 1000 bootstrap replicates for ongoing costs of the social LAE service Costeffectiveness acceptability for ongoing costs of the LAE service 84 PSSRU at the University of Manchester

7 Tables 2.1 The effects and costs associated with a comparison group for the military veteran s clinical service, modelled data Referrals by Commissioning Groups Characteristics of all patients with one or more sessions Characteristics of patients who had unattended sessions ( DNAs ) with those who attended ( nondnas ) Characteristics of patients assessed in the clinical service compared with published data Clinical outcomes improvement, remission and recovery rates for subgroups of patients assessed in the clinical service Clinical outcomes improvement of patients assessed in the clinical service compared with published data Clinical outcomes for early service leavers, physically disabled veterans and reservists with two or more sessions Clinical outcomes for substance and alcohol misusers with two or more sessions Clinical outcomes for different service receipt for veterans with two or more sessions Unemployment and sick pay for subgroups assessed in the clinical service Unemployment and sick pay for those who attended two or more sessions Characteristics of veterans assessed through Live at Ease Outcomes of the social service (GHQ12) improvement and recovery rates for clients assessed through LAE as against other services in the published literature Investment costs of the clinical MV IAPT service The time commitment of Live at Ease caseworkers in providing the service to particular case types (average hours per case) Costs ( 2010/11) per participant between intervention and comparator groups for the clinical service Outcomes on average between intervention and comparator groups for the clinical service Costs ( 2010/11) per participant between intervention and comparator groups for the social LAE service Outcomes on average between intervention and comparator groups for the LAE services Production function individual characteristics predicting changes in outcome between baseline and follow up on the PHQ9 for the clinical service Production function individual characteristics predicting changes in outcome between baseline and follow up on the GAD7 for the clinical service Production function individual characteristics predicting changes in outcome between baseline and follow up on the WSAS for the clinical service Cost function individual characteristics predicting levels of expenditure of the clinical MV IAPT service 91 PSSRU at the University of Manchester

8 PSSRU at the University of Manchester

9 ACKNOWLEDGEMENTS We would like to thank a number of people without whom this evaluation would not have been undertaken or completed. Reagan Blyth, Associate Director of Quality Assurance and Research and Paul Duthie, Research and Evaluation Manager at Pennine Care NHS Foundation Trust, managed the commissioning processes and ongoing data transfer for the evaluation on behalf of the clinical and social service. Managers and senior practitioners in the clinical veterans IAPT service have been generous with their time and committed to making available data to the evaluation team. Dr Alan Barrett, Principal Clinical Psychologist and Clinical Lead for the Military Veterans' IAPT Service (North West) and Helen Lambert, Military Veterans Service Coordinator, were central in giving us an overview of the entire service and in liaising with us regarding the availability of the routine clinical data on which the evaluation depended. We would wish also to thank other members of the clinical service who contributed to the conduct of the evaluation, particularly the practitioners themselves who contributed during an away day workshop, where preliminary findings were disseminated. Others associated with the clinical service have offered their time and contributed to this report through their views and steered the research team through the data and answered queries. We would like to thank in this regard, Andy Bacon, at the time Associate Director and Armed Forces Lead, NHS North West, Ann Touray, NHS North West and Claire Maguire, Clinical Lead, Military Veterans Service. Mike True, from Live at Ease, was our contact in relation to data transfer on behalf of the social service for veterans. We would like to thank him for his time and encouragement of the research. PSSRU at the University of Manchester

10 PSSRU at the University of Manchester

11 EXECUTIVE SUMMARY Introduction The Personal Social Services Research Unit (PSSRU) at the University of Manchester was commissioned by Pennine Care NHS Foundation Trust to provide an independent evaluation of the North West Military Veterans Improving Access to Psychological Therapies (MV IAPT) Pilot Service. This service has two components: (i) a clinical psychological therapies service, conforming to the principles of other IAPT services but specifically directed towards and designed for the unique needs of veterans; and (ii) a social wrap around service through Live at Ease, focused on the social and employment needs of veterans with a signposting function to other services that may offer help, such as benefits, work and debt advice. For the clinical service, the analysis was made possible through the Clinical Minimum Data Set collected in relation to all contacts with the psychological therapies service as with other nonmilitary IAPT services. For the analysis of the social Live at Ease service, a bespoke data set was constructed detailing the areas of need identified and the nature of services that clients were signposted to. Additionally, data from a standardised client questionnaire (the General Health Questionnaire GHQ12) were requested from the service for all clients referred after December 2012 as a marker of wellbeing. Data in this report covers all referrals to the services over 20 months (September 2011 to April 2013) and the associated activity and outcomes around these. The report offers a broad evaluation of the operation of the pilot service to assist Pennine Care NHS Foundation Trust and commissioners to judge the value of the service, in terms of the patients referred, outcomes and costeffectiveness. The research questions were: What are the characteristics of military veterans? What range of services have been provided to them and what do evaluations of veteranspecific services tell us about who accesses them and their effects? Who accessed the pilot MV services and what were its effects? What are the costs of the pilot MV services and how costeffective were they? Are there variations in outcomes for people in different circumstances who gets what, with what outcomes and at what cost? What are the opinions about the services, concerning their value and operational effectiveness, amongst patients and staff? Methodology There were five components to the methodology, broadly following the research questions, above. A purposive literature review describing key characteristics and needs of military veterans, previous service models providing psychological and social assistance to them and data from previous nonveteran IAPT services. An analysis of the characteristics and outcomes of clients using the Pennine Care MV IAPT and Live at Ease (LAE) services. PSSRU at the University of Manchester

12 An analysis of the costs and costeffectiveness of the Pennine Care MV IAPT and LAE services. Analysis of the relationships between costs and outcomes in each service using multivariate statistical methods. Interviews with key service providers to examine the value of the services and any challenges in their implementation. Literature review Reviewing the literature identified several areas of psychological and social wellbeing in UK veterans that require further attention: Only a minority of veterans experience mental health and social problems, yet these veterans constitute an important minority. The predominant mental health problems include depression, anxiety, PTSD and substance misuse. Those veterans that experience problems are illserved by existing mental health and therapeutic services. Once mental health problems are recognised, which represents an issue in itself, veterans tend not to receive extensive treatment for their mental disorder. There are very few services designed exclusively for veterans, which focus on the mental health and social needs of this specific population. Veteransfocused services are rarely evaluated and there is little data on the characteristics of those going through or on outcome measures. There are important subgroups of veterans who may fare badly in comparison with others or who may experience particular symptoms. Early service leavers, reservists, those with physical disabilities and those with substance abuse are amongst the most important subgroups requiring further evaluation. Social aspects of life, including employment and housing, present difficulties only in a minority of veterans; however, without sufficient and representative data, it is difficult to generalise. Support and advice provided for the transition from military to civilian life is limited, which may lead to unemployment and homelessness. Overall, the mental health needs of veterans have been reported to a greater extent than social needs. However, the majority of studies have employed preselection criteria, which lead to the assessment of particularly vulnerable and generally unrepresentative groups of veterans. In addition, the emergence of subgroups, including early service leavers, physically disabled and reservists, has only been touched upon within the literature, similarly to unmet social care needs. The same PSSRU at the University of Manchester

13 can be said for service evaluations, which indicate either a lack of available services or the lack of effectiveness assessments. Taking into account this review, the present service evaluation of the veterans IAPT programme implemented by Pennine Care NHS Trust addresses these issues and extends the limited evidence base. Findings I: Characteristics and outcomes The evaluation of evidence collected relating to the characteristics and outcomes of the clinical and social service for veterans produced the following broad findings. There were data on 952 referrals through the MV IAPT clinical service. 505 patients had an assessment and more than one session, permitting comparisons of standardised measures (PHQ9, GAD7 and WSAS) over time. These data from the clinical service were from large numbers of patients permitting detailed analysis. The clinical MV IAPT service engaged a hard to reach group, not served by traditional IAPT services: 35% had a forensic history, 27% had some degree of alcohol misuse, and 19% had substance misuse. Overall, there were 619 sessions with Did Not Arrive statuses, but quite a large number, 131, from those who subsequently completed treatment indicating that the service holds on to patients ensuring some continuity in response. Clinical outcomes (changes between first and last session for Depression, Anxiety, and Work and Social Adjustment) are good and statistically significant comparable to effects seen in trials and greater in impact than those of antidepressants alone. There was a significant dosage effect the more sessions received, the greater the effect; those who completed a full course of therapy did better than those who dropped out and those who were still engaged in treatment during the evaluation period. There was greater impact of the clinical service if the patient was an Early Service Leaver. ESLs experienced slightly improved remission rates but similar levels of recovery from both depression and anxiety compared to Non ESLs. Those receiving both the clinical and social service were a more psychologically unwell and worse adjusted group than those receiving the clinical service alone. It may be that veterans receiving the combined service were especially targeted. There was also a greater impact, in terms of improvements in depression, anxiety and social adjustment, for who received the combined service, indicating added value from having received both services. Outcomes for the social LAE service alone, using an indicator of wellbeing, were good and statistically significant. The service was comparable in its effects to other similar services serving other populations. However, the same number of clients would be classified as cases of psychological PSSRU at the University of Manchester

14 disorder at both beginning and end of involvement. Thus, there were no clients who would be designated as recovered despite these improvements. However, this was not the original objective of the Live at Ease service. Findings II: Costs and costeffectiveness With respect to the costeffectiveness of the services, the analyses presented calculated the additional costs, over and above usual care for samples of veterans, comparing this with the additional effects gained from receipt of the services. Levels of uncertainty in the data were taken into account and presented through formal analysis. There were several conclusions arising from this part of the evaluation: The analyses indicate for the decisionmaker the probability that the service(s) they have commissioned are costeffective. In effect, they show commissioners the probability that, if the service is funded, this will be the correct decision. The decisions as to costeffectiveness are based on the improvement in effectiveness (here, improvement in depressive symptomatology for the clinical service and in wellbeing for the social service) versus costs of the services against costthresholds. For the clinical service, this threshold was calculated with reference to IAPT services for the general population and antidepressant treatment in primary care. Other costthresholds could be chosen for comparison, which would affect the probability of costeffectiveness. There is, however, very little evidence particularly with regard to the effectiveness of psychological or other services to veterans, with which to offer other appropriate thresholds. For the clinical service, a costthreshold reference value from evaluation of general IAPT services makes sense in that this is the next best alternative response to serving the psychological treatment needs of veterans. The question is: how do veterans fare, in terms of costs versus effects, in comparison? For the social service, the reference value used was from an independent evaluation of an emerging social care technology (Individual Budgets), with costs and effecs from the receipt of standard social care services used as the reference point. These services are obviously not specifically for veterans, but should be treated as representing a costeffectiveness ratio of currently funded, standard nonmedical technology. It is thus a plausible, appropriate comparison against which to judge the costeffectiveness of introducing a new social care type service for veterans. With respect to the clinical service, we found that, in terms of this comparison, for total costs, including the initial investment in the pilot, the service is probably not costeffective. A critical element here is the time period over which the initial set up costs are discounted. However, for ongoing costs, once the service is up and running, then it is more than probably costeffective. After the initial investment, to keep the service running would likely PSSRU at the University of Manchester

15 result in an effective improvement in patient symptoms against costs, below those of reference values from the general IAPT service and antidepressant treatment in primary care. With respect to the social service, we found that for ongoing costs, the service is more than probably costeffective. Running the service would be likely to result in an effective improvement in client wellbeing against costs, below that of a reference value from general, standard social care for adults. Findings III: Cost/outcome relationships The relationships between each of the outcomes considered elsewhere in this report changes in depression, anxiety and work and social adjustment and various circumstances of patients and process measures of the progress of therapy were considered for the clinical service only. Also considered were the relationships between the costs of the service and these other factors. These relationships were explored in statistical models that help us see the independent effects of these various characteristics whilst holding other factors constant. The following broad findings emerged from these models: There was a significant dosage effect remaining in these models even after the impact of other subgroup characteristics is accounted for. For changes in depression, poor social adjustment and whether veterans were early service leavers and had a forensic history predicted improvement. For changes in anxiety, apart from the continued influence of whether patients had completed treatment, none of the factors investigated showed consistently significant effects. This may have been due to either the properties of the measure (GAD7) itself being insensitive to change or to the particular nature of anxiety as part of clinical presentation to the service. For costs, the two most important predictors of lower costs were whether veterans were substance misusers or had a forensic history. Although patients with these circumstances, to the credit of the service, were initially engaged, they tended to drop out earlier than other patients, so leading to reduced costs. Again, whether patients completed therapy was highly predictive of increased costs, since most of the cost variation was dependent on staff time. Anxiety levels of patients showed a significantly positive relationship to costs and it may be that this being a part of many presentations to the service, led to an increase in staff time; this being the greatest contributor to cost variation within the service. Qualitative service evaluation Many of the findings here help to provide a context for some of the material presented in the chapters using the more quantitative data. They show that after uncertain expectations, there was a large volume of referrals to the clinical service, PSSRU at the University of Manchester

16 that the makeup of the referral profiles were different to what was expected there was a range of different mental health problems, ages, and also that veterans acessing the service had family and did not all conform to the expected single male profile. It was also not expected that there would be a high rate of selfreferrals, but in fact this eventually proved to be the case. One of the strengths of the clinical service was perceived to be the use of a good mix of staff skills; a multidisciplinary team approach with a mix of different therapuetic approaches. However, the changes in commissioning were a challenge and it was felt that there was a high degree of uncertainty about the end of the pilot. The clinical service was, however, perceived to be a success in terms of the initial plans managers had for it and also its subsequent operation. Given the very low feedback rate in terms of veteran s service experiences, the positive satisfaction scores elicited cannot realistically be generalised for the whole sample. Summary and conclusions The Pennine Care MV IAPT service was developed in the context of the general IAPT services now operational throughout the country. There is good evidence of the effectiveness of the psychological treatments undertaken in this context, particularly for depression. Moreover this evidence suggests that psychological treatments for depression, for example Cognitive Behaviour Therapy and Interpersonal Psychotherapy are equivalent in their effects to drugs. The question considered in this evaluation was whether these beneficial effects would be similar for a population of hard to reach military veterans who often do not engage with mental health services and have specific needs associated with their military service and experiences since leaving the armed forces. The LAE service was developed in the context of providing social support to veterans who often experience difficulty with accomodation, employment and debt and finances. There is very little evidence of the effects of social care type services to veterans. The evaluation questions were therefore, who receives this service and what impact does it have on levels of wellbeing after continued use? What are the ongoing costs of this service and can it be considered costeffective? The main question, as to whether the beneficial effects cited for the impact of psychological therapies within the general IAPT framework would be similar for a population of hard to reach military veterans, can be answered broadly with an unequivocal yes : the veterans coming through referral routes into the clinical service were engaged with the service; the service held on to patients not traditionally serviced by IAPT and general practice models of care; outcomes were good, comparable to those in the published literature and other service models for veterans; there was an added value for veterans who received the combined clinical and social service; and the probability of cost effectiveness was high, particularly when considering the running costs of the service(s) only, after the initial investment in the pilot programme had already been incurred. The main message in terms of cost effectiveness is: PSSRU at the University of Manchester

17 After the initial investment, to keep the service running would likely result in an effective improvement in patient symptoms against costs, below those of general IAPT services and antidepressant treatment in primary care. There are, of course, limitations to this evaluation. In particular, the evaluation of the services conducted here is disadvantaged by the absence of a comparison group with which to test out whether the improvements shown were real, in the sense that they occurred over and above what would have happened anyway. However, this is also a limitation of most of the evaluations that have already taken place, for example those of general IAPT services. However, against this, the rates of improvement in the outcomes shown here are well above the rates for natural recovery or spontaneous remission of between 520% shown in other studies. In conclusion, this evaluation of routinely generated data from the operation of the clinical and social services of the Pennine Care MV IAPT shows evidence of good engagement and beneficial outcomes for military veterans at a cost which is defensible and compares favourably with both general IAPT services and routine treatment with antidepressants in primary care. PSSRU at the University of Manchester

18 PSSRU at the University of Manchester

19 1 INTRODUCTION The Personal Social Services Research Unit (PSSRU) at the University of Manchester was commissioned by Pennine Care NHS Foundation Trust to provide an independent evaluation of the North West Military Veterans Improving Access to Psychological Therapies (MV IAPT) Pilot Service. This service has two components: (i) a clinical psychological therapies service, conforming to the general principles of other IAPT services (established from late 2005 to implement the National Institute for Health and Clinical Excellence guidelines for people with depression and anxiety disorders) but directed towards the specific needs of veterans; and (ii) a social care wrap around service through Live at Ease, focused on the social and employment needs of veterans with a signposting function to other services that may offer help, such as benefits, work and debt advice. This report presents the analysis of data from both these services. For the clinical service, the analysis was made possible through the Clinical Minimum Data Set collected in relation to all contacts with the psychological therapies service, as employed with other nonmilitary IAPT services, with data from this transferred to the research team. For the analysis of the social care Live at Ease service, a bespoke data set was constructed detailing the areas of need identified and the nature of services that clients were signposted to. Additionally, data from a standardised questionnaire (the General Health Questionnaire GHQ12) were requested from the service for all clients referred after December 2012 as a marker of wellbeing for clients referred through the service. Unique anonymous identifiers enabled these two data sets to be linked, thus making possible analysis for those receiving only the clinical service and those receiving the clinical plus social care service. Data in this report covers all referrals to the services over 20 months (September 2011 to April 2013) and the associated activity and outcomes around these. The data from initial 12month contacts with the clinical service were made available to the PSSRU evaluation team on 7 th September After data cleaning and transfer to statistical software for the purposes of analysis, these data were available to interrogate from 27 th September The routinely generated data from initial sixmonth contacts with the social care Live at Ease service were made available to the evaluation team on 16 th October After data cleaning and transfer to statistical packages for the purposes of analysis, these data were available to interrogate from 29 th October Preliminary findings from the analyses of these interim datasets were made available to commissioners at the North West Armed Forces Community Health Workshop, Thistle Hotel, Haydock, on the 6 th November For the clinical service, further data were transferred to the evaluation team for referrals beyond these initial periods up to the end of April 2013 (on 24 th April 2013). For the social care Live at Ease service, additional data using the outcome measure General Health Questionnaire (GHQ12) were provided on 12 June 2013, relating to a small sample (n=21) of clients referred from January to May The evaluation reported here therefore, as commissioned, concentrates on data from the clinical service supplemented, where possible, by that relating to the social care Live at Ease service. The report is intended to offer a broad evaluation of the operation of the pilot service with data presented to assist Pennine Care NHS PSSRU at the University of Manchester

20 Foundation Trust and commissioners to judge the value of the service, in terms of the patients referred, outcomes and costeffectiveness. 1.1 Background to the services As the literature review commissioned as part of this report will show, the number of service personnel leaving the UK armed forces due to mental health problems is low. However, there is growing evidence that there are particular groups of veterans, such as those who left the forces early, who find it hard to adapt to civilian life. There is also evidence that the usual responses, at the primary care level, for these veterans are found wanting. Many veterans with mental health problems do not approach their GP or consequently mental health services for help. The stigma of receiving such help is one factor explaining this low takeup of support, as is comorbidity such as drug and alcohol problems of veterans, which are often exclusion criteria for receiving help from traditional, civilian psychological therapy services. The Military Veterans IAPT service was set up to provide psychological therapies to veterans, who are described as those who have served for at least a day in HM Armed Forces, whether as a Regular or as a Reservist (Pennine Care NHS Foundation Trust, 2012, p.5). The pilot service was funded for two years (1 April 2011 to 31 March 2013) through Strategic Health Authority IAPT monies. Some of the analyses contained in this report were directed at specifically, offering evidence of outcomes to commissioners to support a bid for interim funding to continue the service after 31 st March Building on the operation of the wider, nonmilitary IAPT services being run throughout the country, the Pennine Trust Military Veterans IAPT service was run in the recognition of these problems for veterans. The aims of the clinical service were to: Provide improved access to psychological therapies for veterans and their immediate families through a dedicated, specialist service. Work with all North West local IAPT services (and other key services working with veterans experiencing mental health issues) to improve confidence and competence in working with this hard to engage group. (Pennine Care NHS Foundation Trust, 2012) The aims of the nonclinical, social care service were to: Provide nonclinical support to clients of the Military Veterans IAPT service. Develop a comprehensive understanding of the provider base in the North West that could offer advice and assistance to this group. This was done through accredited advisers helping clients to access relevant information and sources of help to make and act upon decisions to improve their wellbeing in a nonclinical setting. Reviewing progress with the involvement of each client was also seen as a key aim. PSSRU at the University of Manchester

21 Staffing As of 29 th February 2012, the MV IAPT clinical service consisted of: 1 WTE Clinical Lead/Clinical Psychologist 1 WTE Highly Specialist Psychotherapist 0.6 WTE Family/Systemic Therapist 1 WTE Qualified High Intensity Therapist (HIT) 5 WTE Trainee HITs (including 1 qualified Clinical Psychologist and 2 Registered Mental Health Nurses) 4 WTE Trainee Psychological Wellbeing Practitioners (PWPs) 1 WTE Service Coordinator 3 WTE Administrative Staff As of 8 th January 2013, the social care Live at Ease service consisted of: 6 WTE Caseworkers (1 from a criminal justice background, 2 from Housing Associations, 1 with a charity background, 1 excitizen s Advice Bureau worker, and 1 worker from the military) 1 WTE Manager 1 WTE Marketing staff 2 WTE Administrative staff (involved in telephone case work, taking referrals, planning assessments and completing Verification measures for evaluation of the service). Modes of help The clinical MV IAPT service relied on NICE approved interventions such as cognitive behavioural therapy, psychodynamic therapy and systemic (family) therapy. In terms of the IAPT nomenclature, these therapies are described as Step 3. They are delivered by skilled psychological therapists. The PWP s deliver other interventions informed by cognitive behavioural therapy that focus on particular issues, such as sleep or anger management. They also are involved in supporting clients with daytoday issues such as medication management and activity. In IAPT services these interventions are described as Step 2. The MV IAPT service is different to most other local IAPT services in that registration with a GP at referral is not essential, clients with problematic substance misuse are still accepted into the service, referrals are accepted from a range of sources including third sector organisations and there is positive outreach with communities and with individuals referred to the service. The social care LAE service was orientated particularly towards more practical and personalised help in the areas of: housing and accommodation, training and jobs, money management, debt advice and support. Individual support plans are agreed between veterans and the caseworkers delivering the service. The service is one in which veterans are put in touch with key organisations that help with these difficulties to build a support network around the client. PSSRU at the University of Manchester

22 1.2 Aims of the evaluation As the services were already up and running at the time the evaluation was commissioned, the analyses in this report represent a routine evaluation of the services and the effects that their continued use had on veterans symptomatology and wellbeing. The evaluation is thus not a controlled study and this fact compromises the interpretations of the effects of the services over time. Therefore, we have employed a range of external comparative information to help judge the data presented here. The study, as commissioned by Pennine Care NHS Foundation Trust, was an exploratory one to examine patterns of variation in target groups and differential outcomes and costs between them. The research questions were: What are the characteristics of military veterans? What range of services have been provided to them and what do evaluations of veteranspecific services tell us about who accesses them and their effects? Who accessed the pilot MV services and what were their effects? What are the costs of the pilot MV services and how costeffective were they? Are there variations in outcomes for people in different circumstances who gets what, with what outcomes and at what cost? What are the opinions about the services, concerning their value, amongst patients and staff? A range of analyses was conducted from which to answer these research questions. The particular methodologies employed and the measurement of these key variables characteristics, outcomes and costs are described in detail in the next chapter. 1.3 Structure of this report After this brief description of the services to be evaluated, the following chapters present our plans for the analysis of the routine data provided by the services, the literature on military veterans needs and the services provided to them and the findings from our evaluation. Chapter 2 describes our methodology, which comprised analyses from both the Clinical Minimum Data Set and the Live at Ease (LAE) bespoke data set with the addition of interviews undertaken with key service providers. Chapter 3 presents a purposive literature review with the aim of describing the key characteristics and needs of military veterans, previous service models that have provided psychological and social assistance to them and data from previous nonmilitary IAPT services. These elements are reviewed so as to provide comparative benchmarking of the data from the Pennine Care MV IAPT services against published data. Subsequently, Chapters 4 through 6 presents our findings in relation to key areas of investigation: respectively, the characteristics and outcomes of different target groups who received the MV IAPT services, the costs and costeffectiveness of the services and relationships between costs and outcomes. Chapter 7 presents findings from a qualitative service evaluation detailing staff and veterans service experiences. Chapter 8 reviews the findings as a whole and summarises them in terms of key conclusions relating to the impact and value of the services in the future. PSSRU at the University of Manchester

23 2 METHODOLOGY This chapter describes in full the methodology of the evaluation. There were five components to this, broadly following the research questions described in Chapter 1: A purposive literature review describing key characteristics and needs of military veterans, previous service models providing psychological and social assistance to them and data from previous nonveteran IAPT services. An analysis of the characteristics and outcomes of clients using the Pennine Care MV IAPT and Live at Ease (LAE) services. An analysis of the costs and costeffectiveness of the Pennine Care MV IAPT and LAE services. Analysis of the relationships between costs and outcomes in each service using multivariate statistical methods. Interviews with key service providers to examine the value of the services and any challenges associated with implementation. These five elements are now described. 2.1 A purposive literature review Literature searches were conducted by one researcher (CG) on Pubmed, Web of Science and Google Scholar with the following search terms: UK military veterans, UK exmilitary personnel, UK exservice personnel, mental health, access to therapies, psychological services, housing, employment, social care access and health care costs. Although the nationality was specified within the search terms, the majority of studies resulting from the literature searches were concerning U.S. military veterans. The obtained literature only comprised a small number of studies which contained data on the demographic, military and clinical characteristics of UK veterans. In addition, only a limited evidence base emerged on psychological and social services provided for veterans or those that were in line with the IAPT approach. Based on the sampling strategies, some studies had to be excluded from the literature review. This was, in some cases because veterans were merged into groups with active soldiers, so that these papers did not offer conclusive data about the psychological and social needs of veterans. Also, some studies employed the term veteran in the assessment of still active military personnel who have been deployed to a certain war. In both cases, findings were not considered representative of the general veteran population. Therefore, the majority of these studies were excluded for the purposes of the review and were only used in some occasions as background information. This search was supplemented by papers known to the Pennine Military Veterans services and sent to the research team and also papers retrieved in hand searching of UK mental health journals (Journal of Mental Health, British Journal of Psychiatry) by one of the researchers (PC). 2.2 Characteristics and outcomes of the Military Veterans services The characteristics and outcomes of clients receiving each of the services were analysed slightly differently, with different measures and with the data aggregated in PSSRU at the University of Manchester

24 different ways. This was in keeping with their different objectives and ways of working. The methodology with respect to each service now follows. The clinical MV IAPT service The components of the analysis for the clinical service followed that of other evaluations of nonveteran, general IAPT services (Richards and Suckling, 2009). Data on the characteristics of veterans receiving the service, the processes of therapy and outcome data were collected from the webbased clinical information system used in all IAPT services (PCMIS). This permitted sessionbysession process and outcome data to be collected leading to a rich dataset of client contacts available for analysis. The process measures collected were the date, time and duration of each contact, the purpose of the contact and the specific treatment offered. The current status of the client in terms of treatment was also collected, this falling into predetermined values: patient completed treatment ; patient dropped out of treatment ; patient considered unsuitable for further treatment ; patient remaining in treatment ; and patient deceased. These categories enabled us to construct subgroups of clients by which to aggregate the data on characteristics and outcomes. Process data on clinical activity for different categories of people were calculated: those who completed treatment, for those who remained in treatment, and for those who dropped out during treatment. These data included the average number of sessions (including assessment), the average time overall spent receiving therapy, and the numbers with at least one session of the following: Depression recovery programme Selfhelp anxiety programme Medication management Computerised CBT Information only Books on prescription Highintensity CBT Interpersonal therapy Couple therapy Behavioural activities Stress exercises Counselling Psychoeducation The clinical outcome measures collected at every contact between worker and patient were: For depression, the Patient Health Questionnaire (PHQ9) (Kroenke et al., 2001). This is a symptom measure where higher scores indicate worse functioning. A score of 10 or above on the PHQ9 is indicative of a clinical case of depression. Similarly, the score can be used to indicate severity (not depressed = 04; mild = 59; moderate = 1014; moderate/severe = 15 19; severe = 2027). PSSRU at the University of Manchester

25 For anxiety, the GAD7 (Spitzer et al., 2006). This is also a symptom measure so that higher scores indicate worse functioning. A score of 8 or above is indicative of a clinical case of generalised anxiety, posttraumatic stress disorder, panic disorder or social anxiety disorder. Similarly, the score can be used to indicate severity (not anxious = 04; mild = 59; moderate = 1014; severe = 1521). For social difficulties, the Work and Social Adjustment Scale (WSAS) (Mundt et al., 2002). This is a simple 5item measure of general impairment drawn from studies of change during psychotherapy. Each question is rated on a scale of 08 with higher scores indicating greater impairment. A total score above 20 indicates moderately severe or worse psychopathology with scores between 10 and 20 associated with significant functional impairment. Scores below 10 appear to be characteristic of subclinical populations (Mundt et al., 2002). Descriptive analysis was conducted by presenting the means and standard deviations of all continuous variables and the frequencies and ranges for categorical data, denoting characteristics and outcomes of the veterans attending the service. Changes in clinical outcomes, between the first and last available session, for all clients, for those who completed treatment, for those who remained in treatment, and for those who dropped out during treatment were calculated according to three conventions: (1) Improvement. Mean (SD) scores on the PHQ9, GAD7 and WSAS were compared between pretreatment and posttreatment. As an indication of this change, effect sizes (Cohen, 1988) were calculated by subtracting the posttreatment score from the initial assessment (pretreatment) score and dividing by the posttreatment standard deviation. These provide a standardised, relatively conservative, indication of impact for the service, which was then compared with that from other services and the published literature. (2) Remission. We calculated the numbers and proportion (%) of patients who scored above the cutoff points for probable depression or anxiety pre and posttreatment on the PHQ9 or GAD7. From this, relative risks of depression or anxiety were calculated as the event rate (the numbers with probable depression or anxiety) posttreatment divided by the event rate pretreatment. In this way, the proportion improving to an extent where symptoms had remitted could be discerned. (3) Recovery. We calculated recovery rates as the proportion (%) of patients whose symptom levels for depression and anxiety reduced by 50% or more between pre and posttreatment. The percentage change used in these calculations was given by: (pretreatment score posttreatment score/score range of test)*100. The social care LAE service There are no established conventions by which to measure impact for the social care service, which was established to provide what may be termed socioeconomic and wellbeing support. We thus established some principles with the service for the collection of data pertaining to those referred. The resultant process and outcome PSSRU at the University of Manchester

26 data were delivered to the research team as a bespoke database which was transferred to statistical packages for analysis. The process measures included were: referral source (including self or agency referrals and with a PCMIS number, where applicable, to identify clients also referred through either from or to the clinical MV IAPT service), date of first contact and date of initial assessment. The status of each client in terms of the areas of need identified at referral and initial assessment was also collected, with these falling into predetermined values: employment ; selfemployment ; counselling ; drugs ; alcohol ; education ; budgeting/debt advice ; training ; offender support ; housing ; healthy living ; family ; children ; other. From each of these categories, LAE caseworkers would direct clients to avenues of help and support with the name of the organisations signposted to also being on the database. The numbers in these different areas of need and receiving different kinds of help or advice offered were detailed as descriptive data. As with the clinical service, subgroups of clients were constructed by which to aggregate the data on characteristics and outcomes. These subgroups were: family member as opposed to veteran ; veteran with a disability ; reservist ; and early leaver (those serving less than or equal to four years duration). The outcome measure for this service was one reflecting the nature of the intervention and its broad aims, to provide nonclinical support with a personalised response to the difficulties experienced by veterans. The outcome measure judged suitable to measure any changes experienced by clients was the 12 item General Health Questionnaire (GHQ12) (Goldberg and Williams, 1988). This measures levels of minor psychiatric morbidity and general wellbeing across different symptom areas, concentrating on the ability to perform normal functions and also the emergence of distressing experiences. Each question is rated on a fourpoint scale and there are two conventional methods of scoring for the GHQ12: one provides a continuous score (036) reflecting psychological illhealth, obtained by summing scores on the 12 items (running from 0 to 3, with higher scores indicating worse conditions); the other scores each item as 0 or 1 (reflecting whether circumstances are worse / better than usual ), sums them to provide a total score (012), and calculates the proportion of people with a score of 3 or higher, as indicating that they are a probable psychiatric case (Iversen et al., 2005b). The GHQ12 has been used in service evaluations in the social care field, such as the evaluation of Individual Budgets (Glendinning et al., 2008) and there are general population norms available (Pevalin, 2000), which were used to compare scores from the LAE service. The GHQ12 was administered to a subset of clients of the service from January 2013 to May 2013 by caseworkers at initial assessment and at the end of service involvement. Descriptive analysis was conducted by presenting the means and standard deviations of all continuous variables and the frequencies and ranges for categorical data, denoting characteristics and outcomes of the veterans attending the service. Changes in outcome, between initial assessment and last available session, for all clients who attended the service were calculated according to two conventions: PSSRU at the University of Manchester