Evaluating the ENAT : reconciling findings from a mixed methods study

Transcription

1 School of Healthcare FACULTY OF MEDICINE AND HEALTH Evaluating the ENAT : reconciling findings from a mixed methods study Professor Claire A Hale School of Healthcare University of Leeds On behalf of the Project Team

2 What is Mixed Methods Research Mixed methods research is a design for collecting, analyzing, and mixing both quantitative and qualitative data in a single study or series of studies to understand a research problem. The purpose of this form of research is that both qualitative and quantitative methods, in combination, provide a better understanding of a research problem or issue than either method Cresswell JW.(2007) Designing and conducting mixed methods research. London Sage

3 Focus of this paper 1. To give a brief outline of the study, its design and its aims 2. To present some of the findings ( not all) 3. To discuss how they can be/ could be reconciled

4 Publications Protocol Hardware B, Hale C, Johnson D, Hill J, Young T, Adebajo A: A study evaluating the effect of Educational Needs Assessment Tool (ENAT) focused patient education, on Challenges of Mixed Methods health outcomes research in patients with Rheumatoid Arthritis: Protocol of an ongoing study. Open Journal of Nursing, 2013, 3, OJN doi: /ojn Published Online June 201( ( 390 downloads ) Qualitative study : currently under review Quantitiative study: about to be submitted Mixed Methods study: still in our heads

5 The ENAT Study- Background The ENAT (educational needs development tool) was developed for people with arthritis about 10 years ago. The rationale for its development was to make patient education more focused on the individual needs of the patient rather than what the professionals thought the patients should know. Previous studies in patient education had found that although it was effective the benefits were often short lived. The suggestion was made that professionals were not giving the information the patients actually wanted.

6 The Education Needs Assessment Tool (ENAT) is a self completed questionnaire comprising 39 items divided into 7 domains: Managing pain; Movement; Feelings; Arthritis process ;Treatments; Self-help measures; Support systems It was developed by researchers in Barnsley for people with arthritis It was then validated by our group, and

7 Research Team Professor Ade Adebajo, Chief Investigator, Consultant Rheumatologist, Barnsley Hospital NHS Foundation Trust Bernadette Hardware, Study Coordinator, Barnsley Hospital NHS Foundation Trust Dawn Johnson, Research Associate, Barnsley Hospital NHS Foundation Trust Professor Claire Hale, School of Healthcare, University of Leeds; Academic and Clinical Unit for Musculoskeletal Nursing, University of Leeds Dr Jackie Hill, Academic and Clinical Unit for Musculoskeletal Nursing, University of Leeds Dr James R Maxwell, Consultant Rheumatologist, Rotherham Hospital NHS FT Dr Tracey Young, Statistician, School of Health and Related Research, University of Sheffield Dr Mwdimi Ndosi, Academic and Clinical Unit for Musculoskeletal Nursing, University of Leeds Dr Euthalia Roussou, Consultant Rheumatologist, King George Hospital, Barking, Havering and Redbridge University NHS Trust

8 Study Design A study was designed, funded and then undertaken using a randomised controlled trial (RCT) with an embedded qualitative study. The use of the ENAT was the intervention. The study was carried out between 2010 and 2013 and involved 132 patients with rheumatoid arthritis (16 in the qualitative study) and 7 Clinical Nurse Specialists (CNS) from 6 acute hospitals in the UK.

9 Aims of the study To evaluate the usability of the ENAT in clinical practice To establish whether the ENAT intervention experimental group (EG) and the control group (CG) perceive that they are getting an equally good or equally inadequate educational service for their needs qualitative study To evaluate the effectiveness of ENAT-focused patient education on self, efficacy, patient knowledge and outcomes- quantitative study

10 Methods RCT: patients newly diagnosed with rheumatoid arthritis (RA) were randomised to receive either ENAT-focused education (EG) or usual care (CG) from their CNS at weeks 1, 16 and 32. Self efficacy (ASES), health status (AIMS-SF2) and patients knowledge of their RA (PKQ) were assessed at the SAME time points. The EG patients completed the ENAT questionnaire before the clinic. The ENAT was used for the as a template by the CNS to meet the EG patients perceived educational needs in addition to their usual care. The CNS recorded the patients intervention on a checklist following each consultation. The CG did not complete the ENAT but received their usual care and the CNS also completed the checklist.

11 Methods Qualitative: semi structured interviews were undertaken with a sample of 16 patients from 2 sites and 4 CNS from 3 sites. The analysis focused on the key objectives: the views of the patients and the nurses about the usability of the ENAT, and the patients views of the adequacy of the information and education that they received. Data were analysed using the Framework approach.

12 Design Quant Qual

13 Design Explanatory - sequential Quantitative Data was collected pre and post intervention and the qualitative interviews were carried out for each selected patient, as soon as the quantitative data collection was finished

14 Results from the quantitative arm The RCT sample comprised 132 patients meeting 1987 ACR criteria for RA of whom 88 (66%) were female. Median age was 55 years and RA median disease duration was 5.9 months. In the RCT we found that the ENAT group saw some significant improvements in self-efficacy and some aspects of health status.

15 ASES Scores self efficacy For the ASES (self efficacy) the mean scores were significantly higher for: Pain : P = Other symptoms: p = 0.003

16 Between-group differences in the self-efficacy (ASES) scores

17 Aims 2 SF For the AIMS2- SF, the mean scores were significantly higher for Improved symptoms: p=0.013 Affect ( mood): p=0.006

18 Between-group differences in the AIMS scores at week 32

19 Findings from the qualitative arm The patients who used the ENAT found it straightforward, comprehensive and easy to use. For the majority of patients it was felt that the ENAT had a direct and positive impact on the provision and type of information and education provided by the CNS during the consultation. It made them think of a couple of more things to ask the CNS, helped them to focus and determine their individual needs, and to think of other aspects when managing and coping with their disease.

20 However. Given that in the quantitative study there were differences in outcomes between the patients who got the ENAT and those who did not we were surprised to find no indication of this in the patient responses in the qualitative study. In the qualitative study everyone was happy with the kind of patient education and care that they received from the CNS. So why were the differences in the quantitative findings not reflected in the qualitative findings?

21 Why were the findings from the quantitative study not reflected in the qualitative study? Our qualitative sample was quite small 16/132 did it not reflect the larger group? Possibly - but the mean scores of the qualitative sample on the ASES and the AIMS2- SF were in the same direction as the whole sample.

22 The CNS treated both experimental and control group could there have been some contamination in their intervention? Possibly - and certainly when we looked at the patient education checklist that the CNS completed there was not much difference between the two groups - but that does not explain the differences in the ASES and the AIMS-2SF of the two groups.

23 Were we asking the right questions in the qualitative study? We asked those who had received the ENAT about the ENAT- and they were positive We asked everyone about the education they received from the CNS- and they were positive Should we have asked something else?

24 Thoughts? Are we seeing a CNS effect here where the patient s perception of the quality of education received depends more on the relationship with the CNS rather than the ENAT driven content? But what is the active ingredient in the ENAT that affected the patient outcomes? could it be something to do with active engagement.

25 Summary We used a mixed methods study in the hope that the qualitative study would help to explain the findings of the quantitative study. We found that the findings of the quantitative study were not reflected in the findings of the qualitative study. We still do not know what the active ingredient is with the ENAT.

26 Acknowledgement This article presents independent research funded by the National Institute of Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (grant reference number PB-PG ). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. The study was adopted by the UKCLRN Portfolio (number 9230) and the study team acknowledge the support of the National Institute for Health Research through the Comprehensive Clinical Research Network. The study (REC reference 10/H1310/8) received a favourable ethical opinion from the South Yorkshire Research Ethics Committee on the 3rd March The authors would like to express their gratitude to all patients who took part in the study.

27 Further information Prof Claire Hale - c.a.hale@leeds.ac.uk Bernadette Hardware Bernadette.hardware@nhs.net Dawn Johnson; dawn.johnson2@nhs.net