Critical Appraisal and Interpretation of Qualitative Evidence
Focus of this presentation Critically appraising a qualitative paper using the Critical Appraisal Skills Programme (CASP) worksheet for qualitative research
What is qualitative research? Why would we use qualitative research? Screening Questions What types of questions does qualitative research ask? Critical Appraisal Skills Programme (CASP) What can qualitative research contribute to healthcare? How does qualitative research differ from quantitative?
What is qualitative research? Not everything that can be counted counts and not everything that counts can be counted Multiple realities Naturalistic Words OBJECTIVE So, what is qualitative What is qualitative research? TOUCHY- FEELY Experiences Feelings PEOP LE Meaning SMALL SAMPLE X EASY Y
Discourse Analysis Content Analysis Ethnomethodology Case Study Qualitative Evaluation Interactionism Narrative Heuristic Research Cognitive Anthropology Evaluative Research Hermeneutics Grounded theory Interpretative Naturalistic enquiry Different types Different types of research of qualitative research Life History Study Ethnography Feminist Research Action research Conversational Analysis Participative research Phenomenology Rapid Appraisal Write and Draw Symbolic Interactionism Ethnoscience Humanistic Research
What is the research question and will a qualitative approach answer it? Reading qualitative research What does the research promise to offer healthcare? Is the research relevant? Reading qualitative research: Important questions
Sampling Who do we collect data from in qualitative research? How many people are enough? Critical Appraisal Skills Programme (CASP) How do we find people?
Sampling: How do we find people to take part in qualitative research? Convenience Sampling Purposive Sampling Snowball Sampling Theoretical Sampling
Sampling: How do we know we have enough people? Have we stopped finding any novel data? (Reached data saturation) Are there enough people to cover the range of different experiences? E.g. People of different ages, with different disease severities (provide a concrete example)
Reading qualitative research Have they described the sample well enough for you to decide whether this applies to your patient(s)? Do we know who collected the data? Do we know who didn t take part? And if not, why not? Have they used an appropriate sampling strategy? How did they decide how many people to sample? Was this appropriate? Reading qualitative research: Important questions
Data collection What data collection tools are used in qualitative research? How do we decide which tool to use? What are their advantages/disadvantages?
OBSERVATION INTERVIEWS Data collection Useful to explore in-depth experiences, but expensive to conduct Good to record actual rather than reported behaviour. Misses different perspectives. Potential ethical issues. DIARIES Affordable, and good at recording how people feel at the time. Difficult to ensure that the data is appropriate. FOCUS GROUPS Affordable and useful to explore group interaction, and discuss/ debate range of experiences. Can be difficult to arrange. DOCUMENTS Affordable, but data not recorded with research in mind. How is data collected in qualitative research?
Reading qualitative research Do we have enough detail of how and where the data was collected? Was the method of data collection justified? Was the data collected in a way that addressed the research issue? Are the limitations of the chosen method presented? Reading qualitative research: Important questions
Reflexivity How might the researcher affect qualitative research?
The importance of reflexivity In qualitative the researcher has an influence on the data in a number of ways: Influence on the participant Body language Type of questioning Interview schedule Manner with participant Existing relationship with the participant During data analysis: Pre-existing ideas about what they think they will/want to find It is important that the researcher considers their influence on the participant and the analysis, and that they present an account of their position so readers are able to judge this.
Reading qualitative research Has the researcher talked about who they are and the influence they may have on the data? Did the researcher do anything to limit their impact on the data? Did the researcher s own experience inform the research question? Reading qualitative research: Important questions
Ethical issues What ethical issues might arise in qualitative research?
What are the ethical issues involved in qualitative research? As with all research: If conducted within the NHS must have LREC approval. All participants should give informed consent Data protection and confidentiality issues (presentation of anonymous quotes) Dealing with sensitive issues: Effect on the participant and the researcher Potential to reveal issues such as abuse or neglect Safety of the researcher whilst collecting data outside of the institution
Reading qualitative research Had the researcher considered the ethical implications of the research? Has approval been sought from an ethics committee? How was informed consent, confidentiality and data protection managed? Reading qualitative research: Important questions
Data analysis How is data analysed in qualitative research? How is rigour ensured in qualitative research?
What kind of data is produced by qualitative methods? AUDIO VISUAL WRITTEN Recordings of interviews or focus groups These will usually be transcribed before analysis Videos of observations (perhaps including audio) Drawings Photographs Documents Diaries Notes from observations
Analysing qualitative data The most important thing is to make sure that the summary really does reflect what participants have said and that is the really tricky bit! The aim of data analysis is to go from a large amount of data that is in no particular order to a nice ordered summary of what the data is telling us.
So how is data analysed? DISPLAY THE DATA If necessary, transcribe the recordings from interviews or focus groups The data is read and any field notes added Read the whole transcription and add analytical notes Intention? Does she mean this or does she think she has to say it? Interviewer: So, you were given a prescription by your doctor for antibiotics, can you tell me what happened next? Mary: Well, I left the doctors and I meant to go straight to the chemist to put in my prescription. But I really was feeling very ill and it is a long way to the chemists. I only live about 2 minutes walk from the doctors so I went home instead. After I got home I started to feel even worse and I didn t have anybody to ask to go and pick up my prescription for me. By the time I felt well enough to get to the chemist s I thought it was probably not worth it anymore. She looked like she felt a bit guilty about this. Followed this up later on.
So how is data analysed? DATA REDUCTION Identify significant statements/units of meaning Intention? Does she mean this or does she think she has to say it? Interviewer: So, you were given a prescription by your doctor for antibiotics, can you tell me what happened next? Mary: Well, I left the doctors and //I meant to go straight to the chemist to put in my prescription.// But I really was feeling very ill and it is a long way to the chemists. I only live about 2 minutes walk from the doctors so I went home instead. //After I got home I started to feel even worse// and I didn t have anybody to ask to go and pick up my prescription for me. //By the time I felt well enough to get to the chemist s I thought it was probably not worth it anymore.// She looked like she felt a bit guilty about this. Followed this up later on.
So how is data analysed? Group into categories DATA INTERPRETATION Develop into themes Check representativeness of these from your transcripts Find examples of these Display Qualitative research and analysis represents an inductive approach to research where we start with real-life data and work towards abstract ideas or general principles.
What do qualitative findings look like? Qualitative data analysis often produces themes: Themes are used to describe the data in a concise and easy to understand manner Themes should be used to express the experiences of all participants
Patients had not realised that their symptoms were indicative of anything serious SYMPTOM EXPERIENCE Patients were worried that they would be blamed or stigmatised if they were diagnosed with lung cancer. BLAME AND STIGMA Patients delayed going to the doctor because they were scared of finding out that they had cancer or were scared of the treatments. FEAR Some patients, especially males, felt the need to be stoic about their illness CULTURE Patients did not have much knowledge about lung cancer and its symptoms KNOWLEDGE Tod, Craven & Allmark (2008), looked at the reasons for diagnostic delay in patients with lung cancer and found the following themes.
Dependability and confirmability Can we trust the analysis? How do we check the dependability and confirmability of the reported findings? Can we follow the decision trail? Is there an audit trail? Have I been given enough information about the methods used? Is there enough detailed description for us to interpret the meaning and context of what is being researched? How careful was the researcher to prevent bias affecting their analysis?
Reading qualitative research Is there an indepth description of the analysis process? Is it clear how the themes were derived from the data? Does the researcher consider their influence on the analysis and presentation of findings? Do you know how the presented quotes were chosen? Is there enough data to support the findings? Reading qualitative research: Important questions
Findings How should qualitative findings be presented?
Credibility Do we believe the findings? How do we check the credibility of the reported findings? Are the findings based on the data? Or on what the researcher expected to see? Did the researcher have prolonged engagement with the participants or phenomenon? Did the researcher attempt to triangulate their findings? Did the researcher discuss their data and interpretations with a peer(s)? Did the researcher check for negative cases? Did the researchers send the findings to participants to ask their opinion? Can you see any actual data (quotes)? Do you know why they reported those particular quotes? Do the author s arguments make sense?
Reading qualitative research Are the findings explicit? Are the findings related back to the original question? Does the researcher present data that contradicts their arguments? Has the researcher discussed credibility? Reading qualitative research: Important questions
Value of the research What makes for valuable qualitative research?
Is this relevant to our practice? Can you tell if the findings are transferable to your practice? Is there a description of the sample so you can decide it is relevant? Are there explanations as to what is going on in a particular social location/operation of a set of social processes? Does it provide comparisons that are relevant in supporting your argument
Critical appraisal helps the reader of research... Decide how trustworthy a piece of research is (validity and reliability) Determine what it is telling us (results) Weigh up how useful the research will be(relevance) Critical appraisal