Fraser Health Research Skills Workshop: The ABC s of Focus Groups and Interviews Marla Steinberg, Ph.D. Director of Evaluation The CAPTURE Project www.thecaptureproject.ca
Workshop Objectives 1. To understand when to use focus groups and interviews 2. To understand how to construct interview questions and focus group protocols 3. To gain an appreciation for how to facilitate a focus group 4. To understand appreciate inquiry 5. To understand how to conduct qualitative data analysis 6. To have fun
Workshop Outline Topic Timing Workshop Overview Interview Exercise When to use Focus Groups versus Interviews Exercise Developing Question Exercise Focus Group Activities Interview and Focus Group Tips Break 9:00 10:30 10:30 10:45 ish Focus Group Exercise 10:45 11:15 Appreciative Inquiry 11:15 12:00 Lunch 12:00 1:00 Introduction to Data Analysis 1:00 1:30 Data Analysis Exercise (small groups) 1:30 2:30 Findings (Large group) 2:30 3:00 Review and Evaluation 3:00 3:30 ish
Question Ambiguity?
Ice Breaker/Interview Exercise Choose a partner Using the interview exercise worksheet in your workbook, take turns interviewing each other and recording the responses. You will need the responses for another exercise! 10 minutes Brief report back Introduce partner (name and job) Why attending workshop
WHEN TO INTERVIEW? WHEN TO CONDUCT A FOCUS GROUP?
There are a variety of data collection techniques Surveys Interviews Standardized Tests Observations Focus Groups Case Studies Photography Document Review Method should match the research or evaluation focus. Best practice is to use multiple methods, generating both quantitative and qualitative data To count and recount!
Example of Mixed Methods Study of clinical decision support systems for management of two chronic diseases. Randomized control trial Conducted interviews with practice managers, nurses, and general practitioners Interviews revealed the difficulties of integrating the system into clinical practice Interview results helped explain the failure of the trial of the computerized system. Rousseau, N., McColl, E., Newton, J., et al (2003). Practice based, longitudinal interview study of computerized evidence based guidelines in primary care. British Medical Journal, 326, 314 318.
Focus on Qualitative Data Focus groups and interviews share a focus on qualitative data They both provide in depth information on a topic of interest Raw experience converted into words Huberman, A.M. & Miles, M.B. (1994). Data Management and Analysis Methods (p. 430). In Denzin, N.K., & Lincoln, Y.S. (Eds.), Handbook of Qualitative Research. Thousand Oaks: Sage.
Focus Groups and Interviews Focus Group: A focus group is a carefully planned series of discussions designed to obtain perceptions on a defined area of interest in a non threatening environment Originally used in market research First used for human services Intent is not to arrive at consensus or generate vigorous debate Interview: An interview is a conversation between two or more people (the interviewer and the interviewee) where questions are asked by the interviewer to obtain information from the interviewee.
How to choose? Nature of your topic Is it a topic that is appropriate to discuss in a group? How much time you have How much money you have Characteristics of your participants How many participants you want to involve or need to involve What type of information you want (your research or evaluation question)
Which One Exercise In groups of three, go through the Which One exercise in your workbook For each scenario, decide: Which is better: focus group, interview, either, neither Explain your choice Be prepared to report back 5 minutes
When is a focus group best? Topic is amenable to open discussion You want a large amount of input You have enough time to organize the focus groups, conduct the focus groups, and analyze the data You have the budget Travel for participants and your team Food Transcription
Focus Group Best cont. When it fits with the needs, values or culture of the participants When you want a variety of perspectives For the same amount of money, you can have a larger sample size with focus groups than with individual interviews.
When would a focus group not be appropriate Not fit the needs, culture or characteristics of participants. Not enough time or money Topic is not amenable to group discussion Emotionally charged topic When you want to create consensus
When are interviews best? The topic is best explored in one on one sessions Want in depth information from each respondents You don t need or want a large number of respondents Not concerned about generalizability Topic is conducive to talking about Fits with the needs, values or culture of the participants Have the necessary resources (time & money)
When would interviews not be appropriate? Want to be able to generalize Want lots of respondents Don t have enough time or resources Nature of the topic get more information when asked anonymously
TIPS FOR FOCUS GROUPS AND INTERVIEWS
Planning for your Focus Group or Interview Decide when Determine where Decide who to invite sampling is not a matter of numbers or convenience, but is strategically focused to collect the most appropriate rich data (1) Decide the purpose Determine the budget Write the protocol or interview schedule Find a facilitator or moderator (Focus groups) Record Obtain ethics approval and/or informed consent (1) Daly, J., Wills, K., Small, R., et al. (2007). A hierarchy of evidence for assessing qualitative health research. Journal of Clinical Epidemiology, 60, 43 49.
Focus Group Tips Ideal size about 6 8 participants Need to record the conversation Use two facilitators or moderators Require time to organize Use a number of different activities 1 1.5 hours in length Find neutral space Feed people
The Questioning Route for Focus Groups 1. Opening question Gets everyone talking 2. Introductory question Introduce topic 3. Transition question 4. Key questions About 2 5 5. Ending questions
The Opening Question Very important in a focus group The longer it takes before someone says something in a group, the less likely he or she is to say anything.. Neutral but conveys that all participants have something in common
Developing good key questions Questions need to generate discussion: Do you like our parking facilities? Were you able to follow your discharge plan? How could the discharge instructions be improved? How can we improve your wait time experience? Conversational, clear and short
Ending Questions Signal discussion is drawing to an end all things considered question: of all the things we discussed, which one is most important to you If you had one minute to talk to the CEO on.what would you say? Summary question: Summarize what was said and ask if was accurate. Final question: Begin with a short overview of purpose of the group, ask, have we missed anything? Is there anything that we should have talked about but didn t?
Types of Interview Questions Unstructured: Tell me about the first weeks with your new baby. Structured: I am going to ask you about typical challenges new parents experience, please tell me about you handled each of the following: Cleaning Looking after the baby Grocery shopping Financial management
Developing Question Exercise Individual work, 10 minutes Thinking about your own projects: List some questions that would be good for focus groups List some questions that would be good for an interview questions: Structured interview Unstructured interview Discussion: What differences emerged?
Ethics and Informed Consent Research studies require approval from ethics review board Evaluations do not Tri Council Policy Statement Some journals will not publish results unless reviewed by ethics committee On line ethics screening tool: www.ahfmr.ab.ca/arecci/tool/
Ethical Treatment of Participants Do no harm Avoid or minimize any negative effects from participating in research or evaluation Voluntary participation, not tied to services Evaluation experience will not cause any physical or psychological harm Informed consent
Informed Consent Anyone participating in a study must be able to provide informed consent Need to be told: the purpose of the study (research or evaluation) what they are expected to do how their identity will be handled what will happen to the data how results will be used Contact person for further information
BREAK 15 MINUTES
Focus Group Exercise Number yourselves from 1 3 Get your assignment Participate in a focus group for 15 minutes Topic: How can we improve professional development at Fraser Health? Debrief
Focus Group Debrief What do you need to pay attention to: Dawn Dominator (monopolizers) Negative Ned (Idea zapper) Off track Tron (my own agenda) Suzy Sunshine Quiet Quinton (hesitators) Experts I d rather be somewhere else Emotional reactions
Facilitation Techniques Set ground rules Talking stick Round robins Subtle techniques: stand next to person Direct techniques: Refer back to ground rules Switch to different activity Parking lot/idea bin Say what is going on
Focus Group Activities Need not be just discussion Can include: Brainstorming Art or drawing Complete a sentence Open space World Café Others
APPRECIATIVE INQUIRY
AI Strength based approach to data gathering Uses a deep understanding of moments when we have been at our very best as a launching point for future action AI helps people identify what they want to see more of, to create a shared vision of the future and to make that vision become reality.
The Focus of the Questions change From what problems are you having? To: what s working well around here? And how can we do more of it?
AI Originally developed for organizational development by David Cooperrider. Usually in a workshop setting, participants remember and relate personal experiences of success, identify the common elements of these experiences, and devise statements and action plans for making those experiences occur more often in the organization.
The Appreciative Philosophy What we focus on becomes our reality In every society, organization, or group, something works Reality is created in the moment; there are multiple realities The act of asking question is an intervention People have more confidence and comfort to journey to the future when they carry forward parts of the past It is important to value differences The language we use creates our reality People are motivated to act when they have a choice in what they will do.
Two approaches to solving problems Problem Based Identify problem Conduct root cause analysis Brainstorm solutions & analyze Develop action plans Asset Based Appreciate what is Imagine what might be Determine what should be Create what will be
4 D Model
Discovery Participants interview each other and share stories: Describe a high point experience in your organization a time when you were most alive and engaged. Without being modest, what is it that you most value about yourself, your work, and your organization? What are the core factors that give life to your organization, without which the organization would cease to exist? What three wishes to you have to enhance the health and vitality of your organization.
Discover cont. Participants id key topics or themes common to the stories. Create a customized interview protocol by selecting three to five of the topics or themes and writing several appreciate questions for each. Using the interview protocol, interviews are conducted with as many organizational members as possible, ideally by the members themselves.
When to use AI: Can be used anytime, because it offers a strength based assessment, but particularly good: When dialogue is critical in moving the organization forward When relationships among individuals and groups have deteriorated and there is a sense of hopelessness
Evaluation of a workshop Traditional: AI: Clarity of workshop objectives Format of workshop Handouts /Materials Content Presenter Overall Very Satisfied Satisfied Dissatisfi ed Very Dissatisfi ed Don t Know What did you appreciate most about the workshop? Describe a moment that was particularly meaningful for you during the workshop. What made it so? Imagine future participants coming to this session. What are one to three things that you would add or enhance to make this workshop more meaningful for them?
Katimavic Youth Program Evaluation From: What did you like the most about your experience? What did you like the least? Please rate your team lead on the following: Ability to lead the group Ability to resolve conflicts Ability to coordinate the activities. What changes would you recommend? To: Talk about a time of an exceptional moment that you lived with others in the program in the past year. A time when all of the participants seemed engaged and participated fully. Describe the situation. What was happening at the time? What made it a great moment?
AI Example Developing a Strategic Plan for the Canadian Evaluation Society (CES) Questions for CES Members to answer:
Criticisms of AI Unbalanced and uncritical in its emphasis on accentuating the positive. May discourage inquiry by discouraging constructive criticism May imply an unwillingness to took at weaknesses, problems and things that are not going wrong.
Does not happen in practice Practitioners of AI affirm that problems and weaknesses can and do emerge in an AI inquiry Some problems and weaknesses can be easier to address and surface when an AI approach is used. The art is knowing when it is appropriate to use it!
AI Exercise I would like you to think of a time in your work when were most proud to be part of the Fraser Health Authority. Think back and tell me a story about this experience. In pairs, tell each other a story: Describe a high point experience in a professional development experience a time when you were most alive and engaged Volunteers to share high points.
12:00 1:00 Lunch Break
QUALITATIVE DATA ANALYSIS
There are lots of different approaches Analysis is essentially a taxonomic process Data are sorted to give coherent categories 1.Content analysis 2.Grounded Theory (Glaser & Anselm Strauss) Constant comparative method A general methodology for developing theory that is grounded in data systematically gathered and analyzed. 3.Discourse Analysis analysis of speech units larger than the sentence and of their relationship to the contexts in which they are used.
Data Analysis Three sub linked processes: 1. Data reduction 2. Data display 3. Conclusion drawing and verifying
Steps in Data Reduction in Qualitative Analysis The quantity of qualitative data can be daunting! 1. Get to know your data 2. Focus the analysis By research or evaluation question or topic Time period or event By case, individual or group 3. Categorize information (coding) Identify unique answers, themes or patterns Could be an entire paragraph, a sentence, part of a sentence or one word. Organize them into coherent categories This is very labour intensive and is the crux of qualitative analysis Preset or emergent categories
Steps Continues 4. Organize themes = creating conceptual groupings Existing theoretical framework Emergent framework (grounded theory) Larger categories Relative importance (counting) Relationships 5. Interpretation bringing it all together Diagrams or schematics can be helpful
Validity/Quality in Qualitative Analysis What are the criteria for studies that are based on narrative argument rather than on measurement and control over variables? How well the researcher/evaluator can persuade the reader that the phenomenon under investigation is well understood. They rest on sound methods and defensible conclusions
Best Qualitative Studies for Generating Evidence for Practice At issue is ability to generalize: Sampling focused by theory/conceptual framework and the literature Captures diversity of experience (saturation) Data collection and analytic procedures comprehensive and clear Located in literature to assess relevance to other settings and access discrepancies Daly, J., Wills, K., Small, R., et al. (2007). A hierarchy of evidence for assessing qualitative health research. Journal of Clinical Epidemiology, 60, 43 49.
Qualitative Analysis Exercise (1 hour) Form yourselves into groups of three Using the data provided ( I could more easily collect, share & use evidence if...: 1. Sort data into meaningful units by grouping like ideas together Decide what to call this group. Write the name of the group on a sticky note 2. Organize themes Group like ideas together Assign a name for the theme (coding) Group like themes together If you can, create a graphic or map 3. Report back
Example: Evaluation of Research Capacity Building Program
http://www.visual literacy.org/periodic_table/periodic_table.html
WRAP UP
Review: Solidifying the Learning What we covered: When to choose focus group or interviews for data collection How to develop good questions Tips on focus groups and interviews What to be aware of when facilitating a focus group Asset based data collection: Appreciative Inquiry How to do qualitative analysis (content analysis) How to design qualitative studies or evaluations to maximize generating implications for practice
Evaluation WWW (what worked well) WCB (what could be better)