Handling Qualitative Data in Chemistry Education Research. Mahbub Sarkar

Transcription

1 Handling Qualitative Data in Chemistry Education Research Mahbub Sarkar

2 Please discuss in group What areas of chemistry education research are you interested (or working) in? Why do you think a qualitative approach is suitable with your research interest? What sort of qualitative approaches are you interested (or using) in? 2

3 Nature of Qualitative Research and Qualitative Data

4 A Framework for Research Design Philosophical Worldviews Post-positivism Social constructivism Advocacy/participatory Strategies of Inquiry Qualitative strategies (e.g., ethnography) Quantitative strategies (e.g., experiments) Mixed-methods strategies (e.g., sequential) Research Methods Data collection Data analysis Validation Write-up Creswell (2009, p. 5) 4

5 Philosophical Worldviews Post-positivism Determination Reductionism Empirical observation and measurement Theory verification Constructivism Understanding phenomena Multiple participant meanings Social and historical construction Theory generation 5

6 Qual Vs Quan: Basic Differences Purpose Qualitative To describe a situation, understand a phenomena Quantitative To measure magnitude how widespread a practice is Research questions Broader, contextual, flexible Fixed/focussed Sampling Theoretical Statistical Data In-depth explanatory data from a small sample Wide breadth of data from large statistically representative sample Analysis Draws out patterns Tests hypotheses, uses data to support conclusion Result Illustrative explanations provide meaning, experience and views Numerical aggregation generalise, compare and summarise Expected outcome Usually not predefined Identified in advance 6

7 Qualitative Study Approaches Action research Case study Ethnography Grounded theory Individuals and groups researching their own practice, socio-cultural settings and experiences PLANNING à ACTION à REFLECTION In-depth investigation of a single or small number of units in its real context at a time Study of the culture of group often to develop cultural awareness and sensitivity Developing a new theory grounded on data Phenomenology Study of individual s lived experiences of events Narrative Collect participants stories and then retell their stories 7

8 Choosing a Study Approach 8

9 Qualitative Data Collection Techniques Main data collection techniques: 1. Observations 2. Interviews 3. Documents 4. Surveys 5. Audio-visual materials 9

10 Observations Creswell (2009, p. 179) Options within observations Advantages Challenges Complete participant researcher conceals role Researcher has a first-hand experience with participant. Researcher may be seen as intrusive. Observer as participant role of researcher is known Participant as observer observation role secondary to participant role Complete observer researcher observes without participating Researcher can record information as it occurs. Unusual aspects can be noticed during observation. Useful in exploring topics that may be uncomfortable for participants to discuss Private information may be observed that researcher cannot report. Researcher may not have good attending and observing skills. For some participants, building rapport may be challenging. 10

11 Interviews Creswell (2009, p. 179) Options within interviews Advantages Challenges Face-to-face one-on-one, in person interview Telephone researcher interviews by phone Focus group researcher interviews participants in a group interview interviewees respond to researcher s questions through Useful when participants cannot be directly observed. Participants can provide historical information. Allows researcher control over the line of questioning. Provides indirect information filtered through the views of interviewees. Provides information in a designated place rather than the natural setting. Researcher s presence may bias responses. Not all people are equally articulate and perceptive. 11

12 Documents Creswell (2009, p. 180) Options within documents Public documents, such as minutes of meetings, newspapers, and curriculum documents Private documents, such as journals, diaries, or letters Advantages Enables researcher to obtain the language and words of participants. Can be accessed at a time convenient to researcher. As written evidence, it saves a researcher the time and expense of transcribing. Represents data that are thoughtful in that participants have given attention to compiling them. Challenges Not all people are equally articulate and perceptive. Doubt of authenticity. Search out the information in hard-tofind places. Access to information. 12

13 Surveys Options within Surveys Paper-based Online Structured Open-ended Advantages Useful when the subject population is large or distributed geographically. As written evidence, it saves a researcher the time and expense of transcribing. Challenges Subjects need motivation to respond. The number of open questions should be reduced to a minimum. Not all people are equally articulate and perceptive. 13

14 Audio-visual Materials Creswell (2009, p. 180) Options within A-V materials Audios Photos Videos Art objects Software Advantages May be an unobtrusive method of data collection. Provides detailed information. Creative in that it captures attention visually. Challenges May be difficult to interpret. May not be accessible publicly or privately. The presence of a photographer or videographer may be disruptive and affect responses. 14

15 Qualitative Data Analysis Features Analysis is circular (non-linear) Iterative and progressive Close interaction with the data Data collection and analysis is often concurrent Level of analysis varies Can be sorted in many ways Data by itself may have meaning 15

16 Analysing Data qnoticing: Breaking up, separating, or disassembling of research materials into pieces, parts, elements, or units. qcollecting: With facts broken down into manageable pieces, the researcher sorts and sifts them Notice Things qthinking: Searching for types, classes, sequences, processes, patterns, or wholes. The aim of this process is to assemble or reconstruct the data in a meaningful or comprehensible fashion. Think About Things Collect Things

17 Data Analysis Approaches Deductive approach Using research questions to group the data and then look for similarities and differences Used when qualitative research is a smaller component of a larger quantitative study Used when time and resources are limited Inductive approach Using emergent themes to group the data and then look for relationship Used when qualitative research is the major design of the study Mix of deductive and inductive Most qualitative analysis does both (start with some theoretical ideas derived from literature, research questions and interview schedules and then discover new ideas theories, explanations in the data) 17

18 Qualitative Data Analysis Process Interpreting the meaning of themes Identifying recurrent themes Noticing patterns in the data Modifying themes (interrelating themes) Validating accuracy of the information Coding the data Identify themes - Explanatory (guided by the research questions) - Exploratory (guided by the data) Reading through all data Organising and preparing data for analysis (transcribe, translate) 18

19 Identify Themes - Techniques 1. Repetitions 2. Indigenous typologies or categories 3. Metaphors and analogies 4. Transitions 5. Similarities and differences 6. Linguistic connectors 7. Missing data 8. Theory-related material (Ryan and Bernard, 2003) 19

20 Selecting among Techniques No one right way to find themes, but some techniques are more effective under some conditions than others. We can evaluate the techniques on five dimensions: 1. kind of data types, 2. required expertise, 3. required labor, 4. number and types of themes to be generated, and 5. issues of reliability and validity. (Ryan and Bernard, 2003) 20

21 Coding Coding is the activity where a researcher applies meaning to raw data by assigning key words or phrases. These key words then act as signposts to themes within the data. Coding supports two forms of analysis 1. Retrieval Retrieve all the text coded with the same label Traditionally used envelopes/files now done using computer software Enables cross case comparison on same theme 2. Using the coding frame Use the list of codes to examine further kinds of analytic questions (e.g., relationships between the codes) 21

22 Quality Control Validity and Reliability

23 Improving Validity (=Credibility/ Trustworthiness) 1. Triangulation involves using multiple sources and perspectives to reduce the chance of systematic bias. ü by source ü by methods ü by researcher ü by theories 2. Member checking participants validate the findings of the researcher 3. Use rich, thick description to convey findings 4. Clarify the bias the researcher brings to the study 5. Present negative or discrepant information that runs counter to the themes 6. Spend prolonged time in the field 7. Use peer debriefing 8. Use an external auditor

24 Improving Reliability (=Dependability) üprocedural rigor describe the data-gathering process including issues of gaining access to the site, data collection methods, training data gatherers, the length of time spent gathering data, and the amount of data collected. ümeticulous records of documenting process of analysis (research diary; analytical memos) so that others can follow the process. ücross coding rigorous comparison of coding using multiple researchers. ümaintain reflexive journals.

25 Theme identification and Coding Work on sample transcripts provided 25

26 Use of Computer Software in Qualitative Data Analysis

27 Use of software Pros and Cons Pros: Easy storage of data Automated search and display Manual coding Easy search and retrieval Multiple researchers can work Most useful when you have lots of data Cons: Relying too much on automated coding may increase distance the researcher from the text 27

28 What is NVivo? Computer assisted qualitative data analysis systems Supports qualitative and mixed methods research Collects, organises, analyses content from interviews, questionnaires, social media, etc. Handles Word documents, PDFs, audio files, database tables, spreadsheets, videos, pictures, social media and web pages Transcribing tools for audio and video Team members work in the same project at the same time using NVivo Server 10

29 What NVivo does and what it does NOT do? NVivo does not analyse your qualitative data. You do! NVivo does assist in your analysis. Gather Data sources Literature sources Organise Data sources Coding structure demographics Literature review Analyse Coding Queries Memos Visualise Results Reports

30 Review Merge Refine Reflection Key ideas Any observation Codes Memos Visuals Queries Charts Models Matrices Text search Frequency count Matrix Comparison

31 NVivo: Sources

32 NVivo: Codes/Nodes

33 NVivo: Queries

34 NVivo: Queries word frequency

35 NVivo: System Log

36 NVivo Animated Tutorials QDATRAINING:

37 Qualitative Research should Be rigorously conducted Involve critical self-scrutiny and active reflexivity by the researcher Produce explanations and promote arguments (just not presenting mere descriptions) NOT BE SEEN AS OPPOSITE TO QUANTITAVE RESEARCH, they should be seen as complementary to each other

38 Useful References Babbie, E. (2011). The basics of social research (5th ed.). Belmont, CA: Wadsworth, Cengage Learning. Bodner, G., & Orgill, MaryKay. (2007). Theoretical frameworks for research in chemistry/science education. Upper Saddle River, NJ: Pearson Prentice Hall. Boyatzis, R. E. (1998). Transforming qualitative data: Thematic analysis and code development. Thousand Oaks, CA: Sage. Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education. NY: Routledge. Creswell, J. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage. Creswell, J. (2013). Qualitative inquiry and research design : Choosing among five approaches (3rd ed.). Thousand Oaks, CA: Sage. Denzin, N. & Lincoln, Y. (Eds.). (2005). The Sage handbook of qualitative research (3rd ed.). Thousand Oaks, CA: Sage. Denzin, N. & Lincoln, Y. (Eds.). (2008). The landscape of qualitative research (3rd ed.). Thousand Oaks, CA: Sage. Glaser, B. G. (1992). Basics of grounded theory analysis. Mill Valley, CA.: Sociology Press. Hammersley, M. (1992). What s wrong with ethnography? London: Routledge. Krueger, R. A., & Casey, M. A. (2000). Focus groups. A practical guide for applied research (3rd ed.). Thousand Oaks, CA: Sage. Kvale, S. (1996). Interviews: An introduction to qualitative research interviewing. Thousand Oaks, CA: Sage. Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco: Jossey-Bass. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Beverly Hills, CA: Sage Morgan, D. L. (Ed.) (1993). Successful focus groups: Advancing the state of art. Newbury Park, CA: Sage. Ryan, G. W., & Bernard, H. R. (2003) Techniques to identify themes. Field Methods 15 (1), DOI: / X Saldana, J. (2009). The coding manual for qualitative researchers. Los Angeles: Sage. Silverman, D. (1993). Interpreting Qualitative Data: Methods for analysing talk, text and interaction. London: Sage. Stake, R. E. (2010). Qualitative research: Studying how things work. New York: The Guilford Press. Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Thousand Oaks, CA: Sage

39 Is there anything new here that can be useful in your own research? Could you add some aspects that were not discussed today?

40 Thanks