Analysing Qualitative Data Workshop Professor Debra Myhill
Philosophical Assumptions It is important to think about the philosophical assumptions that underpin the interpretation of all data. Your ontological position, your view of the nature of reality or truth; Your epistemological position, your view of the nature of knowledge or knowing Research is sometimes seen as divided into two paradigms: positivist with a positivist ontology and epistemology; and interpretivist, with an interpretivist ontology and epistemology. (though things are rarely this clearly divided!)
Positivist Perspective A positivist ontology and epistemology: the world out there is objective, with truth and reality independent of human action or thinking. So positivist research sets out to establish objective truth. For example, in the case of an educational intervention, a positivist view would hold that the intervention is an empirical way to establish a single, verifiable truth which informs what works..
Interpretivist Perspective The interpretivist ontology and epistemology: social interactions shape multiple truths and multiple realities within any given context. So an interpretivist view of an educational intervention would see it as a socially-mediated enterprise. Thus, whilst the research might seek to determine the efficacy of the intervention, it would do this through investigating the different ways the intervention is realised in different classrooms and the different ways it is experienced by different teachers, acknowledging that even a highly efficacious intervention cannot be simply replicated in different school settings. Thus the research sets out not to discover the single truth of the efficacy of the intervention, but the multiple truths of the nuanced and socially-constructed ways in which the intervention is realised.
Why does this matter? Qualitative data is crucially about a process of interpretation; of making meaning from data. It is hard to approach qualitative data from a positivist perspective. Qualitative data is not inferior to quantitative data, but it is different kind of data. It is important to know what qualitative data can do that quantitative data cannot, and vice versa. A questionnaire and an interview can investigate identical constructs but they do so in different ways (and with different philosophical assumptions).
Discussion point As junior researchers, what are your philosophical assumptions about truth and knowledge? Have your research experience between predominantly positivist or interpretivist? Can you explain why a questionnaire relies on a positive perspective whereas an interview is interpretive? Can you think of ways which would make a questionnaire more interpretive or an interview more positivist?
The Importance of Coding Any researcher who wishes to become proficient at doing qualitative analysis must learn to code well and easily. The excellence of the research rests in large part on the excellence of the coding. (Anselm L. Strauss, Qualitative Analysis for Social Scientists, 1987, p. 27)
What is a code? A code is a word or phrase that attributes a summative, descriptive label to a segment of language-based or visual data Data Extract (HA)When girls come into school they are really quite ready just to sit down and to learn (UA) Well what can I say about Jade she just does not see the importance of school I don't think men have the same sense of urgency as women (HA) Emily is very lively and enthusiastic about all of her school work Possible Codes? girls readiness for school; girls as passive; girls fit school life negative statement about an underachieving girl; girl underachievers attitude to school differences between men and women; gender as dichotomous; women as driven girls as active; girls engaged in school work; high achievers and engagement
Qualitative data Qualitative data can take many forms Transcripts of discussions or focus groups Transcripts from school lessons or any real life setting Field notes Observation records Video data/audio data Photographs Children s drawings Children s written texts Interview transcripts
Qualitative (interview) data is rich You can explore opinions and attitudes Tell me more Do you mean... Why and how questions You can be faithful to the tentative, speculative nature of people s responses You can reflect on the emergent, fluid nature of people s ideas rather than fixing them in stone You can explore the creation of meaning You can pick up on recurrent themes and ideas You can explore what is informing their ideas
Coding procedures Bottom-up coding: beginning with no predetermined codes and working closely with the data to determine the codes (sometimes referred to as a grounded theory approach). This is an inductive process. Top-down coding: beginning with a set of pre-determined codes, often using a coding frame, and analysing the data for segments which match those codes. This is a deductive process.
Bottom-Up coding Stage 1: Open coding In the initial stages, open coding is iterative, fluid, messy and emergent Through immersion in the data, the researcher attributes codes to segments of data The naming of codes is tentative at this stage and subject to change and refinement As more data is analysed, typically the initial tentativeness develops into a set of codes which fit the data well This is a micro-level process: the researcher is not thinking about the big picture but about what codes best describe the segments of data The researcher undertakes a process of constant comparison (Glaser and Strauss 1967)
Keeping Records Managing qualitative data analysis is complex and it also occurs over a long time period so keeping records of what you have done is important. Write definitions of your codes so that you can remember exactly what each code was capturing Write memos or keep a research journal capturing your thoughts about the coding or about the bigger ideas that you might be seeing When open coding is complete, keep a record of the codes (number and names) attributed at this stage This information is an important part of writing up the analysis.
Research Log 01/12/2009 07:59 I m struggling with comments that reflect teachers' confidence in teaching writing eg I am confident teaching creative writing; or I 'm not sure how to teach viewpoint. At the moment they are going in Teacher as Writer (or Genre Knowledge) but neither are correct. We may need a code about pedagogical confidence? 10/12/2009 07:08 Practising and exercising is nearly always interpreted negatively; even though they often did this in the lessons.
Workshop Task Read the interview extract supplied. It is from a research study where teachers were interviewed to determine their beliefs about grammar teaching and their pedagogical practices in teaching grammar. Working in pairs, undertake open coding on this interview, bearing in mind the research focus, attempt to attribute codes to segments of data. You might: Code two words or a much longer segment of data Attribute two codes to the same or boundary segments
Constant Comparison While coding, the researcher undertakes a process of constant comparison (Glaser and Strauss) The method of comparing and contrasting is used for practically all intellectual tasks during analysis: forming categories, establishing the boundaries of the categories, assigning the segments to categories, summarizing the content of each category, finding negative evidence, etc. The goal is to discern conceptual similarities, to refine the discriminative power of categories, and to discover patterns. (Tesch, 1990:96) In practice, constant comparison means constantly reading and re-reading data and codes being currently attributed to data and comparing with the same code attributed in earlier coding and checking both consistency and appropriacy. It also means making decisions about coding names, sometimes merging two codes into one, or sometimes splitting one code into two.
Workshop Task Work with another pair, and compare how you have both open coded the interview extract Are there times where your coding is totally different? Are there times where you have coded similarly but the labels have slightly different meaning? Are there times where your coding varies in terms of how much of the data you have segmented to one code? Can you discuss and reconcile these differences to arrive at one set of codes by constantly comparing each other s codes and naming? NB Constant comparison is really a process for comparing across data, not coders: how your second interview coding compares with your first interview; the coding that arises from this study compared with the coding that arose from a previous study etc
Bottom-Up coding Stage 2: Axial coding When all the data has been analysed through open coding, there are usually a high number of codes At this stage each code stands on its own; the coding has established no relationship between codes Axial codes establishes relationships between the codes, usually through clustering related codes together under an over-arching heading. This stage also frequently involves further merging, splitting and refining the code. NB Although open and axial coding are expressed as separate stages, in practice there is overlap sometime clusters of related codes become evident during the open coding stage
CODES AND SUB-CODES FOR THE THEME: GRAMMATICAL CONTENT KNOWLEDGE Fear, Anxiety or Inadequacy Grammar as Technical Skill Grammatical Confidence Grammatical Knowledge Problems Making Connections between Grammar and Writing Adding to Improve Lexical Choices Syntactical Choices Making Informed Choices Linking Grammar and Writing Pedagogical problems with Grammar Problems with Grammatical Explanations Problems explaining Sentences and Syntax Problems handling students grammar questions Unanticipated classroom problems Perceptions of Students' Grammatical Knowledge Forgetting Previous Grammar learning Student confidence Student Difficulty Grammatical Terminology
Workshop Task Using the extract you have just coded, can you see: any relationships between the codes? any codes that could be clustered together? any headings that would describe the clusters? You can use tree diagrams, concept maps, tables whatever you like to help you do this.
Top-Down Coding A simpler process because you are not making decisions about coding names and how best to describe your data, but are simply looking to match data against the predetermined categories On a positivist-interpretivist continuum, it tends to be regarded as more positivist, as it relies less on interpretation It may be perfectly appropriate for some research questions; or indeed it may be used in tandem with bottom up coding. For example, analysing interviews with children about their metalinguistic knowledge using Gombert s taxonomy of metalinguistic knowledge as a coding frame.
Top-down or Bottom-Up In practice many real qualitative data analyses use a bit of both. So a top-down coding frame may well have space for coders to add new codes which they find in the data A bottom-up coding process may decide to systematically capture all segments of data on a specific pre-determined category eg all references to grammar as technical or mechanical, at the same time as going the process of open coding.
Trustworthiness Qualitative research uses trustworthiness in place of the concepts of reliability and validity Other researchers need to be able trust your data because they cannot access, or do not have time to access it, to do the analysis themselves. Qualitative research opens up disagreement as a positive process: if others can see how you have coded, they can disagree with your choice; Much of the trustworthiness is achieved through the rigour of the analysis process and keeping a record of the decisions made Trustworthiness is undermined by cherry-picking your data for evidence which suits your view, and by ignoring contradictory data.
CONCEPT METHODS USED TO SECURE TRUSTWORTHINESS Transferability: The extent to which the findings can be generalised to other contexts with different participants and situations Dependability: The extent to which similar findings would be obtained if the study were repeated Confirmability: The extent to which the findings are neutral and free from researcher bias Credibility: The extent to which the findings seem believable The collection of 3 interviews/observations per participant to achieve greater descriptive depth Sampling strategy which includes randomisation and which maximises number of different context represented (32 schools with one class per school in different geographical locations). Constant comparison across cases during data analysis Triangulation of data through teacher and student interview, lesson observation and statistical data Maintenance of a research log of coding decisions and project log of methodological decisions Systematic coding consistency checks for inter-coder agreement More than one team member involved with all aspects of data analysis Alternative interpretations of data sought through team meetings Open coding sought to code as much of the data as possible Research team reviewing of coding processes and code labelling Cross-checking results with statistical results, where applicable The use of multiple sources of data The discussion of results with teacher participants at a plenary conference The avoidance of over-generalisation or over-claiming in all reporting of data
The Analysis Process Coding qualitative data is painstaking and slow Researchers need discipline to constantly monitor their consistency Working in small research groups where coding is done collectively, separately, then checked collectively is a very good approach Quite often, at a late stage in the process, some data needs to be recoded because of a problem or new insight that has emerged. But it is highly rewarding!