4. Open Coding of In Vivo Codes consistent with criteria of selection and unit of analysis.

Transcription

1 Qualitative Data Analysis Assignment. There are several sections as follows. Please download this and print it out and use it to guide your answers, which should be typed in the blank spreadsheet and submitted to Assignments under blackboard. Then do the assignment, which involves creating two documents, one a Word or Wordperfect document which has the answers to the below questions. And the other a spreadsheet with your raw data and analysis. Post both as attachments to the Assignment under Blackboard. To facilitate this, following are the subject headings for the word processing document. Save this page until DETAILS BELOW as YOURNAMEQualitativeAssignment.DOC. 1. Criteria of selection (include labeled subsections A-F). 2. Unit of Analysis: Identify the unit of analysis you plan to use (p. 228)? For instance, do you plan to use words, words and phrases, phrases, phrases and sentences, sentences, sentences and paragraphs, etc.. 3. Hypothesized In Vivo Codes: Construct a list of hypotheized in vivo codes (p. 227). 4. Open Coding of In Vivo Codes consistent with criteria of selection and unit of analysis. You can do the work on a print-out of the raw data using highlighter pens, and in the spreadsheet, but then sort them into unique codes there and copy and paste them here. 5. Axial Coding/Coding Frames (the sorting of In Vivo Codes into either Frames or Themes suggested by the In Vivo Codes or based upon a hypothesized qualitative typlogy of frame/themes.) Just give me here a list of the frames, the main work is done in the spreadsheet 6. Analysis of the codes and cases based upon which frames they fall into. For each of the below, write a paragraph describing the results of your spreadsheet analysis. 6A. Code by Frame analysis 6B. Case by Code analysis 6C. Case by Frame analysis 7. Conclusions: What does your analysis say about the answer to the research question posed? DETAILS BELOW

2 As indicated, either do these steps in the Word Document which you should start, using the preceding page as the basis for it, or the Spreadsheet. Use the spreadsheet provided, Qualitative Data Analysis Assignment Blank1.xls At the bottom left, there should be a sheet entitled Raw Data. Start in that. If it isn t showing, click on the left arrow button at the bottom left to make it show. Start by placing the original actual full response in columns B and C in the Raw data tab. 1. Introduction and Criteria of selection (Word document): Note: The purpose of arriving at a criteria of selection in advance of qualitative analysis of content is to increase the reliability and validity of the analysis, by ensuring that the analysis is not done in a way which is biased towards confirming the researcher's hypotheses. The criteria of selection are not "advance codes", in other words they aren't a way in which you "guess" in advance what the answers might be, the themes you might see, etc.. They are merely a way in which you set rules in advance for what kinds of content you will select out of the total mix of content. When a research question is a very specific one, it is usually the case that the bulk of the content is related to the question at hand. Another way of thinking about manifest and latent content is that manifest content is clearly and directly related to the criteria of selection but latent content may be less clearly and more indirectly related to the content. A. State the research topic of the overall survey and the nature of the actual textual data being analyzed in this qualitative analysis. Before doing so, see p. 236 about the need to ask, "What study are these data pertinent to? In other words, what was the original objective of the research study." B. Having specified the general topic, now specify the actual research question to which that the choice of the qualitative item on the questionnaire is related. C. Specify the actual question asked to collect the qualitative data being analyzed, in other words the question on the instrument. D. Explain that your method involves first choosing the criteria of selection. Read what Berg means by the criteria of selection (p. 224), and then paraphrase what he means in your own words. E. Based upon the definition in A and the topic in B, state clearly what your criteria (single criteria or multiple criteria) are for what textual data will be included and what textual data will be excluded from your analysis. (Later, in order to implement this, a good thing to do is to use the Strikeout feature in Word to show what text is excluded because it is not consistent with the criteria of selection or in Excel you can bold-face the included text, excluding non-bold faced text.) Hint: Generally, if this is primary data analysis (original data, first time it was analyzed), the criteria of selection is that only that content related to the question is analyzed and other irrelevant material is excluded from analysis.

3 F. Also, decide for yourself if you want to address only manifest content or whether you also want to address latent content (see Berg p. 225). Explain in your own words what these concepts mean to you and explain why you do or don t want to include latent content. Another way of thinking about this is deciding how strict will you be in general in applying it? Very Strict, Strict, Somewhat Strict, or Not Very Strict? As you proceed, however, you will see that how strict you are will determine which words you actually include in your analysis and which you exclude. G. In the spreadsheet, go through column C and delete any of the words that aren t consistent with the criteria of selection. If Column B is irrelevant, put 97 in column E. If Column B basically says don t know, but 98 b in Column E. If column B is empty, put 99 in column E. (Sorry, I can t figure out what happened to columns C and D!) 4 2. Unit of Analysis (Word Document): Identify the unit of analysis you plan to use (p. 228)? For instance, do you plan to use words, words and phrases, phrases, phrases and sentences, sentences, sentences and paragraphs, etc.. See also p. 231, "What to count." (However, do not consider use of "themes", this concept will be used in a different way and it is confusing to think about themes here.) Why do you propose to count at this level? Keep in mind that the more specific the question asked, the less general the unit of analysis. In other words, very specific questions lend themselves to one or two word units of analysis, while more complex questions tend to lend themselves to longer units of analysis. In other words, specific questions tend to produce specific answers, and the unit of analysis tends to be more discrete (such as the word or the phrase), while general questions tend to produce themes which emerge or be observable only at high levels of analysis such as a sentence or paragraph. Keep in mind that you can re-analyze the same data later at a higher level of analysis, such as the concept, once one has initially identified in vivo codes at lower levels of analysis (see next section). 3. Hypothesized In Vivo Codes (Spreadsheet work): In Column F, construct a list of in vivo codes (p. 227), or the literal terms (words or phrases), which the you, the researcher, think might be used by the respondents. You should try to do this before spending time reading the responses. Later, you will do open coding of the actual responses and generate a list of the actual in vivo codes, but here you are trying to hypothesize what you think they will be. The researcher's hypotheses would tend to suggest themes would be present that might be represented by specific in vivo codes. Your in vivo codes must be (a) consistent with what is suggested by your hypothesis and (b) containable within the unit of selection you have derived from the topic.

4 Normally, if you were doing an inductive research design or research using "grounded theory", you wouldn't do this. However, in a deductive research design you would always do this, as your hypotheses would likely be tied to results which would be linked to such codes. In this case, for educational purposes, we will all do this. So, for instance, if you hypothesized that respondents would say that CMU is super, you would identify as an in vivo code the theme super. In this section, please predict, before reviewing the data, what would you hypothesize some of the in vivo codes (literal words or phrases) found in the answer might be? In a sense, these in vivo codes are the empirical indicators of the concepts for which you have provided a conceptual definition, and which you wish to operationalize with the data at hand, the "literal words". That is why it is not inappropriate to speculate in advance about what kinds of words you think you will see, as long as you do so after the criteria for selection is established. Remember, the idea is not to guess what kinds of answers there might be, only to guess what the answers might be as related to your hypothesis and unit of selection. Do not try to visit your subsequent open coding of in vivo codes into your prior hypotheses! 4. Open Coding of In Vivo Codes consistent with criteria of selection and unit of analysis. (Use the spreadsheet provided). You are now ready to do your qualitative analysis. Start with the data in a spreadsheet, and a column with the case number in column A and the full response in Column B and the response consistent with the criteria of selection in Column E. Do open coding. One goal of the analysis is to identify "minutely" (see p. 237) all in vivo codes (all actual content, not the stuff you predict in see step 2 above) at the unit of analysis selected (see step 3 above) which is consistent with the criteria of selection (step 1 above). Accordingly, using a highlighter pen, identify all such content. Some students list the in vivo codes and color code them in preparation for step 5. In most cases, words and phrases are the unit of analysis. In other words, identify the words and phrases which help answer your question, those words and phrases most actively related to answering your question. Type each in vivo code in its own cell, one line per in vivo code, in column F of the blank spreadsheet. Have the respondent number at the left. There can be more than one code per respondent, so you have to do one of two things: A. Start with one row per case (you can just paste in your original raw data). If there are more than one in vivo codes per case, high light the row below that case, right click and choose insert to insert an additional row, and put the correct case number in Column A (see example below) and the 2nd or subsequent in vivo code in column D. After you have typed in each in vivo code, save your data, then try sorting on column F and seeing what codes are truly unique, i.e. they are the only case which produced such an in vivo code, and other identical or nearly identical codes which were shared by a number of cases. Give each identical or nearly identical code a number and put that number in

5 column G. Don t mix this up with axial coding (step 5). These codes must truly be identical or nearly identical in wording at the level of analysis chosen. Now you can sort by that number, as this will facilitate axial coding. OR Type each code in column F, separated by a semi-colon. Then after you are done copy the column, paste as text to a word processor, make one line per code, and then sort the lines alphabetically in Wordperfect or Word. Then you can work on counting how many lines there are per code or extremely similar code. Once you have them you should be able to copy them into column B of the Code by Frame sheet to the right of the Raw Data Sheet. Either way, you should paste your list of in vivo codes into column B of the 6A Code by Frame spreadsheet. 5. Axial Coding/Coding Frames (the sorting of In Vivo Codes into either Frames or Themes suggested by the In Vivo Codes or based upon a hypothesized qualitative typlogy of frame/themes.) Frames are higher level abstract concepts that frame or contain one or more different codes (sometimes a single code can represent a frame). In the Word document, start a list of the frames you think emerge from the codes. You ll have to work on this, it may be easier to do with paper and pencil at first. Another place to do it is in the #6A Code by Frame sheet. The purpose of this step is to take the relevant data (the list of in vivo codes that is consistent with the unit of analysis and criteria of selection) and code it into themes (a frequently used word, not used in this way by Berg), in other words into frames. Again, sometimes students take the previously highlighted in vivo codes and further color them into different colors or number them into different numbers, one for each frame. This involves the exercise of reasoned judgment about the extent to which in vivo codes can be combined into frames that are considered to be synonymous. The goal here is to analyze results in a way which can help answer the research question, simple as that. Accordingly, to the extent that data can be coded in a way which operationalizes the conceptual definitions, and can be coded in a way which has face validity, it is valuable to collapse two or more different in vivo codes into distinct frames or themes. But there must be an clear empirical relationship between the original wording and the frames. When you can't find a word which summarizes all of them, it is often best to create a composite frame, in which you merely group all the relevant words into a group of

6 similar ones. Even if you can identify a word which best describes the frame, be sure to list the in vivo codes you have used. You end this step with a smaller list of frames derived from the in vivo codes. Normally you try to derive the frames from the in vivo codes. Frames must be mutually exclusive but they don t have to be exhaustive. In other words, you can t have two frames which mean the same thing. List these frames one line at a time below in this document and number them. For example: Frame 1. Frame 2. Frame 3. Required Plan to get BSW Interested 6. Analysis of the codes and cases based upon which frames they fall into. Analyze the codes and cases based upon the frames. First, think about the logic of your analysis here. There are several possibilities. One Case with one or more in vivo codes, one or more of which may be truly unique (the only case that uses it). One Frame, with one or more in vivo codes that you feel are expressive of that frame/theme. Of course, there are many cases, and several frames, and there may be dozens and dozens of in vivo codes. But basically, you start with many cases, you are likely to end up with fewer in vivo codes than cases due to duplication between cases (despite the fact one case may produce more than one in vivo codes), and you boil these down into frames. When you conduct your analysis, be sure you are clear on whether you are looking at cases or in vivo codes. You have to be careful how you interpret the data. If there were 10 respondents, and one respondent talked for ever and produced 100 in vivo codes and 12 frames, how much did they overlap? How much did the talkative respondent represent an anomaly? In quantitative terms, there are what is known as outliers (extreme cases), and this can be the case with such an extremely talkative respondent, but not necessarily. If that talkative respondent s 100 codes and 12 themes are much like 100 codes and 12 themes identified by the other 9 respondents, that respondent is not an outlier! So when you do your analysis, you need to consider both cases and codes and frames. So in this step, there are a couple of sub-steps, one is the #6A Code Analysis, and the other is #6B Case Analysis, and I ve created spreadsheets for each (see the tabs at the bottom).

7 Note: Do not try to force respondents into a single frame. One respondent may have several words or phrases, each of which is a code, and those codes may fit in one frame or they may be consistent with many frames. 6A. Code by Frame Analysis I ve created a sheet in the spreadsheet where you can do this kind of code analysis. There, you need to create one row for each identical/near-identical code and the number you gave it in the primary spreadsheet. And you show which frame that was associated with. When you are done, sort it using a sort which sorts by Frame 1, then Frame 2, then Frame 3, then Frame 4, all the way to your last frame. This will easily show the relationship between each in vivo code and each frame. In some ways, this is the same as the main sheet, but it has a lot fewer rows as duplicate codes are eliminated, and you don t need case numbers here. So what this part of the analysis does is analyze the in vivo codes and frames produced by the aggregation of all responses. But you have to be careful what conclusions you can draw (in step 7) based upon this, because of the problem pointed out above, that they may overly reflect unique cases. For the 6A part of this written assignment, just write up a paragraph in which you describe and summarize the results you found in the 6A part of the spreadsheet. 6B Case by Code Analysis Here all you do is count how many cases (column A) are found for each In Vivo Code. The In Vivo Codes are numbered already from the Raw Data column. Here you can try to analyse how many respondents were identified with each in vivo code. The bottom line here is to display how many cases are associated with each in vivo code. You can play with the data, count it manually, sort it, whatever it takes. You might try pasting everything you did in the Raw Data, In Vivo Codes tab into the 6B sheet and sort it and then after you get a count for each code, enter the number to the left of that code. 6C Case by Frame Analysis Here you analyse how many respondents were identified with each frame! The bottom line here is to display how many cases are associated with each frame. You can play with the data, count it manually, sort it, whatever it takes. You might try pasting everything you did in the Raw Data, In Vivo Codes tab into the 6B sheet and sort it and then after you get a count for each frame, enter the number to the left of that code. In writing this up, it is still a problem when you try to generalize, since there can be more than one frame per case!!! But discuss that! Optional: If you think it is feasible, try forcing each case into one primary frame. This would make it easier to generalize from your sample.

8 7. Conclusions: What does your analysis say about the answer to the research question posed? Based upon this analysis, how would you interpret the results? In other words, what does this qualitative analysis tell you about the answer to the original question? Briefly describe the outcome. How would you interpret the results, given the problem that there can be more than one frame per case.