T-106.5550 Research methods Spring 2016 7.4.2016 7.4.2016 1
Today Data collection revisited Reflections on the data collection task Qualitative data analysis methods Grounded theory exercise on qualitative data analysis Taking a critical look at results Validity / reliability / trustworthiness issues Setting stage for the final day 2 7.4.2016
Aspects for choosing methods 2) What is your research question(s)? 1) Which phenomenon you are interested in? How you can represent it? 3) What do you want to achieve with your research? Why do you do your research? The choice of research method(s) 4) What do you think is valid knowledge? What is your paradigm? 3.3.2016 Kinnunen, 2015 course 3
The six principles Pose significant questions that can be answered empirically Link research to relevant theory Use methods that permit direct investigation of the question Provide a coherent and explicit chain of reasoning Replicate and generalize across studies Disclose research to encourage professional scrutiny and critique (Shavelson, Towne: Scientific research in education, 2002) 4 3.3.2016
Designing research Research questions Choosing research framework Choosing data sources and collection methods Choosing data analysis methods 5 3.3.2016
Data collection Where do we get our data? Pre-existing data Documents, reports, data bases, Natural setting Data collected or available as a natural part of process, such as, exam results, essays, submitted project works, demographic information Research setting Data collected specifically for the research, such as questionnaire data, interviews, observations, pre/posttest results, specific assignments and exam questions, instruments Laboratory setting Research data collected in a specific occasion designed for the research, like eye-tracking data, lab task data, Technical system Log data, performance data 6 3.3.2016
Other aspects When using natural, pre-existing or log data, the research does not generally interfere with the investigated phenomenon. In most cases, collecting research or laboratory data is needed, which can make data collection more complex to organize. 7 3.3.2016
Ethical aspects When collecting human data, ethical aspects need to be considered. Possible ethical guidelines of the university Human consent forms Withdrawing data from research? Anonymizing data It is not just using statistics or pseudonymes Storing data What can be published, e.g., from interviews Reviewing data (by sources) Can data be reused for other research? Destroying data 8 3.3.2016
Data collection task revisited Work in pairs Design a pilot study in two phases to find out, how to improve the working practices of your research groups. Prepare a questionnaire with both closed and open-ended questions Collect data from both groups Prepare a summary Carry out 3 interviews with your colleagues (some 15 minutes) to clarify the results of the survey Transcribe the interview data Bring the transcription to the next session in April, where we analyze it, as well as submit is to MyCourses by April 4 th 9 7.4.2016
How to do interviews Different types of interviews: Structured Ready made answering options (= filling in the questionnaire) Semi-structured No ready made answering options Theme interview Themes that interviewer goes through with each interviewees but not necessarily in the same order Wording and timing of the question may change from interview to interview Open interview All themes are not necessarily covered with all interviewees Deep interview Several long lasting interviews with the same person, themes vary Group interviews T-106.5550 Research methods spring 2013, Päivi Kinnunen 10
Procedure Review your research questions & do a literature review and see if interview is a suitable data collection method Decide on the interview type Do an interview plan (questions you are going to ask) Consider the order and wording of the questions (no leading questions) Consider could/should you use some material (photos, videos, homework submissions ) during the interview to help the interviewee remember the situation better/ in more detail Test the interview plan (make a pilot/rehearsal interview) Find interviewees and conduct interviews Transcribe discussion into text (consider how much details and additional information you need to transcribe, e.g., long pauses, laugh, sights ) Analyze the data T-106.5550 Research methods spring 2013, Päivi Kinnunen 11
Interviewer s tasks Before the interview Find out if you need a permit to do your research Write an interview plan Learn your interview plan by hard Test your interview plan by making a pilot interview Find people to interview. Try to find few more people than you think would be enough some might cancel or be very quiet. Find a neutral location for your interview Schedule interviews (preferably no more than 1-2 interviews/day) Test the sound recording devices Prepare for interviewees who do not want that their interview will be recorded Find out what kind of dress code the interviewee s work place/community follows and try to fit in/wear neutral clothes. T-106.5550 Research methods spring 2013, Päivi Kinnunen 12
During the interview Introduce yourself Tell why you are interviewing, who is going to have an access to the data, and to what purpose the data will be used Tell you discussion will be confidential keep your promise (no discussions in corridors/cafeterias or in a crowded bus with your colleague/friend!) Ask permission to record the interview, test that your device is really recording Create a trusting atmosphere Do not talk about your own opinions & experiences on the topic you are there to give the interviewee a chance to talk (interview casual discussion with your friend). Give the interviewee time to think and answer Listen attentively Show interest towards what the interviewee is saying (don t overdo it though, try to be neutral). Encourage for more elaboration of the question, if needed Keep conversation on the topic Take care of timetable, it is your job to make sure you start/end on time T-106.5550 Research methods spring 2013, Päivi Kinnunen 13
After the interview Transcribe the notes you made during the interview Make backups of the audio file, create some logical way to name the files Store the files in the safe place Transcribe the interview, add date, time, the name of the interviewer and interviewee to the file Transcription will take about 4-10 times more time than the actual interview (Let the interviewer read the transcription through s/he might be able to fill in some missing words or s/he might remember something she wants to add to the discussion) Read the transcription through several times so that you know it well Analyze the data, interpret the results (Send the results of the analysis to the interviewers do they agree with your interpretation?) T-106.5550 Research methods spring 2013, Päivi Kinnunen 14
More information on interviews Kvale, S. (1996). Interviews. An Introduction to Qualitative Research Interviewing, London: SAGE Publications. http://srmo.sagepub.com/view/doing-interviews/n1.xml? row=3&rskey=rqnnnj http://www.socialresearchmethods.net/kb/intrview.php T-106.5550 Research methods spring 2013, Päivi Kinnunen 15
Discussion Your reflections on data collection Challenges in preparing the questionnaire? Are collected results valid? Challenges in the interviews Problems with preparing the interview script? Did the results of questionnaire provide some extra info here for your script design? Was the interview script working? How did the interview work out? Any issues with transcribing the interviews? What would you do in a different way now? 16 7.4.2016
Break 17 7.4.2016
About qualitative research Päivi Kinnunen s slides from last year course 18 7.4.2016
Qualitative analysis Päivi Kinnunen s slides from last year course 19 7.4.2016
Research plan, part 3 Finalize your research plan by adding the following: Data analysis plan Arguments for choosing analysis methods Discussion on validity / reliability / trustworthiness issues in your plan and how you will try to mitigate them. Future work after this part of your research has been carried out, if any. Initial publication plan Submit it to MyCourses by April 29 th Prepare a presentation of your plan for the final session on May 9 th. (20 minutes at most) 20 7.4.2016
Research plan, peer review On May 2 nd I will send the plans to all of you. Prepare 3-5 to questions on each plan concerning what possible problems you see in it, and how it could be improved or extended. These questions will be discussed on May 9 th in the final session after each presentation. 21 7.4.2016