Clicker Lessons: Assessing and Addressing Student Responses to Audience Response Systems Tim Pelton Mary Sanseverino Leslee Francis Pelton University of Victoria Outline Introduction Study Context/Data/Methodology Results/Interpretation Discussion A self-evaluation tool ARQ1: Why might we use an audience response system? a. Encourage student engagement b. Identify misunderstandings/guide instruction c. Open up discussion on sensitive topics d. Support students development of self confidence e. Support grading of students 1
Goals of this research Generate and validate a Clicker Use Survey Generate a set of baseline distributions to support teacher reflection and self-evaluation Examine impact of clickers on student learning Examine impact of linking ARS to grading practices Compare two specific audience response systems Systems used in study einstruction RF, dual platform Online registration Ongoing fees model Higher threshold and friction iclicker RF, multi platform Class based registration One-time cost model Open source software 2
The Study Context/Data Methodology 1107 participants over 11 classes Coding was reversed for negatively oriented items Generated baseline response distributions and comparison group distributions. (Class, Grading, System) Both a priori and post hoc comparisons Distribution comparison - Likert Linear Limitations All instructors who used the graded model were using einstruction Instructors who chose to link clicker responses to grades were less experienced Instructors required purchase and/or online registration of einstruction units 3
ARQ2: How do you think you would answer: Class time passes more quickly when we use clickers a. Strongly Disagree b. Disagree c. Undecided d. Agree e. Strongly Agree 1. Class time passes more quickly when we use the clickers. 6 5 3 2 1 Class 10 Class 8 Class 6 Class 4 Class 2 2. When we use the clickers my participation increases in other ways too. 6 5 3 2 1 Rati C Class 11 Class 10 Class 9 Class7 Class 6 Class 4 Class 3 Class 2 Class 1 4
Student comments: Clickers were very helpful in understanding the course material. They forced us to have an active role in the class and actually think, but without being in the spotlight. They work. Some people don't take them seriously, but it is a good balance of class feedback and anonymity. 3. I found the use of clickers to be distracting and unhelpful (reversed). System Used 5 45% 35% 3 25% 2 15% 1 5% iclicker einstruction ARQ3: Clickers will increase classroom participation without increasing student discomfort a. Strongly Disagree b. Disagree c. Undecided d. Agree e. Strongly Agree 5
4. I feel uncomfortable sharing my responses via the clickers (reversed). System Used 6 5 3 2 1 Rati iclicker einstruction Student comments: The clickers for this class were supplied by the school so each person's individual response was confidential, but the class information as a whole was displayed. In other classes person individual response is used. I prefer the confidential nature of how this class was run and would have responded differently had I been in the other situation. 5. Learning with clickers improves my understanding of the course content. 6 5 3 2 1 Rati Class 9 Class 8 Class 4 Class 3 Class 2 Class 1 6
Student comments: Clickers made the class learning feel more active and less passive. In a big lecture hall it's easy to space out or just write down the notes without processing tem (sometimes I go home and don't remember things I've written down). Clickers helped me to process the material better in class because I had to think instead of just listening. 6. Learning with the clickers gives me confidence to ask more questions. 45% 35% 3 25% 2 15% 1 5% Rati Ungraded Graded ARQ4: Grading responses will increase the amount of time a student will spend preparing for class a. Strongly Disagree b. Disagree c. Undecided d. Agree e. Strongly Agree 7
7. Using the clickers encourages me to spend more time preparing for class. Participation/Response Graded 5 3 2 1 Ungraded Graded 8. Using clickers encourages me to attend more classes. Participation/Response Graded 35% 3 25% 2 15% 1 5% Ratin Ungraded Graded 9. Using the clickers promotes more focused discussion during class. 6 5 3 2 1 Class 11 Class 9 Class 8 Class 6 Class 5 Class 4 8
10. I would like to use the clickers in other courses. 6 5 3 2 1 Class 10 Class 9 Class 7 Class 6 Class 4 Class 3 Class 2 Class 1 11. The graphs provided by the clicker system are useful. 6 5 3 2 1 iclicker einstruction Student comments: At first it was tough to get this system running and we wasted a lot of time trying to figure it out. But once they were functional, I found the clickers really helped and I did want to go to class more often. I like seeing the graphs and knowing how other students responded. I actually started participating once we were using the clickers. 9
12. I benefit by seeing how other students respond to a question. 6 5 3 2 1 Rati Ungraded Graded 13. Using the clickers in class is too time consuming. (reversed) 6 5 3 2 1 iclicker einstruction Student comments: I found that the questions posed were not always incredibly relevant and that it took too much time away from teaching. They were finicky and I don't believe it focused much discussion instead they just allowed for a chat break in class. I thought the clickers were easy to use. Their helpfulness depended on mostly how helpful the questions was to me. They didn't seem to take up too much class time at all. 10
14. I would do better in this class without the clickers.(reversed) 5 45% 35% 3 25% 2 15% 1 5% Ungraded Graded Student comments: Clickers were fun at the start, but near the end it was just as beneficial to watch as opposed to participate. Good for xxxxx class questions. Useless for class yyyyy. Could have easily raised hands. 15. I had difficulties getting my clicker to work in class. (reversed) 6 5 3 2 1 icliicker einstruction 11
ARQ5: I would have liked to used the clickers more often in this presentation a. Strongly Disagree b. Disagree c. Undecided d. Agree e. Strongly Agree 16. I would have liked to use the clickers more often in class. 6 5 3 2 1 Rati Class 11 Class 10 Class 8 Class 6 Class 4 Class 3 Class 2 Class 1 17. At first, learning with the clickers was enjoyable but later I was bored (reversed). 6 5 3 2 1 Ungraded Graded 12
18. Using the clickers helped to better prepare me for quizzes and exams. 5 3 2 1 Class 11 Class 10 Class 8 Class 6 Class 4 Class 3 ARQ6: Which of the following clicker related factors do you expect would have the greatest positive impact on student learning? a. the content of the course b. the instructor s teaching style c. the instructor s usage of the equipment d. grading participation or responses e. the type of clicker system adopted Summary of Survey Findings Students using clickers: generally felt comfortable sharing responses found benefit from seeing the graphs and others responses had less positive responses when marks were associated with clicker use Analysis Single factor accounts for 40-5 of var. Some statistically significant differences - but Clickers did not appear to: give students more confidence to ask questions encourage students to prepare more for class 13
Discussion We had some confounded factors (system, grading, teacher experience, course content, self-selected) limit interpretation of some results Note other advantages of clicker MC questions - they supply students with question ideas (language, concepts, etc.) Distribution differences between classes or groups are generally as expected Other points? Introducing a tool to support teacher self-evaluation of clicker use Survey is available to download and use in your class - 10 min administration - simple tally Spreadsheet containing baseline distribution provides a meaningful comparator Identify differences and reflect (TD) Plan/adjust usage to affect change (TD) Transmit distribution summary to support ongoing baseline development (optional) Share thoughts on issues affecting distributions (optional) Contact us Tim: tpelton@uvic.ca Mary: msanseve@uvic.ca Leslee: lfrancis@uvic.ca Resources: http://www.csc.uvic.ca/~msanseve/class_clickers/ 14