Eective strategies for online testing in large classes

Transcription

1 School of Mathematics and Statistics University of New South Wales, Australia Maple TA User Summit 2014

2 UNSW Australia and me University of New South Wales Sydney Australia students Large Science and Engineering Faculties 25% international students UNSW in top 8 of 35 Australian Universities School of Mathematics and Statistics teaches approximately 3000 rst year students Senior lecturer in the School of Mathematics and Statistics At UNSW since 2001 Creating and administering online tests since 2004

3 Courses using Maple TA at UNSW School of Mathematics and Statistics MATH1131/1141 Science and Engineering (1700) assumed knowledge / class test preparation / online Maple / invigilated Maple lab MATH1151 Actuarial Stuides (200) class test preparation / online Matlab MATH1011 Lower level rst year course (100) weekly tutorial preparation, online Maple MATH1031 Maths for Life Sciences (400) class test preparation, Maple MATH1041 Stats for Life Sciences (400) online quizzes, invigilated R or Excel lab Higher Year Stats/Numerical Methods/Engineering Maths (2000) intro to matlab, various online tests Faculty of Engineering Higher year undergraduate and Masters Engineering (1000) mostly longer assignments

4 Why use online testing and Maple TA? Regular testing encourages students to keep up to date Provides immediate feedback, anytime, anywhere Reduces labour on routine marking (What should be done with that saving?) Allows repeated attempts according to student need Personalisation of learning encourages collaboration that isn't just copying Analysis of student responses can be used to improve teaching Very exible algorithmic question generation and grading It lacks features we don't want during invigilated tests eg, messaging or access to course materials Plan to use adaptive questions in the future

5 Dierent scenarios for use of Maple TA Formative assessment for learning Assumed knowledge quiz (placement test) for students in MATH1131 who should be in MATH1011 Class test preparation in the week preceding a written class test Weekly tutorial preparation a short online test preceding each week's tutorial Progressive (mastery) tests must pass each test to access the next test Practice followed by actual test unlimited practice but limited number of attempts count Practice tests no marks awarded, feedback only Summative assessment of learning Invigilated lab tests traditional exam conditions but Maple TA graded

6 How students view Maple TA Students expect to be rewarded with marks. Students expect things to work without having to read instructions. Student confusion about the naming of Maple TA and Maple lead to an increase in student use of Maple! From a MATH1131 survey in 2011:

7 Student comments (2013 and 2014) Feedback from quizzes was very helpful. Constant tests are good. Online tests are mostly just annoying and take up valuable time. (Course could be improved by) more online questions... The online tests and Maple tests instead of assignments were the best features for this course. The online tests helps in getting a better grip over the subject. (Best features of the course) on-line tests I found the online tests provided a good revision tool as they provided immediate feedback which I could use to improve. Should have more smaller online tests on maple ta so that it forces students to learn and be up to date with the course as it moves quickly. Regular online and class tests prompt us to be continually up to date... The online quizzes kind of force you to learn how to do each question, and since you have 5 tries you can work on a question until you get it right. More weighting on online tests. The time used to complete the online tasks was not appropriate in comparison to the percentage it makes up. Online quizzes were well set out. It was useful to have several attempts.

8 Engagement Do students engage with tests? In MATH1131 online tests are worth approximately 1% each. Approximately 90% of students attempt each test. In MATH2019 online tests do not count towards nal grade. Between 1% and 10% of students attempt the tests. Incentives Marks count towards nal grade 0.5% per 30 minute online test seems sucient! Immediate feedback Flexibility allow some tests to be missed or failed eg only count best 8 of 12 allow repeated attempts only count best mark allow plenty of time More and smaller tests to avoid unnecessary repetition

9 Set and forget Our main current use of Maple TA is to provide practice questions and feedback to students with minimal intervention. Provide written and screen capture video instructions. Link to Maple TA provided next to link to instructions. Maple TA class homepage tells students to read the instructions. Compulsory Using Maple TA test must be completed rst.

10 Using Maple TA

11 Using Maple TA

12 Set and forget Allow multiple attempts. Revision only versions of tests available after deadline. Course outline states no extenstions or additional attempts allowed. Only best attempt counts. Allow some tests to be missed without penalty. Questions contain embedded instructions.

13 Tutorial preparation tests in MATH1011 Encourage regular preparation for tutorials. MATH1011 has separate algebra and calculus components. Weekly tutorial preparation tests for the calculus introduced in Amnesty for missed tests in week 5 of semester. Course survey comments requested similar tests for algebra. Evidence of increase in average score for nal exam questions related to online tutorial prep questions.

14 Tutorial preparation tests in MATH1011

15 Class test preparation Provide feedback to help students prepare for class tests. Initially allowed 3 attempts but spent too much time dealing with special pleadings for extra attempts so increased to 5 attempts.

16 Invigilated tests Maple component of MATH1131 is assessed with invigilated lab test students book time for test on Moodle student ID cards are checked computers (112) have exam mode with no internet access Before 2010 paper based test student Maple worksheets collected, printed then marked by hand and returned in tutorial class very labour intensive students had no feedback prior to test Since 2010 Maple TA test test is available for unlimited practice on Maple TA before lab test test restricted to lab IP range easy immediate grading once all tests have nished Since 2014 MATH1041 Stats for Life Sciences has an invigilated lab test partly graded by Maple TA.

17 Maple Lab Test

18 Allowing for special cases Although we say no extenstions or additional attempts will be granted, in reality there will be special cases. We use revision only versions for special cases. Revision only tests also depend on Declaration and Using Maple TA tests. Revision only test become available as soon as `for marks' deadline has passed.

19 Summary Expect bugs in your questions. Expect students to do uncontrolled tests collaboratively. Students won't attempt optional tests. Assume students won't read the instructions. Assume students will start the test when they are not prepared. But if you just want students to learn, the solution to all of this is...

20 Summary Expect bugs in your questions. Expect students to do uncontrolled tests collaboratively. Students won't attempt optional tests. Assume students won't read the instructions. Assume students will start the test when they are not prepared. But if you just want students to learn, the solution to all of this is... don't worry! Oer enough marks that the students care but you don't. Provide lots of ways for lost marks to be recovered. Have a strict non-intervention policy.