Whenever the subject of distance education comes up, teachers inevitably. Designing Online Tests. The Role of Assessment in Web-Based Learning

Transcription

1 Designing Online Tests The Role of Assessment in Web-Based Learning In addition to being effective, Internet testing Whenever the subject of distance education comes up, teachers inevitably raise questions about testing, grading, and cheating. How should online assessments be designed? Do traditional methods of testing work, or are new approaches required? This article describes some of the benefits of online testing and identifies special consideration for Web-based testing. is user-friendly. Benefits of Online Testing The Internet provides an effective avenue for testing students on knowledge and attitudes. More and more certification examinations are available online, offered through computer-based testing centers. Researchers report that tests conducted over the Internet yield similar results to standard paper-and-pencil tests in a traditional classroom, leading to the conclusion that online tests can be as valid and reliable as any other kind of exam. 1 In addition to being effective, Internet testing is user-friendly. Researchers in Pennsylvania who studied 400 adults reported that a majority of students indicated, after taking an online examination, that they preferred taking the test over the Internet to the more conventional method, even though more than two-thirds of the group had had no previous experience with the Internet. 2 Students like computer testing for a variety of reasons, including the fact that most of them are very comfortable working online. They also like getting their results more quickly often immediately. Many teachers report that Internet-based tests take less time to prepare and conduct than conventional tests. An important benefit of Internet testing is that data can be more easily, thoroughly, and quickly analyzed. Many instructors who use a Web-based instructional program, like WebCT or Blackboard, find the assessment manager quite easy to use. Quia.com is another user-friendly Internet testing program for designing quizzes or tests. 3 By Gail Rice Journal of Adventist Education April/May

2 Special Considerations for Web- Based Testing When you use the Internet as a testing resource, you will find that although test question development and overall test design may remain essentially the same, you will have new and different decisions to make. The following guidelines may help minimize difficulties as you undertake this new challenge: 1. Use tests and quizzes less for grading and more for learning enhancement. In general, tests and quizzes will play a different role in the online class than in the traditional classroom. Whereas in the face-to-face setting, tests were essential elements in assigning grades, now they will be more valuable in the broader instructional scheme. Researchers report that tests conducted over the Internet yield similar results to standard paper-and-pencil tests in a traditional classroom, leading to the conclusion that online tests can be as valid and reliable as any other kind of exam. If possible, limit tests and quizzes to less than 50 percent of the grade. Replace some of the quizzes with discussion postings, application exercises, written work, and creative projects Decide how and when feedback will be delivered to students. A major advantage of using computers for testing is the ability to provide quick response. Immediate feedback offers tremendous benefits. It is a powerful motivational tool. As time passes, students care less and less about how well they did on test questions. When they get feedback right away, their interest and desire to get it right remains high. Quick response makes tests feel more like games and less like drills. This keeps students motivated and enhances class participation. 5 It allows students who miss a 32 Journal of Adventist Education April/May 2003

3 question to receive a response from the instructor through the feedback mechanism of the online test. The teacher can thus quickly clear up misconceptions, or provide a link to the original material or to additional resources. 3. Decide if and how tests will be graded. Your overall assessment philosophy will help you establish policies on grading. Will individual tests or quizzes count toward the student s final grade? If so, how much? You can choose from a number of possibilities when grading tests: (1) Students earn points for correct answers on a graded test, which apply toward a final grade an approach students are very familiar with. (2) Taking a quiz counts for a set number of points; quizzes are not graded. (3) Students can retake certain quizzes until they have mastered the material. Only the grade on the final quiz is recorded. (4) Quizzes are treated as learning devices for self-assessment with no grades or points being given for their completion. In subjects that emphasize the learning of facts, such as some science and history courses, multiple choice, matching, and true-false questions work well. Mastery of other subjects, such as philosophy or creative writing, will be more difficult to test using these types of exams. 4. Decide whether the test will be closed book/ closed- Web or open-book/ open-web and whether it will be timed. In online testing, it is generally best to use open-book/ open Web quizzes or tests, rather than closed-book/closed- Web tests. You can encourage students to study before an open-book quiz by timing the quiz. Every course-management system has built-in features so that you can time students the test disappears or they are shut out after a certain amount of time. This helps to keep students from searching through the textbook as they are writing the test. There is simply no time to look for answers! Keep in mind that some students with special needs may require special arrangements or exceptions when taking timed tests. 5. Decide on the setting for the test and establish procedures to resolve technical problems. If you are teaching a totally online course with no face-toface contact (in contrast with a Web-enhanced course, which combines classroom time and online resources), you will likely find it difficult to use monitored tests. Therefore, you will probably need to design your online tests so students can take them anywhere they can find a computer with an Internet connection. Students often feel very anxious about taking an online test on their home computer, which may not have high-speed, dependable Internet access. No one wants to be kicked off the Internet just before completing the exercise or Many teachers report that Internetbased tests take less time to prepare and conduct than conventional tests. hitting the all-important Submit button! When learners cannot take or complete a test because of computer crashes, aborted Web connections, or failed servers, they need to know how to proceed. Provide them with instructions and encourage them to keep these directions available in case a problem occurs. Simply knowing the teacher has a contingency plan in case of a technical problem will help to allay some of the anxieties many students experience when encountering a new learning environment. Again, minimizing the role of unmonitored tests and quizzes in calculating grades will make some of these concerns more manageable. 6. Decide whether you will allow learners to retake the test. If you let learners take a quiz or test a second or third time, this can help them to improve their learning and possibly their scores, if the test counts toward the grade. However, it is best to use different questions on the second test, since reusing the questions may not measure anything other than test question recall. Most course-management/learning systems will have ways to create a different form of the test for subsequent attempts. If you allow students the option of retaking a graded quiz or test, you will need to clearly communicate your policy about scoring. Will you record the first and last attempts, the last attempt, the best attempt, or an average of all attempts? In any case, it is not usually wise to allow more than two or three retakes Develop a comprehensive test with valid questions. It is always a challenge to develop a good test. Online tests are no exception. While most test development engines available on the Internet help you create a good-looking, dynamic test document, you are is still responsible for the content. Start with a test plan or blueprint, in which you outline the objectives to be tested, the cognitive level at which each of the objectives will be assessed, and the type of question best suited for that learning objective. 7 Online quizzes and tests should be primarily objective in nature, in order to maximize the computer s testing capabilities. The three most common types of objective questions used in online testing are multiple choice, true-false, and matching. Because a good test is no better than its individual parts, seek to create well-written questions that encourage higher cognitive functioning. Review student responses each time the test is given to check for confusing wording or questions that many students answer incorrectly. See Tables 1, 2, and 3 for a brief review of the principles for developing high-quality items that don t confuse students or inadventently provide clues that make the correct answer obvious Take advantage of the testing capabilities of the Internet. Journal of Adventist Education April/May

4 Depending on the capacity of your server, you may be able to do some really creative things with online tests. For example: Have students view a short video clip by clicking on an icon and then answering questions about that clip. Include downloadable detailed drawings, diagrams, photographs, or images in your questions. Have students click on an icon to hear a musical recording about which they are to answer questions. Include specialized kinds of questions, like drag and drop questions, peek-a-boo questions, or click-in-picture questions. 9 Table 1 Checklist for Multiple- Choice Items Does each question address a specific learning objective? Do a large proportion of the questions require higher cognitive thinking? Does each question have one clearly right answer and several plausible distracters? Have you avoided none of the above and all of the above answers? Has the use of negatively stated items been avoided as much as possible? Has all excessively technical language or jargon been eliminated from items? Is each item worded clearly and concisely? Is each response grammatically parallel with the question? Have you simplified the items as much as possible while providing all of the necessary information to answer questions correctly? Are the questions placed in an appropriate sequence? Are important words emphasized? (e.g., boldfaced not ) In online testing, it is generally best to use open-book/ open-web quizzes or tests rather than closedbook/ closed-web tests. 9. Allow students to collaborate on selected online tests. The benefits of collaborative testing have received significant recent attention in the literature. 10 Allowing students to team up to take a quiz or exam can have positive effects on anxiety levels, team development skills, peer cooperation, communication skills, self-esteem, confidence and assertiveness, cheating prevention, critical thinking skills, learning, and test performance. When students are allowed to collaborate on quizzes early in the course and later each take an individual final exam, their final exam scores are significantly better than if none of the quizzes or tests were collaborative. 11 How do you make an online test collaborative? Assign teams that meet in private online chat rooms, which are open to them during the assigned test time. Most course software, such as Blackboard and WebCT, allows you to record the chat, so you can view the transcript later for assessment if desired. Students can chat together about each of the questions and then move back and forth from the test to the chat room until they have each completed their exams. They may not agree on each answer, but they will benefit from the input of the others while working on the test. Assign students to groups of no more than five members and let them decide together how to answer each question. One member then submits the test for the entire group. Give students the objectives or questions beforehand and let them each other until they are satisfied with their preparation. They then complete the actual test individually. 10. Minimize opportunities for cheating. Preventing cheating is always a concern when testing student knowledge. Learners may do any of the following: Get someone else to take the test for them, give test questions and/or answers to someone who has not yet taken the test, ask other people for help in preparing for the test, or use Web or text resources while answering test questions. Horton discusses three ways to deal with cheating. 12 The fence method Table 2 Checklist for True-False Items Does each item ask only one question? Are there exceptions that could make both answers right (or wrong)? Have you asked students to correct false items? Table 3 Checklist for Matching Items Have you made one list longer than the other, to eliminate random guessing? Have you kept items parallel (i.e., terms with definitions, diseases with symptoms, etc.)? Have you designed the questions so that all choices are visible on one screen? 34 Journal of Adventist Education April/May 2003

5 tries to make cheating impossible. The threat method uses punishment to deter and punish cheating. The trust method attempts to reduce the incentives for cheating and appeals to students integrity. Specific ways to reduce cheating addressed earlier in this article include: Keep tests relatively unimportant in the overall grading scheme, give open-book or open-web tests, allow students to retake the quiz until they are satisfied with their grade, give timed tests, or give points for simply completing a test. 13 Conclusion In conclusion, just as in traditional education, tests and quizzes in the online environment play an important part in measuring course effectiveness and student learning. Internet-enhanced or Internet-based courses can be made more Simply knowing the teacher has a contingency plan in case of a technical problem will help to allay some of the anxieties many students experience when encountering a new learning environment. effective through the wise use of testing. The online instructor should view tests as effective learning aids and use a variety of measures in assigning grades. Class assignments and exams should emphasize higher-level cognitive functions, such as the development of portfolios, projects, and creative synthesis, in addition to traditional quizzes and tests. According to Bugbee, Although paper and pencil testing may never go the way of the dinosaurs, computerized testing is definitely in ascendancy, especially in distance education, certification, and licensure, and will probably eclipse paperand-pencil testing in the future. 14 Online assessment is an important part of the evolving educational teaching opportunities provided by the World Wide Web. Doing it well will enhance the learning of online students. Gail Rice, Ph.D., is a Professor at Loma Linda University (Loma Linda, California), with appointments in the School of Allied Health Professions and the School of Public Health. Dr. Rice is also responsible for campuswide faculty development and teaches several online courses using the university s Blackboard system. This article has been peer-reviewed. NOTES AND REFERENCES 1. E. Bicanich, et al., Internet-Based Testing: A Vision or Reality? T.H.E. Journal Online (September 1997); Alan C. Bugbee, Jr., The Equivalence of Paper-and-Pencil and Computer-Based Testing, Journal of Research on Computing in Education 28:3 (Spring 1996), pp ; D. Lloyd and J. G. Martin, The Introduction of Computer-Based Testing on an Engineering Technology Course, Assessment and Evaluation in Higher Education 21:1 (March 1996), pp ; Melody Alexander, James Bartlett, Allen Truell, and Karen Ouwenga, Testing in a Computer Technology Course: An Investigation of Equivalency in Performance Between Online and Paper and Pencil Methods. Journal of Career and Technical Education 18 (2001), pp Melody Alexander, in a study of 220 business students, found overall positive perceptions of online testing and no significant differences based on gender, age, or grade-point average. See Students Perceptions of Online Testing, Delta Pi Epsilon Journal 44:1 (2000), pp See also Bichinich, et. al. 3. To check out these programs, go to WebCT.com or Blackboard. com and try their testing programs. In addition, Quia.com has some options to develop learning aids, such as word searches, crossword puzzles, etc., that you can use in preparing students for the test. 4. See A Brief Overview of Assessing Learning in Online Classes by George Conway at 5. Robert Pike, Creative Training Techniques Handbook (Minneapolis: Lakewood Publ., 1994), p W. Horton, in Designing Web-Based Training (New York: John Wiley and Sons, 2000), p. 283, provides a helpful discussion of when to use each of four approaches to student retakes. 7. For a good review of test question development and cognitive levels, see Barbara Davis, Tools for Teaching (San Francisco: Jossey-Bass Company, 1993), chapters 30 and See the upcoming article in the Journal of Adventist Education on the topic of clueing answers in multiple-choice questions. 9. To view examples of these kinds of questions, go to /. 10. Antonio Russo and Susan Warren, Collaborative Test Taking, College Teaching 47:1 (Winter 1999), pp K. Ley, R. Hodges, and D. Young, Partner Testing, Research and Teaching in Developmental Education 12:1 (1995), pp William Horton, Web-Based Training (New York: John Wiley and Sons, Inc., 2000), pp Good Web sites with ideas on avoiding cheating in online classes can be found at and Bugbee, p Journal of Adventist Education April/May