14 RULES FOR WRITING MULTIPLE-CHOICE QUESTIONS 1. Use Plausible Distractors (wrong-response options) Only list plausible distractors, even if the number of options per question changes Write the options so they are homogeneous in content Use answers given in previous open-ended exams to provide realistic distractors 2. Use a Question Format Experts encourage multiple-choice items to be prepared as questions (rather than incomplete statements) Incomplete Statement Format: The capital of California is in Less Direct Question Format: In which of the following cities is the capital of California? More 3. Emphasize Higher-Level Thinking Use memory-plus application questions. These questions require students to recall principles, rules or facts in a real life context. The key to preparing memory-plus application questions is to place the concept in a life situation or context that requires the student to first recall the facts and then apply or transfer the application of those facts into a situation. Seek support from others who have experience writing higher-level thinking multiple-choice questions. EXAMPLES: Memory Only Example Which description best characterizes whole foods? a. orange juice b. toast Less c. bran cereal d. grapefruit Prepared by Timothy W. Bothell, Ph.D. Faculty Development Coordinator: Assessment of Student Learning 422-8194 tim_bothell@byu.edu
Memory-Plus Application Example Sally s breakfast this morning included one glass of orange juice (from concentrate), one slice of toast, a small bowl of bran cereal and a grapefruit. What whole food did Sally eat for breakfast? a. orange juice b. toast c. bran cereal d. grapefruit Memory-Plus Application Example Which one of the following best illustrates the law of diminishing returns? a. The demand for a farm product increased faster than the supply of the product. b. The population of a country increased faster than the means of subsistence. c. A machine decreased in utility as its parts became worn. d. A factory doubled its labor force and increased production by 50 percent. Ability to Interpret Cause-and-Effect Relationships Example Why does investing money in common stock protect against loss of assets during inflation? a. It pays higher rates of interest during inflation. b. It provides a steady but dependable income despite economic conditions. c. It is protected by the Federal Reserve System. d. It increases in value as the value of a business increases. Ability to Justify Methods and Procedures Example Why is adequate lighting necessary in a balanced aquarium? a. Fish need light to see their food. b. Fish take in oxygen in the dark. c. Plants expel carbon dioxide in the dark. d. Plants grow too rapidly in the dark. 2
3. Emphasize Higher-Level Thinking (continued) Faculty Comment: I had previously thought that these higher-level questions would require a lot more work than they do. I also enjoy being more creative as I cast the topic into a unique setting that requires my students not only to recall but also to apply concepts. However, what has surprised me most is how much more my students enjoy higherlevel assessment questions than recall questions. 4. Keep Option Lengths Similar Avoid making your correct answer the long or short answer 5. Balance the Placement of the Correct Answer Correct answers are usually the second and third option 6. Be Grammatically Correct Use simple, precise and unambiguous wording Students will be more likely to select the correct answer by finding the grammatically correct option 7. Avoid Clues to the Correct Answer Avoid answering one question in the test by giving the answer somewhere else in the test Have the test reviewed by someone who can find mistakes, clues, grammar and punctuation problems before you administer the exam to students Avoid extremes never, always, only Avoid nonsense words and unreasonable statements 8. Avoid Negative Questions 31 of 35 testing experts recommend avoiding negative questions Students may be able to find an incorrect answer without knowing the correct answer 3
9. Use Only One Correct Option (Or be sure the best option is clearly the best option) The item should include one and only one correct or clearly best answer With one correct answer, alternatives should be mutually exclusive and not overlapping Using MC with questions containing more than one right answer lowers discrimination between students 10. Give Clear Instructions Such as: Questions 1-10 are multiple-choice questions designed to assess your ability to remember or recall basic and foundational pieces of knowledge related to this course. Please read each question carefully before reading the answer options. When you have a clear idea of the question, find your answer and mark your selection on the answer sheet. Please do not make any marks on this exam. Questions 11 20 are multiple-choice questions designed to assess your ability to think critically about the subject. Please read each question carefully before reading the answer options. Be aware that some questions may seem to have more than one right answer, but you are to look for the one that makes the most sense and is the most correct. When you have a clear idea of the question, find your answer and mark your selection on the answer sheet. You may justify any answer you choose by writing your justification on the blank paper provided. 11. Use Only a Single, Clearly-Defined Problem and Include the Main Idea in the Question Students must know what the problem is without having to read the response options 12. Avoid the All the Above Option Students merely need to recognize two correct options to get the answer correct 13. Avoid the None of the Above Option You will never know if students know the correct answer 14. Don t Use MC Questions When Other Item Types Are More Appropriate limited distractors or assessing problem-solving and creativity 4
5