Multiple Choice Questions Some Key Principles

Transcription

1 Multiple Chice Questins can be an efficient way t check students recall f factual knwledge and their ability t perfrm rutine prcedures. They ften wrk well fr frmative self-assessments and plling. Advantages f Multiple Chice Quizzes and Exams They encurage less guessing than true-false questins They can be scred quickly using tls like Quiz Generatr Students can receive immediate feedback Quiz data available with tls like Quiz Generatr can reveal trends and sticking pints Disadvantages f Multiple Chice Quizzes and Exams Effective quizzes and exams are time-cnsuming t cnstruct Prly written questins cannt accurately measure r reinfrce student learning Questins assessing higher-rder thinking can be difficult t write They are nt useful fr measuring students ability t Articulate explanatins Display thught prcesses Organize their thughts Generate riginal ideas Prvide unprmpted examples Best Practices fr Creating Multiple Chice Quizzes and Exams 1. Decide exactly what yu want t test (check yur test questins against yur learning bjectives) 2. Write questins with a stem and 3-5 pssible respnses ne is the best answer and the thers are distracters 3. Create distracters based up n cmmn miscnceptins and faulty thinking 4. Eliminate as many ambiguities as pssible 5. Write feedback fr yur distracters explaining why they're wrng 6. Write feedback fr the crrect answer explaining why it's right 7. Ask a clleague, frmer student r TA t take the quiz/exam and slicit their feedback abut the clarity cnsistency, accuracy, and rganizatin f the questins 8. Allw time fr revising the quiz/exam befre sharing it with students Ntes: While students may scre better n self-assessment questins (especially when they are allwed multiple attempts), ideal exam questins are answered crrectly by 60-65% f the test takers. Prviding feedback (explanatins) fr crrect and incrrect answers greatly increases the quiz r exam s value as learning tl. 1

2 Strategies fr Writing Effective Multiple Chice Questins Questins shuld cntain a stem and 3-5 pssible respnses ne is the best answer and the thers are distracters Questin Types 1. Remembering factual knwledge Write the fact as a statement Transfrm the statement int a questin Ask the student t select the apprpriate answer 2. Understanding cnceptual knwledge Which f the fllwing is an example f? 3. Applying prcedural knwledge: Prepare a shrt case study r example Write a stem that asks the student t demnstrate the use f the prcedural knwledge r slve a prblem 4. Analyzing cnceptual knwledge Present the student with a diagram and ask fr analysis fr example, Given the fllwing chart, which f the fllwing is mst likely t ccur next? 5. Evaluating prcedural knwledge Give the student a shrt case study Ask Which f the fllwing wuld have been a better plan f actin? Best Practices fr Writing Effective Multiple Chice Questins Use nly a single, clearly-defined prblem and include the main idea in the questin students shuld knw what the prblem is withut having t read the respnse ptins Write the crrect answer befre writing the distracters this ensures yu pay attentin t frmulating the ne clearly crrect answer Use the active vice Avid extreme wrds like "all" "always" and "never" Avid vague wrds r phrases like "usually," "typically" and "may be" Avid negative phrasing such as nt r except 2

3 Best Practices fr Writing Effective Multiple Chice Questins cntinued Dn't try t be t clever (e.g. include a reference t Shakespeare in a physics quiz (unless that s what yu are testing) Dn t use yur wn capabilities as a yardstick and frget the years f practice t that went int hning yur skills Dn t insert humr unless yu knw yur audience very well students might misinterpret a questin r answer that is meant t be funny Base each questin n curse learning bjectives, nt trivial infrmatin Avid prviding cues frm ne questin t anther; keep questins independent f ne anther Keep the length f ptins fairly cnsistent (preferably shrt) Cnsider alternatives t text nly questins (incrprate diagrams, charts, tables, figures, audi r vide) Cnsider the rle f the questins Recall: ability t recall r recgnize previusly learned (memrized) knwledge ranging frm specific facts t cmplete theries Interpretive skills: ability t utilize recalled knwledge t interpret r apply verbal, numeric r visual data, cause and effect relatinships Prblem slving: ability t utilize recalled knwledge and the interpretatin/applicatin f distinct criteria t reslve a prblem r situatin, make an apprpriate decisin and/r justify methds and prcedures Memry + applicatin: The ability t recall principles, rules, r facts in a real life cntext; place the cncept in a life situatin r cntext that requires the student t first recall the facts and then apply r transfer the applicatin f thse facts int a situatin Multiple Chice Questin Stems Typically, stems 1. Ask a questin 2. Give an incmplete statement 3. State an issue 4. Describe a situatin Best Practices fr Writing Stems Avid superfluus infrmatin Typically avid the use f persnal prnuns such as "yu" State stems in the psitive frm (negative statements are nt characteristic f nrmal thught prcesses) Use questins rather than incmplete statements Write stems that fully state the prblem and all qualificatins (t make sure that the stem presents a prblem, always include a verb in the statement) 3

4 Best Practices fr Writing Questin Respnses The best answer is the respnse the quiz writer (and experts) wuld cnsider the mst apprpriate answer The distracters (wrng answers) are lgical miscnceptins f the crrect answer The distracters shuld seem plausible t learners wh have partial, incmplete, r inapprpriate knwledge If there are nt three plausible distracters, then eliminate a 4 th ptin rather than include an implausible ptin Distracters shuld be mutually exclusive and nt verlapping Respnses shuld be parallel in cntent length and categry f infrmatin The grammatical structure f the respnses shuld be a lgical cnclusin t the situatin, questin, r statement presented in the stem Avid repetitive language within the respnses (wrds which are repeated in every respnse may be placed in the stem) Avid using nne f the abve as a respnse (this respnse des nt test what students knw, but nly that they can recgnize that the crrect answer is nt present). Avid all f the abve (students can recgnize that tw r mre respnses are crrect withut knwing the imprtance r crrectness f the remaining respnses) Respnses shuld be written in a lgical rder (e.g. chrnlgical, large t small, alphabetical if a single wrd, in rder f magnitude if numerals, in tempral sequence, r by length f respnse) Surces Davis, B. G. (1993). Tls fr teaching. San Francisc: Jssey-Bass. Ebel, R. L., & Frisbie, D. A. (1986). Essentials f educatinal measurement (4th ed.). Englewd Cliffs, NJ: Prentice-Hall. Haladyna, T. M., & Dwning, S. M. (1989). A taxnmy f multiple-chice item-writing rules. Applied Measurement in Educatin, 2(1), Osterhf, A. C. (1990). Classrm applicatins f educatinal measurement. Clumbus, OH: Merrill Publishing. Suskie, L. (2009). Assessing student learning: A cmmn sense guide. San Francisc: Jssey-Bass. Zimmerman, B. B., Sudweeks, R. R., Shelley, M.F., & Wd, B. (1990). Hw t prepare better tests: Guidelines fr university faculty. Prv, UT: Brigham Yung University Testing Services. 4

5 Checklist fr Multiple Chice Questins General Questins are aligned with curse learning bjectives Sme questins assess higher rder thinking Questins address learning bjectives, nt trivia There is feedback fr each respnse (feedback fr distracters explains why they're wrng, fr crrect respnses, why they are right) Questin Stems Questins cntain a single, clearly-defined prblem r main idea Stems are in questin frmat (rather than an incmplete statement) Stems state the prblem and all qualificatins (t make sure that the stem presents a prblem, always include a verb in the statement) Respnses and Distracters Clarity Each questin has 3-5 pssible respnses ne is the best answer and the thers are distracters Implausible distractrs have been eliminated (limit respnses t 3 rather than include an implausible distracter) Distracters are based upn cmmn miscnceptins and faulty thinking Distracters reveal partial, incmplete, r inapprpriate knwledge The length f respnses and distracters is fairly cnsistent (preferably shrt) Nne f the abve has nt been used as a respnse r distracter All f the abve has nt been used as a respnse r distracter Distracters are mutually exclusive and nt verlapping Respnses and distracters are parallel in cntent length and categry f infrmatin Questins and respnses are written in active vice Extreme wrds like "all" "always" and "never" have been avided Vague wrds r phrases like "usually," "typically" and "may be" have been avided Negative phrasing such as nt r except have been avided (if they must be used, capitalize the negative wrd/phrase, e.g. NOT) Repetitive language within the respnses has been eliminated (wrds which are repeated in every respnse may be placed in the stem) Respnses are written in a lgical rder (e.g. chrnlgical, large t small, alphabetical if a single wrd, in rder f magnitude if numerals, in tempral sequence, r by length f respnse) 5