Guidelines for Creating Multiple-Choice Assessments

Transcription

1 Things to Remember When Creating a Multiple-Choice Assignment for Assessment 1. We assess program outcomes and MOs through their alignment to course outcomes, as demonstrated by the course map on the master course syllabus. Therefore, course outcomes, as opposed to MOs or program outcomes, are primarily tied to multiple-choice questions. However, for the assessment of General Education courses, in which specific skills that are not necessarily reflected in the course outcomes must be assessed, MOs can be directly connected to multiple-choice questions. 2. For statistical validity, it is important to have no less than three (5 is ideal) questions tied to each course outcome. 3. Each question should be linked to no more than two outcomes (one is ideal). 4. There shouldn t be more than 5 alternatives per multiple-choice question (a, b, c, d, e, for example) 5. There shouldn t be more than 100 questions on the assignment. 6. If an assignment is going to be given in multiple sections and the department does not want to use the same test in all sections (it wants to allow individual instructors to add additional questions, etc.), please remember that there will be only one key for the exam. This means the following: a. The keyed answer option always needs to be the same in all versions of the test. For example, regardless of what question 1 is, the correct answer has to be the same (A, for example). b. Likewise, the questions need to tie to the same course outcome. For example, regardless of what question 33 is, it must be tied to the same course outcome in all versions of the test (outcome 3, for example). 7. There are several measures a department can take if it is concerned about test security when using the same assessment across multiple sections, such as keeping exam questions in a locked cabinet in the department, numbering the exam copies and requesting them back from students before they leave the room after taking the exam, keeping the exam copies and the key in a different location, etc. The Process for Using a Multiple-Choice Assignment for Assessment 1. Create the assessment with the items above in mind. 2. To maintain consistency, all exams and answer keys should be saved using the following file name template: Course_Name of Assessment_Type of Document_Draft/Final_Semester & Year. First, list the course name and number, followed by the name of the assessment. Next, indicate the type of document being submitted: exam questions or answer key. Indicate if the file is a draft or the finalized version. Finally, for the date, include the semester and year that the course is assessed, as opposed to the current semester and year. For example: EGL1010_LibraryFinalExam_AnswerKey_Final_FA14 3. Submit the draft to the Dean s office by the due date. OPAIR verfall2013- Guidelines Multiple Choice Page 1

2 4. Review the feedback from the AAAC and revise, if necessary. 5. Identify a department point person for handling the assignment. For more information, see below. 6. Submit a copy of the final draft of the assignment, a completed template found on page 3 of this document which provides the answers to each question and the corresponding outcome for each question, and a copy of the completed course maps on page 5-7 (if the course will be used to assess multiple programs) to Dean s office by the deadline communicated by the department DAT. 7. Once the point person has submitted the test and the template(s) to S://Assessment, your division s assessment coach will make an electronic answer key for the assignment and upload it to the department s assessment folder on the S drive: S:\\Assessment\[Department Name]\Assessment Files\[Current Semester]. 8. As benchmark answer forms must be used for the exams, each department chair should ensure that the department has enough forms to cover the assessment. Additional forms can be ordered from Ed Bentley 9. Decide how the answer key should be distributed to the faculty giving the assignment. 10. Distribute the test key to faculty. a. Please remind faculty that students must fill in their student id on the Benchmark sheet in order for the data to be useable. It might be a good idea for instructors to bring copies of their Owl Link rosters, which contain student id numbers, to class on the day of the assessment in case students don t know their id numbers. ALL PGCC credit students have a 7 digit ID number. Most IDs begin with 0 (e.g., ). The scanner form has 10 columns for the student ID. It does not matter if the students places their ID on the left, middle, or right within the 10 columns as long as the student s COMPLETE ID number including the leading 0, if applicable, is inserted. Please tell students to enter ONLY their ID number and no other numbers in the additional columns. b. Likewise, please make sure the section ID number is filled out on the #20150 form. This needs to be the 6 digit Section ID for Assessment, NOT the 5-digit SYN number. The Section ID for Assessment can be found on the Owl Link roster. The #20150 form should be completed and scanned first, as it will automatically populate the student forms (#29240) with the correct course section ID. 11. Be sure that faculty have the Benchmark 3000 Quick Guide and know where to find the machines and the Benchmark 3000 scanner sheets for grading and saving the assignment. This information can be found on the S drive: S:\\Assessment\Guidelines and Forms 12. Have faculty administer the test and grade them using the electronic answer key. OPAIR verfall2013- Guidelines Multiple Choice Page 2

3 13. The faculty should save the file using the following naming convention: Course_Name of Assessment_Type of Document_Semester & Year_SectionID#. First, list the course name and number, followed by the name of the assessment. Next, indicate the type of document being submitted as Student Data. For the date, include the semester and year that the course is assessed. Finally, include the 6-digit section ID number for the course. This 6-digit ID number is located on your Owl Link class roster under Section ID for Assessment and should not be confused with the 5-digit SYN#. For example: EGL1010_LibraryFinalExam_StudentData_FA14_ Each faculty member should send their Benchmark file (the file should be an.apxt file, not an exported Excel file), which contains the scores, to the faculty point person when they have completed grading. 15. The faculty point person should upload the Benchmark files to the S drive: S:\\Assessment\[Department Name]\Assessment Files\[Current Semester] 16. The faculty point person should upload the Benchmark file from each instructor into the folder. 17. The point person should contact his/her assessment coach when all of the sections scores have been uploaded. 18. For more information on creating valid and reliable multiple choice questions, please refer to the Best Practices for Writing Multiple-Choice Questions document, which can be found on S:\\Assessment\Guidelines and Forms. OPAIR verfall2013- Guidelines Multiple Choice Page 3

4 Multiple-Choice Template Is this a new assessment or a reassessment? new assessment reassessment If a reassessment, have changes been made to the answer key or alignment since the initial assessment? yes no Department: Click here to enter text. Course: Click here to enter text. Name of Assessment: Click here to enter text. Course Outcomes*: Click here to enter text. *If this course is a General Education course with MOs that are not reflected in the course outcome, the MO (number and wording) can be entered in this column. Program Name*: Click here to enter text. Program Outcomes: Click here to enter text. * If this course is being used to assess more than one program, please make sure that a course map (see below) has been completed for each program. See pg Total points the assessment is worth (100, 200, etc.): Click here to enter text. 2. Are all of the questions worth the same amount (5 pts. each, etc.)? Yes No 3. If the answer to question 2 is yes, how many points is each question worth? Click here to enter text. 4. If the answer to question 2 is no, please fill out the third column in the chart below. Question Number Answer (A, B, C, D, E) Point Value for Each Question (If you answered no to question 2 above, please fill out the point value for each question) Corresponding Course Outcome Number (No more than 2 per question. One is preferable) OPAIR verfall2013- Guidelines Multiple Choice Page 4

5 Question Number Answer (A, B, C, D, E) Point Value for Each Question (If you answered no to question 2 above, please fill out the point value for each question) Corresponding Course Outcome Number (No more than 2 per question. One is preferable) OPAIR verfall2013- Guidelines Multiple Choice Page 5

6 Assessing Multiple Programs with a Single Assessment. If you are assessing multiple program outcomes with a single course, please make sure that you have filled out a course map (see below) for each program and submit these forms with the form above. Department: Click here to enter text. Program: Click here to enter text. Course: Click here to enter text. Course Outcome (Number and Outcome) Program Outcome (Number and Outcome) MO (Number and Outcome) Department: Click here to enter text. Program: Click here to enter text. Course: Click here to enter text. Course Outcome (Number and Outcome) Program Outcome (Number and Outcome) MO (Number and Outcome) OPAIR verfall2013- Guidelines Multiple Choice Page 6

7 OPAIR verfall2013- Guidelines Multiple Choice Page 7