Description of the ICTS Standard Setting Process ILLINOIS CERTIFICATION TESTING SYSTEM (ICTS) 1

Transcription

1 Description of the ICTS Standard Setting Process 1

2 ICTS Test of Academic Proficiency Illinois students are required to pass the ICTS TAP as a prerequisite to entering an Illinois educator preparation program. Out-of-State candidates applying for licensure in Illinois must also pass the ICTS TAP. Those who have never taken the Basic Skills test and are seeking a subsequent certificate must also take the TAP. 2

3 ICTS Test of Academic Proficiency Assesses knowledge and skills in four domains: Reading Comprehension: 48 scorable items Language Arts: 48 scorable items Mathematics: 40 scorable items Writing: one constructed-response assignment Total: 136 scorable multiple-choice items and one constructed-response assignment Can take all subtests at once or individually 3

4 ICTS Test of Academic Proficiency Candidates have up to five hours to complete the four subtests or up to 2.5 hours to complete one subtest Administered on computer under standard conditions Those with documented special needs are given accommodations Score reports include total subtest scaled scores and objectivelevel diagnostic performance information Five attempt limit Must pass each subtest separately 4

5 Overview of Activities Part 1: Simulated Test Taking Activity Part 2: Round One Multiple-Choice Passing Score Judgments *Defining the Just Acceptably Qualified Candidate Part 3: Round One Constructed-Response Passing Score Judgments Part 4: Round Two Multiple-Choice Passing Score Judgments Part 5: Round Two Constructed-Response Passing Score Judgments 5

6 Overview of Activities Part 6: Round Three Multiple-Choice Passing Score Judgments Part 7: Round Three Constructed-Response Passing Score Judgments Part 8: Round Four Multiple-Choice Passing Score Judgments Part 9: Round Four Constructed-Response Passing Score Judgments Part 10: Conference Evaluation 6

7 Simulated Test Taking Activity Standard setting panelists participate in a simulated testtaking activity in order to experience the test items from the point of view of an examinee. 7

8 Orientation to the Just Acceptably Qualified Candidate Review of the Just Acceptably Qualified Candidate Definition 8

9 Round One Performance Level Judgments for Multiple-Choice Items Panelists rate every test item to judge how well the Just Acceptably Qualified Candidate would perform. They repeat this task for each of the TAP multiple-choice subtests: Reading Comprehension Language Arts Mathematics 9

10 Round One Performance Level Judgments for Constructed-Response Item Panelists review the constructed-response item on the Writing subtest, the scoring scale, sample examinee responses, and judge how well the Just Acceptably Qualified Candidate would perform. 10

11 Round Two Performance Level Judgments For Rounds Two and Three, the set of panelists is divided into two separate groups that work independently. In each group, panelists are given additional information and data and repeat their Round One performance level judgments for multiple choice and constructed-response items. Reading Language Arts Mathematics Writing 11

12 Round Three Performance Level Judgments In Round Three, previous item level judgments are used to produce a calculated passing score for each subtest for each panelist. In each of the two groups, panelists review the definition of the Just Acceptably Qualified Candidate and recommend the number of correct responses needed to pass each multiple choice subtest and the holistic score needed to pass the Writing subtest. 12

13 Round Four Performance Level Judgments For Round Four, the two separate groups are reconvened for a final set of judgments. Panelists review the results of the previous rounds and make a final recommendation regarding the number of correct responses needed to pass each multiple-choice-based subtest and the holistic score needed to pass the Writing subtest. 13

14 Final Outcomes The results of the standard setting conference will be presented to the Illinois State Educator Preparation and Licensure Board and the Illinois State Board of Education. 14